When to Use SFT vs RFT: The Book's Decision Tree

The previous two subtopics established WHAT SFT and RFT each do; this subtopic covers this course's own decision tree for choosing between them, which it presents as 'a quick guide on which fine-tuning method to use based on your data and the nature of the task.'

The tree starts with one question: do you have labelled (ground-truth) data? If NO labeled data exists, the next question is whether the task is verifiable — can you automatically check whether an output is correct? If the task is NOT verifiable, this course routes you to RLHF (Reinforcement Learning from Human Feedback), since humans must provide preference signals when correctness can't be checked mechanically (e.g. 'which of these two response tones is more helpful' has no automatic checker — a human has to judge it). If the task IS verifiable, RFT works, because correctness can be automatically checked (a math answer, a passing unit test, a well-formed JSON output) — exactly the reward-function pattern from the previous subtopic.

If you DO have labeled data, the branch depends on how much of it you have. Large datasets point you to SFT — with enough labeled examples, directly training the model to match them is straightforward, stable, and effective. Tiny datasets require one more question: does reasoning (like Chain-of-Thought) help this task? If YES, this course recommends RFT even though you have (a small amount of) labeled data — a tiny dataset isn't enough to teach robust patterns via direct imitation, but it can still be used to verify outcomes for a reward function, letting the model explore its way to good reasoning despite having little labeled data to imitate directly. If reasoning does NOT help (the task doesn't benefit from multi-step deliberation — e.g. a simple lookup or classification), this course recommends SFT anyway, since a small direct-imitation dataset is enough for a task that doesn't need exploratory reasoning.

This course's own framing of the payoff: this decision tree helps you quickly identify the most efficient and reliable fine-tuning strategy for your use case. The practical value is that it converts a fuzzy 'should I use SFT or RFT' question into two concrete, answerable questions about your OWN data and task — do you have labels, and is the task verifiable/does it benefit from reasoning — rather than a judgment call based on which technique sounds more sophisticated.

# Implementing the book's decision tree exactly as a function --
# a direct, runnable version of its flowchart logic.

def choose_finetuning_method(has_labeled_data: bool, is_verifiable: bool = None,
                             dataset_size: str = None,   # "large" or "tiny"
                             reasoning_helps: bool = None) -> str:
    """Reproduces the book's SFT vs RFT vs RLHF decision tree."""
    if not has_labeled_data:
        if not is_verifiable:
            return "RLHF (not verifiable -> needs human preference signals)"
        return "RFT (verifiable -> correctness can be checked automatically)"
    # has_labeled_data == True
    if dataset_size == "large":
        return "SFT (large labeled dataset -> direct imitation works well)"
    # tiny dataset
    if reasoning_helps:
        return "RFT (tiny dataset, but reasoning/CoT helps -> explore via reward)"
    return "SFT (tiny dataset, reasoning doesn't help -> direct imitation is enough)"

scenarios = [
    {"has_labeled_data": False, "is_verifiable": False},                                   # -> RLHF
    {"has_labeled_data": False, "is_verifiable": True},                                    # -> RFT
    {"has_labeled_data": True, "dataset_size": "large"},                                  # -> SFT
    {"has_labeled_data": True, "dataset_size": "tiny", "reasoning_helps": True},         # -> RFT
    {"has_labeled_data": True, "dataset_size": "tiny", "reasoning_helps": False},        # -> SFT
]

for s in scenarios:
    print(f"{s} -> {choose_finetuning_method(**s)}")