Fine-Tuning: When to Use It

Fine-tuning occupies the mirror-image quadrant from RAG: it's the right tool when the goal is changing the model's behavior, not expanding its knowledge. This course's takeaway is exactly this: use fine-tuning to change the structure (behavior) of the model rather than its knowledge.

This course's own worked example makes the target case concrete: an LLM might struggle to summarize the transcripts of company meetings because speakers use internal vocabulary in their discussions — this isn't fundamentally a MISSING-KNOWLEDGE problem (retrieval-augmenting a definitions glossary might genuinely help somewhat), but the deeper issue is that the model needs to consistently WRITE and PARSE that specific internal vocabulary and jargon fluently, throughout, as its natural mode of operation — that's a behavioral, structural adaptation, not a one-off lookup RAG can supply per query.

Other classic fine-tuning-shaped problems: consistently reproducing a very specific output format or tone across every response (a legal document generator that must always structure output a particular way), domain-specific reasoning patterns that differ from the model's general-purpose defaults (specialized medical or legal reasoning conventions), or a task where the necessary context would need to be re-explained in every single prompt if you tried to handle it via prompting alone (making the prompt itself impractically long and expensive on every call) — baking that context into the weights via fine-tuning removes the need to re-supply it every time.

The trade-off fine-tuning brings, beyond RAG or prompting: it requires an actual training run (compute cost, data curation effort, and time), and unlike RAG's near-instant document updates, updating a fine-tuned model's knowledge or behavior generally means re-running fine-tuning, not a lightweight document swap. This is exactly why the decision framework treats knowledge and behavior as two SEPARATE axes rather than one — reaching for fine-tuning when your actual gap is missing knowledge (RAG's territory) means paying this real cost for the wrong problem.

# Illustrating the target case: fine-tuning to consistently handle
# internal jargon/vocabulary as a natural writing style, not a
# per-query lookup — conceptually what a fine-tuning dataset for this
# task would look like.
fine_tuning_examples = [
    {
        "messages": [
            {"role": "user", "content": "Summarize: 'The ARR growth this quarter "
                "was driven by NRR expansion in our top-decile ICP segment.'"},
            {"role": "assistant", "content": "Revenue grew this quarter mainly "
                "because existing high-value customers spent more (net revenue retention), "
                "especially among our best-fit customer profile."},
        ]
    },
    # ...hundreds more examples teaching the model to fluently parse
    # and translate this company's specific internal vocabulary,
    # baked into the weights rather than re-explained every call
]

# In practice: upload these as a fine-tuning dataset via the provider's
# fine-tuning API (e.g. client.fine_tuning.jobs.create(...)), producing
# a model that handles this vocabulary as its DEFAULT behavior —
# no glossary needs to be re-supplied in every prompt afterward.