intermediate~12h

Training Stages: Pretraining, SFT, RLHF, GRPO

Deconstruct the four stages of building a production reasoning LLM from scratch.

Subtopics

Exercises

Projects

Quiz Qs

Flashcards

▶📚 Prerequisites(2)

🎓 Learning objectives

•Contrast Pre-training and Supervised Fine-Tuning (SFT)
•Describe RLHF using Reward Models and PPO optimization
•Explain DeepSeek's Group Relative Policy Optimization (GRPO) and how it eliminates the critic model

Stage 1 — Pre-training: Teaching the Basics of Language

Stage 2 — Instruction Fine-Tuning: Teaching the Model to Follow Prompts

Stage 3 — Preference Fine-Tuning & RLHF: Aligning with Human Preferences

Stage 4 — Reasoning Fine-Tuning & GRPO: Reinforcement Learning with Verifiable Rewards

reinforcement-learningllm-foundations

grpo-reasoning →finetuning-peft →