Stage 4 — Reasoning Fine-Tuning & GRPO: Reinforcement Learning with Verifiable Rewards
~20 min read
For tasks with a single correct answer (math, logic), you don't need human preference data — correctness itself is the reward signal. This is Reinforcement Learning with Verifiable Rewards (RLVR), and GRPO (used by DeepSeek) is a popular technique for it.
Stage 4 — Reasoning Fine-Tuning & GRPO: Reinforcement Learning with Verifiable Rewards is a Pro topic
Sign in, then upgrade to Pro or Power to unlock this topic and the full AI Engineering curriculum.
Key points
- •Reasoning tasks (math, logic) have a single verifiable correct answer — no human preference judgment needed
- •This is Reinforcement Learning with Verifiable Rewards (RLVR) — correctness itself is the reward signal
- •GRPO (Group Relative Policy Optimization, from DeepSeek) generates a group of candidate answers and scores them relative to each other, avoiding the need for a separate reward model
- •GRPO's 'group relative' name comes from comparing each candidate's reward against the group's own average, not an absolute learned score
- •The full pipeline: random init → pre-training (language) → instruction FT (follow commands) → preference FT/RLHF (human alignment) → reasoning FT/GRPO (verifiable correctness)