How GRPO Works: Groups, Rewards, and the GRPO Loss
~15 min read
GRPO generates multiple candidate responses per prompt (a 'group'), scores each with reward functions, and uses those aggregated rewards to compute gradients that improve the model's reasoning over time.
How GRPO Works: Groups, Rewards, and the GRPO Loss is a Pro topic
Sign in, then upgrade to Pro or Power to unlock this topic and the full AI Engineering curriculum.
Key points
- •GRPO generates MULTIPLE candidate responses per prompt using a sampling engine — this is the 'group' in the name
- •Each candidate is scored by deterministic reward functions, and rewards are aggregated into a per-candidate score
- •'Relative' means scoring each candidate against its own group's rewards, not an absolute externally-calibrated score
- •This group-relative scoring is what lets GRPO work without a separately-trained reward model, unlike PPO/RLHF
- •A GRPO loss function turns these rewards into gradients that update the model, reinforcing better-scoring candidates over training