Reward Shaping for Language Agents: From Format Checks to Trajectory Rewards

~13 min read

Designing a good reward function is the hardest part of RFT/ART-style training. The book's own GRPO reward functions show a practical pattern — layer cheap, deterministic checks — that generalizes to trajectory-level agent rewards.

Reward Shaping for Language Agents: From Format Checks to Trajectory Rewards is a Pro topic

Sign in, then upgrade to Pro or Power to unlock this topic and the full AI Engineering curriculum.

Key points

  • The book's own GRPO reward functions layer 4 separate checks (format-exact, format-approximate, answer, numbers) rather than one binary signal, giving smoother partial credit
  • The same layering principle extends to trajectory-level agent rewards: outcome rewards (sparse, fires at the end), process rewards (denser, scores intermediate steps)
  • Sparse outcome-only rewards give the model little information about WHICH steps in a long trajectory actually helped — process rewards help address this
  • Efficiency penalties discourage unnecessarily long or redundant trajectories, mirroring the book's 'match format' checks rewarding the right SHAPE, not just correct content
  • Prefer automatically-checkable, hard-to-game reward signals over subjective ones — the same reward-hacking risk from the sft-vs-rft tradeoffs subtopic applies directly here