The Verbalized Sampling Prompting Pattern
~15 min read
Instead of asking for one answer, verbalized sampling asks the LLM to generate several responses with their corresponding probabilities. This simple reframing — from requesting an instance to requesting a distribution — is enough to bypass mode collapse.
The Verbalized Sampling Prompting Pattern is a Pro topic
Sign in, then upgrade to Pro or Power to unlock this topic and the full AI Engineering curriculum.
Key points
- •VS is training-free — it's purely a prompting technique, no fine-tuning or weight changes involved
- •The prompt shifts from requesting an instance ('tell me a joke') to requesting a distribution ('5 responses with probabilities')
- •This shift forces the model to reach past its single most-reinforced answer into its broader learned distribution
- •The output format becomes a structured list of candidate responses with self-assigned probabilities
- •You can then pick the top one, sample weighted by probability, or surface multiple options to the user