Using VLMs in Practice: API Usage and Prompt Design for Images
~13 min read
Calling a vision-capable API follows the same request/response shape from api-basics, with images sent as base64 or URLs alongside text — and prompt design for images has its own specific best practices.
Using VLMs in Practice: API Usage and Prompt Design for Images is a Pro topic
Sign in, then upgrade to Pro or Power to unlock this topic and the full AI Engineering curriculum.
Key points
- •Vision API requests use the same message-list request/response shape as text-only chat APIs (from api-basics), with images added as an additional content type
- •Images are sent either as a publicly accessible URL or as base64-encoded data embedded in the request — the response is always ordinary generated text either way
- •Prompt design for images benefits from specificity (targeting VQA-style precision) and explicit image references when multiple images are in one request
- •Higher image resolution generally improves accuracy on document/OCR-style tasks — heavily compressed or low-res images can make fine text illegible to the model
- •Images are typically billed as additional tokens scaling with resolution, so image-heavy production features need their own cost estimation, distinct from text-only requests