Tracing LLM Calls: LangSmith, W&B Weave and Arize
~13 min read
Tracing captures the full story of each request — inputs, outputs, prompts, tool calls, latency and cost — across every step of a chain or agent. LangSmith, W&B Weave and Arize Phoenix are the common tools.
Tracing LLM Calls: LangSmith, W&B Weave and Arize is a Pro topic
Sign in, then upgrade to Pro or Power to unlock this topic and the full AI Engineering curriculum.
Key points
- •Tracing captures the full record of each request as a tree of spans — every LLM call, retrieval, and tool step with its exact inputs/outputs
- •It's essential for LLM apps because one request fans out into many steps (RAG stages, agent loops); you need to see where quality broke
- •Capture the RESOLVED prompt (after templating and context injection) — the literal text the model received is usually where bugs hide
- •LangSmith (chain/agent tracing + eval), W&B Weave (decorate-and-log + versioning), Arize Phoenix (open-source, OpenTelemetry-based) are the common tools
- •OpenTelemetry conventions let you instrument once and send anywhere; capture inputs/outputs deliberately with sampling, PII redaction, and retention limits