🚀
Production & Observability
Model optimization, serving (vLLM), evaluation (G-Eval), and telemetry.
12345
Model Optimization: Compression & KV Cache
advanced~10hLearn quantization, pruning, distillation, and the size and structure of KV Caches.
LLM Evaluation: Rubrics & Judges
advanced~8hLearn evaluation frameworks: G-Eval, Arena-as-Judge, Multi-turn evaluation, and Red Teaming.
Model Deployment: vLLM & LitServe
advanced~8hLearn deployment requirements, PagedAttention, Continuous Batching, and serve models via vLLM and LitServe.
LLM Observability: Metrics & Telemetry
advanced~8hLearn observability requirements, key performance metrics (TPS, TTFT), and integrate telemetry tools.
AI System Architecture Patterns
advanced~8hCapstone view of production AI systems: how context construction, guardrails, model routers/gateways, semantic caching, agent orchestration, observability, and user feedback loops fit together into one coherent, operable architecture. Grounded in Chip Huyen's AI Engineering Ch.10 framework.