Probability Fundamentals: Events, Distributions and Joint/Conditional/Marginal Probability
~12 min read
The basic vocabulary of probability — events, distributions, and the three ways of asking 'how likely' (joint, conditional, marginal) — that every later ML idea builds on.
Probability Fundamentals: Events, Distributions and Joint/Conditional/Marginal Probability is a Pro topic
Sign in, then upgrade to Pro or Power to unlock this topic and the full AI Engineering curriculum.
Key points
- •Probability is a number from 0 (never) to 1 (always) that captures how likely an event is
- •A probability distribution lists every possible outcome with its probability, and they all sum to 1 — exactly how an LLM scores its whole vocabulary at each step
- •Joint probability P(A and B) = both things happening; marginal probability P(A) = just one thing, ignoring the other
- •Conditional probability P(B | A) = the chance of B given that A already happened — often different from B's plain probability
- •P(A and B) = P(A) x P(B | A) links all three — the exact formula behind how a language model scores a full sentence