Foundations of Generative AI & LLMs
Before a single line of Spring AI code, you need to know what actually happens when text goes into a model and text comes back out. Everything downstream — memory limits, RAG, embeddings, even why LLM output is non-deterministic — traces back to the ideas on this page.
Learning objectives
- Beginner: Understand that "the AI remembers what I said" is actually "my client resent everything I said" — this single reframe prevents most confusion later.
- Intermediate: Reason about token costs correctly: a 20-turn conversation resending full history costs roughly the sum of all prior turns' tokens on every single new call, not just the cost of the new message.
- Advanced: Use the static-vs-contextual embedding distinction to choose an embedding model appropriately for a RAG system where disambiguating polysemous terms (e.g. a domain glossary with overloaded terms) materially affects retrieval quality.
Artificial Intelligence is the broad field of building systems that perform tasks normally requiring human intelligence. Machine Learning is a subset of AI: instead of hand-coding rules, a system learns patterns from data. Deep Learning is a subset of ML using layered neural networks. Generative AI is a category of deep learning models trained to produce new content (text, images, audio) rather than just classify or predict a number. A Large Language Model is a generative AI model specialized in producing text — and it's the one Spring AI is built around.
| Layer | What it means |
|---|---|
| AI | Any system performing tasks that normally require human intelligence. |
| ML | AI systems that learn patterns from data instead of following hand-written rules. |
| DL | ML using multi-layer neural networks, capable of learning far more complex patterns. |
| Generative AI | DL models trained to generate new content, not just classify existing content. |
| LLM | Generative AI specialized in text — the category Spring AI wraps. |
◆ The problem
It's tempting to think of an LLM as "understanding" your question and composing a thoughtful answer the way a person would. Understanding what it's actually doing mechanically is what lets you predict where it will fail.
At its mechanical core, an LLM does one thing repeatedly: given everything so far, predict the single most statistically likely next token, append it, and repeat. A whole paragraph of coherent, seemingly reasoned text is the result of this one-token-at-a-time prediction loop running enough times — not a separate "reasoning" step layered on top (setting aside newer explicit reasoning/chain-of-thought techniques, which are still built from the same next-token mechanism underneath).
◆ Under the hood
This is precisely why LLMs can confidently produce wrong answers (hallucinate): the model isn't checking a fact against a database, it's generating the token sequence that's statistically most plausible given its training. A very plausible-sounding sentence and a true sentence are correlated but not the same thing to the underlying mechanism — which is the entire reason Module 16's evaluators and Module 09's RAG exist as engineering disciplines, not just nice-to-haves.
Computers only understand numbers, so before any text reaches the actual model, it's converted into tokens — and each distinct token is mapped to a numeric Token ID. A token isn't necessarily a word: it can be a whole common word, part of a word, a punctuation mark, or even a leading space attached to the next word.
Input: "playfulish" Tokens: ["play", "ful", "ish"] IDs: [1974, 2870, 895]
"playfulish" isn't a real word, but the model doesn't need it to be — it decomposes into three tokens it already has IDs for (play, a common suffix ful, another common suffix ish), which is exactly how LLMs stay flexible with typos, slang, and made-up words without ever needing a token for every possible string.
◆ Under the hood — token IDs are surprisingly literal
Token IDs are sensitive to details you might not expect: capitalization and leading whitespace change the ID entirely. The word play at the start of a sentence, play after a space, and Play with a capital P can all be three different token IDs to the same model — because the tokenizer was trained to treat "a space followed by a lowercase word" as its own frequently-occurring unit, distinct from that same word capitalized or without a leading space. You can verify this yourself with OpenAI's public tokenizer utility at platform.openai.com/tokenizer.
▲ Pitfall
Token count, not character or word count, is what determines cost and context-window usage (see §01.3 continued in Module 04). Estimating capacity by "roughly how many words fit" is unreliable — a technical document full of code, unusual identifiers, or non-English text can tokenize far less efficiently than plain English prose, using noticeably more tokens per character.
💻 Code example
Input: "playfulish" Tokens: ["play", "ful", "ish"] IDs: [1974, 2870, 895]
During training, a model builds its own fixed vocabulary: the finite set of tokens it knows, derived algorithmically from the most commonly occurring word pieces across billions of words of training text. This vocabulary is baked in at training time and doesn't change afterward — it's why different model families (and even different versions of the same family) can tokenize identical input text into a different number of tokens with different IDs; there's no universal token vocabulary shared across all LLMs.
◆ The problem
A token ID is just an arbitrary integer — 1974 for "play" tells you nothing about how "play" relates to "game" or "toy". For a model to reason about meaning, it needs a representation where similar meanings are numerically close to each other.
An embedding is a list of floating-point numbers (a vector) assigned to a piece of text such that texts with similar meaning end up positioned near each other in that high-dimensional space. Embeddings are computed by a dedicated embedding model (a different model from the chat model that generates responses), trained specifically so that semantic closeness becomes geometric closeness.
Semantically related words ("dog", "puppy", "cat") cluster together; unrelated concepts ("stock market", "inflation") cluster elsewhere — despite forming a different, unrelated cluster themselves.
Static vs. contextual embeddings
A static embedding assigns one fixed vector per token, regardless of context — the word "bank" gets the same vector in "river bank" and "bank account". Modern transformer-based models instead produce contextual embeddings, where a token's final vector is adjusted based on the surrounding tokens (via attention, §01.7) — so "bank" ends up with different effective representations in those two sentences. This distinction matters directly for Module 08, where the embedding model you choose for RAG determines how well this disambiguation works.
A transformer processes all tokens of a sequence in parallel, not one at a time in order — which means, on its own, it has no inherent sense of word order ("dog bites man" and "man bites dog" would look like the same unordered set of tokens). Positional embeddings solve this by adding a position-derived signal to each token's representation before processing, so the model can distinguish where each token sits in the sequence.
Attention is the mechanism that lets every token's representation be updated based on how relevant every other token in the input is to it. This is what resolves something like "The trophy didn't fit in the suitcase because it was too big" — the model has to weigh "trophy" against "suitcase" as the likely referent for "it", and attention is the mechanism that computes and applies those relevance weights.
◆ Under the hood
Attention computes, for each token, a weighted combination of every other token's representation — the weights (how much each other token "matters" to this one) are themselves learned during training, not hand-coded. This is what lets a single architecture handle both short-range relationships (adjacent words) and long-range ones (a pronoun referring back to a noun mentioned several sentences earlier) with the same mechanism.
◆ The problem
After a satisfying multi-turn chat, it's natural to think the model "remembers" you. It doesn't — and not knowing this leads directly to misunderstanding what Module 07's Chat Memory actually does.
A raw LLM API call is stateless: the model has no memory of any previous call. Every single request must include the entire conversation history the model needs to "remember," resent from scratch every time. What feels like a continuous conversation in ChatGPT or any Spring AI chat app is an illusion constructed by the client (or, in Spring AI's case, a ChatMemory advisor) resending the accumulated history alongside every new message — covered in full in Module 07.
✓ Quick recap
What does an LLM actually predict, one step at a time? The next token — not a word, not a full response; token-by-token generation is what produces the appearance of a full, reasoned answer. Why can the exact same word have different token IDs in different sentences? Because capitalization and leading whitespace are part of what the tokenizer treats as distinct units. What's the key difference between a static and a contextual embedding? A static embedding is one fixed vector per token regardless of context; a contextual embedding adjusts based on surrounding tokens via attention. Is an LLM API call stateful between requests? No — every request must resend the full context the model needs; the model itself retains nothing between calls.
Want a visual for this concept?
Generate a diagram tailored to “Foundations of Generative AI & LLMs” — the AI picks whichever visual (flowchart, comparison, sequence, etc.) best fits.
Sign in to generate a visual →