Spring AI Interview Questions

A token is a chunk of text (roughly 3-4 characters in English, not always a whole word) that the model reads and generates one at a time. Providers bill per token and every model has a maximum context window measured in tokens, so a long conversation history or a huge retrieved document can silently hit that ceiling or blow up your bill well before you run out of 'words'.

An LLM generates the statistically most plausible next token given its training data and the prompt — it has no built-in fact-checking step or notion of 'I don't know', so a plausible-sounding but false answer is generated with the same confidence as a true one. This is why production systems pair LLMs with retrieval (RAG) or tool calling for anything factual, rather than trusting the model's raw internal knowledge.

A starter dependency for the specific provider (e.g. `spring-ai-openai-spring-boot-starter`) plus an API key set via `application.properties` or an environment variable — Spring Boot's auto-configuration then wires a `ChatModel` bean automatically from that starter being on the classpath, which `ChatClient.Builder` uses under the hood.

`ChatClient` gives one consistent fluent API (`.prompt().user(...).call()`) across OpenAI, Anthropic, Ollama, and others, so switching providers or supporting multiple providers in one app doesn't mean rewriting business logic — only the underlying `ChatModel` bean configuration changes.

The system message sets persistent behavior/persona/constraints for the whole conversation (e.g. 'You are a customer support agent, only answer questions about billing') and is typically not shown to the end user. The user message is the actual per-turn input from whoever is chatting — mixing instructions into user messages instead of the system message makes them easy for a user to override or ignore in later turns.

Hardcoding prompt text inline everywhere makes it impossible to consistently update wording, test variations, or reason about exactly what's being sent. A `PromptTemplate` with named placeholders keeps the prompt's structure in one place, separate from the runtime values being substituted in, so a copy change doesn't require hunting through business logic.

Temperature controls how much randomness is injected when the model picks its next token — near 0 makes it almost always pick the highest-probability token (deterministic, consistent output), while higher values increase variety and creativity at the cost of predictability. Structured extraction or classification tasks want temperature near 0 for consistent, repeatable results; creative writing wants it higher.

A full LLM response can take several seconds to generate; streaming sends tokens to the client as they're produced, so the UI can render them progressively (like ChatGPT's typing effect) instead of showing a blank screen until the entire response is ready — this dramatically improves perceived latency even though total generation time is unchanged.

It appends formatting instructions to the prompt describing the exact JSON schema derived from your target Java record/class, asking the model to respond only in that shape, then parses the response back into that class. It's still fundamentally the model choosing to comply with instructions in the prompt, not a hard guarantee — which is why validating/retrying on parse failure is still necessary in production.

The model can still occasionally return malformed JSON, extra prose around the JSON, or a schema that's close-but-not-quite what you asked for, so code parsing the response needs defensive handling (retry with a stricter prompt, or a repair step) rather than assuming the parse will always succeed.

An Advisor is an interceptor that wraps a `ChatClient` call, letting you inject cross-cutting behavior (logging, adding conversation memory, retrieval augmentation) around every request/response without repeating that logic at every call site — similar in spirit to a servlet filter, but for AI calls specifically.

Each advisor handles one concern (e.g. one for memory, one for RAG retrieval, one for logging) so they can be composed and reordered per use case, and a new cross-cutting concern can be added as its own advisor without touching the existing ones — the same single-responsibility reasoning that applies to servlet filters or middleware chains.

Each API call to the model is stateless — the model has no persistent memory between calls. What looks like 'remembering' is the application resending the entire prior conversation history as part of the new prompt every single time, so the model is just reading the full transcript fresh on each turn.

Every message in history counts against the model's token limit and increases cost per call, so an unbounded conversation eventually either hits the context window or gets prohibitively expensive. Common mitigations are a sliding window (keep only the last N messages) or summarizing older turns into a compact summary that's kept instead of the full text.

An embedding is a list of numbers (a vector) that represents a piece of text's meaning in a high-dimensional space, produced by a specialized embedding model. Texts with similar meaning end up as vectors that are close together (by cosine similarity or distance), which is exactly what lets a vector database find 'similar' documents to a query without any exact keyword match.

SQL comparisons are exact or pattern-based (equality, LIKE) — they have no concept of meaning, so a query for 'cheap laptop' would never match a document saying 'affordable notebook' via SQL. A vector database instead compares embedding vectors using similarity math, which captures meaning rather than literal text overlap.

An LLM's knowledge is frozen at training time and it can't answer accurately about your private/proprietary/recent documents it never saw. RAG retrieves the most relevant chunks from your own document store at query time (via vector similarity search) and injects them into the prompt as context, so the model answers grounded in real, current, specific data instead of guessing from stale training knowledge.

Chunks that are too large dilute the embedding's specificity (a paragraph about five different subtopics doesn't closely match any single specific query) while chunks that are too small lose surrounding context needed to make sense of the excerpt on its own. Getting chunk size and overlap right is often the highest-leverage tuning knob in a RAG pipeline, more so than which embedding model you pick.

A normal cache only hits if the new request is byte-for-byte identical to a previous one. Semantic caching embeds the incoming query and checks for high similarity against previously cached queries, so a rephrased question ('What's Java's garbage collector?' vs 'How does GC work in Java?') can still hit the cache even though the literal text differs.

Two queries that are similar in wording but meaningfully different in intent (e.g. 'cancel my subscription' vs 'pause my subscription') could be treated as a cache hit and served the wrong cached answer, which is worse than a cache miss — a slightly stricter threshold trades some cache-hit rate for correctness.

Raw documents (PDFs, HTML, Word docs) need to be parsed into clean text, split into appropriately-sized chunks, and have metadata attached (source, page number) before embedding — skipping this means embedding messy formatting artifacts or chunks with no traceability back to their source, which hurts both retrieval quality and the ability to cite where an answer came from.

Without de-duplication or an update strategy, you either accumulate duplicate stale chunks alongside the fresh ones (degrading retrieval quality since outdated info can still be retrieved) or you have to fully wipe and re-embed everything on every change, which is wasteful — most pipelines track a content hash per chunk to only re-embed what actually changed.

No — the model only ever outputs structured text saying 'call function X with these arguments.' Your application code intercepts that, actually executes the real function (a database query, an API call), and sends the result back to the model as a new message, which the model then uses to compose its final answer. The model never directly executes anything.

It grounds the model's response in real, live, verifiable data (current stock price, actual database record) fetched by deterministic code you control, rather than the model guessing from static training data that may be outdated or simply wrong — it turns the LLM from an all-knowing oracle into an orchestrator that decides WHEN to fetch real data and how to phrase the result.

Without a standard, every AI app has to write custom integration code for every external tool/data source it wants to use. MCP standardizes the interface between an AI application (the 'client') and any tool/data provider (the 'server'), so a tool built once as an MCP server can be plugged into any MCP-compatible AI client without bespoke integration work each time.

Tools (callable functions the model can invoke), Resources (readable data/context the client can pull in, like files or API responses), and Prompts (reusable prompt templates the server provides) — together these let a server expose a full slice of capability, not just a single function call.

Sampling lets an MCP SERVER ask the CLIENT's LLM to generate a completion on the server's behalf — inverting the usual direction where the client always initiates. This lets a server perform its own AI-driven logic (like summarizing something internally) while borrowing the client's already-configured model access rather than needing its own API key.

Elicitation lets an MCP server pause mid-operation and ask the client to prompt the human user for additional input (e.g. missing information needed to complete a booking) rather than failing outright or guessing. This turns a hard stop into an interactive back-and-forth without the server needing its own separate UI.

A simple tool-calling bot does one round: user asks, model optionally calls one tool, model answers. An agent runs a loop — plan, act (call a tool), observe the result, and decide whether to act again or produce a final answer — potentially chaining many tool calls and reasoning steps autonomously until it decides the task is complete.

Without a hard cap, a model that gets stuck in a reasoning loop (repeatedly deciding 'I need more info' without converging) can call tools indefinitely, burning cost and time with no forward progress — a max-iterations or max-cost limit forces the loop to terminate and fall back to a 'couldn't complete this' response instead of running forever.

LLM output is non-deterministic and phrased differently each time even when factually correct, so exact string matching almost always fails even for a 'correct' answer. Evaluators instead use techniques like using a second LLM call to judge whether the output meets criteria (relevance, factual grounding, absence of the actual answer inside the prompt/leakage), producing a score rather than a pass/fail on exact text.

It catches cases where the retrieved context and the generated answer have drifted apart — the model answered something plausible-sounding but not actually grounded in (or even contradicting) the documents that were retrieved for it, which defeats the entire point of doing retrieval in the first place.

AI calls have highly variable latency and cost per request (unlike a fixed-cost DB query), can silently fail in 'soft' ways (a technically-200-OK response that's actually a bad hallucination), and their behavior can drift as providers update underlying models — without tracing token usage, latency, and cache-hit rate per call, none of that is visible until a user complains.

A single trace spanning embedding-the-query, vector search, and the final LLM call lets you pinpoint exactly where time or cost is going and, more importantly, lets you see WHAT was actually retrieved when debugging a bad answer — logging only the final answer gives you no way to tell if retrieval or generation was the actual point of failure.

Different models have different cost/latency/quality trade-offs — a cheap, fast model might be fine for simple classification while a more expensive frontier model is reserved for complex reasoning, and having a fallback provider protects against one provider's outage taking down your whole AI feature. Spring AI's abstraction over `ChatModel` makes this routing swap relatively mechanical rather than a rewrite.

A committed secret is permanently in git history even if removed later, is visible to anyone with repo access, and if the repo is ever public or breached, the key can be used by anyone to rack up charges on your account. Keys belong in environment variables or a secrets manager (Vault, AWS Secrets Manager) injected at runtime, never in version-controlled config files.

The audio is sent to a specialized speech-to-text model (e.g. Whisper), which is architecturally different from a chat LLM — it's trained specifically to map an audio waveform to a text transcript, not to reason or converse. Spring AI wraps this behind the same consistent abstraction pattern as `ChatModel`, so calling it fits the same mental model as a chat call.

Text generation and image generation are fundamentally different model architectures (autoregressive text transformers vs diffusion-based image models), so 'chat' models that mention generating an image are usually actually calling out to a separate image-generation model under the hood via a tool call — Spring AI's `ImageModel` abstraction reflects that these are genuinely separate capabilities, not one unified model doing everything.

Each of those adds its own model call or vector search — retrieval is a search call, tool calling can trigger multiple back-and-forth model round-trips, and memory means resending growing conversation history every time — so an agent doing all three per request can easily rack up 3-5x the latency/cost of a single plain chat call if none of it is cached or bounded.

Unlike a database, an external LLM API call is a third-party dependency you don't control and it WILL occasionally be slow or unavailable — a production feature needs a timeout, a retry with backoff, and ideally a degraded fallback response (or a fallback provider) rather than letting the whole user-facing feature hang or 500 whenever the provider has a bad moment.