What Is a Large Language Model (LLM)? — AI Engineering Explained

A Large Language Model (LLM) is a neural network with billions of parameters trained on massive text corpora to predict the next token given a sequence of previous tokens. The core architecture is the Transformer — a stack of attention layers that learn relationships between tokens at any distance. LLMs like GPT-4, Claude, and Gemini are not retrieving stored answers — they compute probability distributions over vocabularies, token by token, generating output one piece at a time.

Transformer architecture: input tokens are embedded into vectors (dim 512–4096), then passed through N layers of Multi-Head Self-Attention (MHSA) + Feed-Forward Networks (FFN). MHSA computes: Attention(Q,K,V) = softmax(QKᵀ / sqrt(d_k)) × V. This operation is O(n²) in sequence length — the fundamental scaling bottleneck.

Training: next-token prediction on a corpus (15 trillion tokens for Llama 3). Loss = cross-entropy between predicted and actual next token. Gradient descent updates all ~70B parameters per step. Training a frontier model costs $50–100M in compute.

Inference: autoregressive — one token per forward pass. A 4K-token response = 4K forward passes. Each pass through a 70B model requires reading ~140GB of weights from GPU memory — inference latency is dominated by memory bandwidth, not raw compute.

Mixture of Experts (MoE): instead of one dense FFN per layer, MoE uses N expert FFNs and routes each token to K of them (K=2 of N=8 is common). GPT-4 is believed to be a ~1.8T parameter MoE — only ~200B parameters activate per token, giving frontier capability at dense-200B inference cost.

import anthropic
client = anthropic.Anthropic()

response = client.messages.create(
    model="claude-opus-4-5",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Explain attention in 2 sentences"}]
)
print(response.content[0].text)

# Cost: 1024 output tokens × $15/1M = $0.015 per call