Embedding Dimensions: What 768 vs 1536 Means and the Tradeoffs

You'll constantly see embedding models advertised by a specific number — 'this model produces 384-dimensional embeddings,' 'text-embedding-3-small uses 1536 dimensions.' This subtopic is about what that number actually means and why it's not simply 'bigger is always better.'

A dimension is just one entry in the embedding's vector — recall from the first subtopic that an embedding is a fixed-length list of numbers, like [0.021, -0.384, 0.157, ...]. A '768-dimensional embedding' means that list has exactly 768 numbers in it, every single time, for every piece of text that model embeds, whether it's one word or three paragraphs. Each dimension doesn't correspond to one clean, human-readable concept like 'fruitiness' or 'formality' — that's a common misconception. The dimensions instead form a coordinate system that the model learned during training, where meaningful DIRECTIONS and DISTANCES emerge across many dimensions working together, not any single dimension in isolation.

Common dimension counts you'll encounter: 384 (many small, fast sentence-transformer models, like all-MiniLM-L6-v2), 768 (BERT-base and many mid-sized models), and 1536 or 3072 (OpenAI's text-embedding-3-small and text-embedding-3-large, respectively). Generally, MORE dimensions give the model more 'room' to represent fine-grained distinctions in meaning — two texts that are subtly different might end up at genuinely different positions in a 1536-dimensional space, but collapse to nearly the same position in a cramped 128-dimensional one, simply because there isn't enough room to spread out every nuance.

But more dimensions aren't free. Storage scales directly with dimension count — a million 1536-dimensional embeddings take 4x the disk space of a million 384-dimensional ones (at 4 bytes per number: 1536 x 4 bytes = 6KB per embedding versus 384 x 4 bytes = 1.5KB). Search gets slower too, since computing distance between two vectors is a sum over every dimension — more dimensions means more arithmetic per comparison, and this multiplies across potentially millions of stored vectors. This is exactly why choosing an embedding model is a real engineering tradeoff, not just 'pick the biggest number': a 384-dimensional model might be the better choice for a latency-sensitive app with millions of documents, while 1536+ dimensions might be worth the cost when retrieval QUALITY on subtle distinctions matters more than raw speed. (Some newer models, like OpenAI's v3 embeddings, even support 'Matryoshka' truncation — letting you cut a large embedding down to fewer dimensions after the fact, trading some quality for less storage, without re-embedding everything from scratch.)

# Demonstrating the storage-cost tradeoff directly, and simulating
# why higher dimensions can separate subtly-different meanings better.

def storage_bytes(num_vectors: int, dimensions: int, bytes_per_float: int = 4) -> int:
    return num_vectors * dimensions * bytes_per_float

for dims in (384, 768, 1536, 3072):
    mb = storage_bytes(1_000_000, dims) / 1e6
    print(f"{dims:5d} dimensions -> {mb:8.1f} MB for 1M embeddings")

import random
random.seed(0)

def random_embedding(dims: int) -> list[float]:
    return [random.gauss(0, 1) for _ in range(dims)]

def cosine_similarity(a, b):
    dot = sum(ai * bi for ai, bi in zip(a, b))
    norm_a = sum(ai ** 2 for ai in a) ** 0.5
    norm_b = sum(bi ** 2 for bi in b) ** 0.5
    return dot / (norm_a * norm_b)

# Simulate two 'subtly different' embeddings by nudging one slightly --
# more dimensions give more room for that nudge to register as distinct
for dims in (16, 1536):
    base = random_embedding(dims)
    nudged = [v + random.gauss(0, 0.05) for v in base]  # small, subtle change
    sim = cosine_similarity(base, nudged)
    print(f"{dims:5d} dims -> similarity after a subtle nudge: {sim:.6f}")