MongoDB — Indexing, Replica Sets & Sharding
A query without the right index doesn't error — it just quietly scans your whole collection until the collection is too big for that to stay quiet. This chapter is about catching that before production does, plus the two mechanisms — replica sets and sharding — that let one MongoDB deployment become many machines pretending to be one.
Learning objectives
- Choose the right index type — single-field, compound, multikey, or text — for a given query shape.
- Read explain("executionStats") to tell a collection scan from an index scan, and judge whether an index is selective enough.
- Explain the role of the oplog in replica-set replication and why it's the reason standalone MongoDB can't run transactions.
- Choose a shard key that avoids a write hotspot, and explain why a monotonically increasing key creates one.
| Index type | Created on | Notes |
|---|---|---|
| Single-field | One field, ascending or descending | The default _id index is always present and can't be dropped |
| Compound | Multiple fields, each with its own sort direction | Field order matters — see below |
| Multikey | Automatic, whenever an indexed field's value is an array | One index entry is created per array element |
| Text | A field (or fields) marked for full-text search | Only one text index is allowed per collection |
db.orders.createIndex({ customerId: 1 }); // single-field db.orders.createIndex({ customerId: 1, createdAt: -1 }); // compound db.orders.createIndex({ "items.sku": 1 }); // multikey — items is an array db.products.createIndex({ description: "text" }); // text
A compound index on { customerId: 1, createdAt: -1 } can also serve queries that only filter on customerId (the leftmost prefix), the same prefix rule relational compound indexes follow — but it cannot efficiently serve a query that filters on createdAt alone, skipping customerId. MongoDB's own documented guidance for ordering compound index fields is commonly summarized as the ESR rule: Equality fields first, then Sort fields, then Range fields — put fields you filter with $eq leftmost, fields you sort by next, and fields you filter with $gt/$lt/$in last.
▲ Common mistake
A compound index on two array fields (e.g. { "items.sku": 1, "tags": 1 } where both are arrays) is disallowed outright — MongoDB rejects the createIndex call. A multikey index can only multiply out one array field per index; if you need to query on two different array fields efficiently, you need two separate indexes, not one compound one.
💻 Code example
db.orders.createIndex({ customerId: 1 }); // single-field db.orders.createIndex({ customerId: 1, createdAt: -1 }); // compound db.orders.createIndex({ "items.sku": 1 }); // multikey — items is an array db.products.createIndex({ description: "text" }); // text
Creating an index doesn't guarantee MongoDB uses it for a given query — and a query with no matching index doesn't error, it just silently falls back to a full collection scan, quietly getting slower as the collection grows until someone notices in production. explain("executionStats") is how you check, before that happens.
db.orders.find({ customerId: "cust_4521" }).explain("executionStats"); // no useful index — full collection scan: // "winningPlan": { "stage": "COLLSCAN" } // "totalDocsExamined": 480000, "nReturned": 12 // after db.orders.createIndex({ customerId: 1 }): // "winningPlan": { "stage": "FETCH", "inputStage": { "stage": "IXSCAN" } } // "totalDocsExamined": 12, "nReturned": 12
The single number worth watching first: totalDocsExamined versus nReturned. A COLLSCAN examines (close to) every document in the collection regardless of how few actually match. An IXSCAN should bring totalDocsExamined down close to nReturned — the index let MongoDB go almost straight to the matching documents instead of checking every one. A large gap between the two, even with an IXSCAN present, usually means the index exists but isn't selective enough for this specific query (Chapter 05 goes much deeper on reading explain() as a real debugging workflow, including the fully index-only "covered query" case where totalDocsExamined drops to zero).
A replica set is one primary node plus N secondary nodes holding copies of the same data. All writes go to the primary; the primary records every write, in order, to a special capped collection called the oplog (operations log), and secondaries continuously replicate by reading and replaying that oplog. This single fact — that MongoDB's replication mechanism is the oplog — is why a genuinely standalone mongod (no replica set at all, not even a one-node one) cannot run multi-document transactions: MongoDB's transaction snapshot machinery is built directly on oplog infrastructure that a standalone instance simply doesn't have (Transaction Mastery, Chapter 14 covers the practical fallout of this — for local development you need at least a single-node replica set, not a true standalone).
If the primary becomes unreachable, the remaining secondaries hold an election and promote one of themselves to primary — but only if a majority of the replica set's voting members can still reach each other. This is the same quorum requirement NoSQL Foundations, Chapter 02 covers for split-brain prevention in general: it's exactly why production replica sets are deployed with an odd number of members (3, 5, 7) — a 2-vs-2 split from a 4-member set can leave neither half with a majority, meaning the whole set goes read-only until the network heals, which is strictly worse than the same fault hitting an odd-sized set.
◆ Under the hood — why replication lag is worth watching, mechanically
The oplog is a capped collection — fixed size, oldest entries automatically dropped as new ones are written. If a secondary falls behind the primary by more than the oplog's total time window (because of a slow network link, an overloaded secondary, or a long maintenance pause), it can fall off the oplog entirely: the write it needs next has already been evicted, and the secondary can no longer catch up by simply replaying more entries — it needs a full resync from another member instead. NoSQL Production Best Practices (NoSQL Foundations, Chapter 04) already named replication lag as a leading indicator worth alerting on; this is the concrete mechanical reason why letting it grow unbounded is dangerous specifically in MongoDB, not just "stale reads get worse."
Why this matters day to day, even if you never touch replica-set configuration directly: read preference. By default, all reads also go to the primary; setting a driver's read preference to secondaryPreferred spreads read load across secondaries, but at the cost of potentially reading data that's a few oplog entries behind the primary — a direct, concrete instance of the CAP/replication trade-off NoSQL Foundations, Chapter 01 introduces in the abstract.
◆ The problem
NoSQL Foundations, Chapter 02 told the story of a leaderboard partitioned by date — every write on launch day landed on one partition while the rest of the cluster sat idle, because dates only ever increase. Sharding a MongoDB collection makes the exact same mistake possible, mechanically, if you pick the wrong shard key.
A sharded MongoDB cluster splits one logical collection's data across multiple shards (each shard is itself typically a replica set), routed by a query router process called mongos — your application talks to mongos, never to a shard directly. Every document is assigned to a shard based on the value of its shard key, a field (or compound set of fields) you choose once, at the time you shard the collection, and generally cannot change afterward.
sh.shardCollection("shop.orders", { customerId: "hashed" });
If a query includes the shard key in its filter, mongos can route it straight to the one shard that owns that value — fast, targeted. If a query omits the shard key entirely, mongos has no way to know which shard holds the answer, so it broadcasts the query to every shard and merges the results — a scatter-gather query, which scales far worse as shard count grows.
▲ Common mistake — sharding on a monotonically increasing key
MongoDB's default _id (an ObjectId) embeds a four-byte creation timestamp as its leading bytes, so _id values created close together in time sort close together. Sharding directly on _id (or on any timestamp field) with range partitioning means every new insert — always the newest timestamp — lands on the same shard: the one that owns the highest range. That single shard absorbs 100% of write traffic while the rest of the cluster does nothing, the identical hotspot from the leaderboard story above. The standard fix is hashed sharding — hashing the shard key's value before routing, so consecutive inserts scatter evenly across shards, at the cost of losing the ability to do a cheap, contiguous range scan across that key.
| Shard key choice | Hotspot risk | Trade-off |
|---|---|---|
_id or a timestamp, range-sharded | High — new writes always hit the same shard | Efficient range queries on that field, if you weren't hotspotting |
A low-cardinality field (e.g. status with 4 possible values) | High — at most 4 usable shards worth of spread, ever | Simple to reason about, but doesn't scale past a handful of shards |
A high-cardinality field, hashed (e.g. customerId: "hashed") | Low — writes spread near-uniformly | Loses efficient range scans on that field |
💻 Code example
sh.shardCollection("shop.orders", { customerId: "hashed" });
Want a visual for this concept?
Generate a diagram tailored to “MongoDB — Indexing, Replica Sets & Sharding” — the AI picks whichever visual (flowchart, comparison, sequence, etc.) best fits.
Sign in to generate a visual →