advanced~3h

MongoDB Performance — Explain Plans, Covered Queries & Pagination

A dashboard that's fast in a demo and slow at page 500 isn't a different bug from a query with no index — both come from the same instinct to skip past work instead of seeking directly to it. This chapter covers reading explain() as a real debugging tool, reaching the fully index-only covered-query state, and fixing pagination that gets slower the deeper a user scrolls.

Learning objectives

  • Diagnose a slow query from explain("executionStats") using totalKeysExamined, totalDocsExamined, and in-memory SORT stages.
  • Reach and verify a covered query, including the _id: 0 projection requirement.
  • Explain why skip()-based pagination gets slower with page depth.
  • Implement range-based (keyset) pagination in MongoDB, including tie-breaking on a non-unique sort field.

Chapter 02 introduced COLLSCAN vs IXSCAN and the headline totalDocsExamined vs nReturned comparison. Used as an actual production debugging tool, the workflow looks like this:

// a customer support dashboard query, reported as "slow" db.orders.find({ region: "south", status: "DELIVERED" }) .sort({ createdAt: -1 }) .explain("executionStats"); // BEFORE — no useful index: // executionStats.executionStages.stage: "COLLSCAN" // totalDocsExamined: 480000, nReturned: 340, executionTimeMillis: 1180 // AFTER db.orders.createIndex({ region: 1, status: 1, createdAt: -1 }): // winningPlan.stage: "SORT" -> ... -> "IXSCAN" (an in-memory SORT stage here means // the index didn't fully satisfy the sort order) // totalKeysExamined: 340, totalDocsExamined: 340, executionTimeMillis: 8

Two extra signals worth reading beyond the top-line numbers:

  • totalKeysExamined vs totalDocsExamined — under an IXSCAN, these should be close to each other and close to nReturned. A large gap between keys examined and docs actually returned means the index narrowed the search less than you'd expect for that query shape — often a sign a compound index's field order (Chapter 02 §1's ESR rule) doesn't match this query's filter/sort combination.
  • A SORT stage appearing above the IXSCAN — this means MongoDB had to sort results in memory after fetching them, because the index alone didn't already return them in the needed order. An index whose field order matches both the equality filters and the sort direction (again, the ESR rule) lets winningPlan skip the in-memory sort stage entirely — a genuinely faster and lower-memory query.

queryPlanner.rejectedPlans (visible in the fuller explain() output) shows which other candidate indexes MongoDB considered and discarded before settling on winningPlan — useful when you have several plausible indexes and want to confirm the one you expect is actually the one chosen.

A covered query is answered entirely from the index — MongoDB never has to fetch the actual document from the collection at all, because every field the query needs (both the filter and the returned projection) already lives inside the index itself.

db.customers.createIndex({ name: 1 }); // NOT covered — projection includes fields not in the index (and _id is // included by default, and _id is not part of this index either) db.customers.find({ name: "Max" }).explain("executionStats"); // totalDocsExamined: 1 — still had to fetch the actual document // covered — filter field AND every projected field are in the index, // and _id is explicitly excluded db.customers.find({ name: "Max" }, { _id: 0, name: 1 }).explain("executionStats"); // totalDocsExamined: 0 — answered entirely from the index

Two conditions have to hold together: every field in the query's filter must be in the index, and every field in the projection must be in the index too — including _id, which MongoDB includes by default unless you explicitly write _id: 0.

▲ Common mistake

Building an index on exactly the fields a hot query filters and returns, then forgetting to add _id: 0 to the projection — the query silently falls out of the covered-query path (back to fetching full documents) because _id is being returned by default and isn't part of the index. This is a genuinely easy, easy-to-miss regression: the query still works and returns correct results, it's just quietly no longer covered. It's not worth building indexes purely to chase covered-query status for every query — the write-side cost of maintaining more indexes is real (Chapter 02) — but for a small number of genuinely hot, simple read paths, reaching covered-query state is a real, measurable win.

💻 Code example

db.customers.createIndex({ name: 1 }); // NOT covered db.customers.find({ name: "Max" }).explain("executionStats"); // totalDocsExamined: 1 // covered — filter field AND every projected field are in the index, _id excluded db.customers.find({ name: "Max" }, { _id: 0, name: 1 }).explain("executionStats"); // totalDocsExamined: 0

◆ The problem

db.products.find().sort({ createdAt: -1 }).skip(10000).limit(20) — MongoDB's index has no way to "jump" directly to the 10,000th matching entry. To honor skip(10000), it still has to walk the index from the start, one entry at a time, discarding the first 10,000 before it can return your 20. Page 1 is fast. Page 500 of the same listing does roughly 500 times the work of page 1, to return the exact same number of results — the query gets slower the deeper a user pages, purely from skip()'s cost, with the index itself never able to shortcut past entries it hasn't walked yet.

This is the identical lesson SQL Mastery, Chapter 02 teaches about OFFSET in SQL — the mechanism is different (a B-tree walk vs. an index walk) but the root cause is the same: an offset-based cursor has no way to seek directly into the middle of an ordered result set, in either database, so it pays for every row or document it skips past, every single request.

Instead of "skip N," keyset pagination asks a question the index can answer in constant time: "give me the next 20 after this specific point I already saw." The cursor is the last-seen document's sort-key value, not a row count — so the index seeks directly to it, no matter how many pages deep you already are.

// Page 1 — no cursor yet db.products.find().sort({ _id: -1 }).limit(20); // Page 2+ — pass the LAST _id seen on the previous page as lastSeenId db.products.find({ _id: { $lt: lastSeenId } }).sort({ _id: -1 }).limit(20); // the index SEEKS directly to lastSeenId, then reads 20 forward from there — // constant-time work regardless of how many pages deep you are

Because MongoDB's default _id (ObjectId) is roughly time-ordered (Chapter 02 §4), it's often directly usable as a natural keyset cursor for a createdAt-style feed with no extra field needed. Sorting on a non-unique field (like price) needs a compound tie-breaker so two documents with the same price don't get skipped or duplicated across pages — the same technique SQL keyset pagination uses:

db.products.find({ $or: [ { price: { $gt: lastPrice } }, { price: lastPrice, _id: { $gt: lastId } } ] }).sort({ price: 1, _id: 1 }).limit(20);

▲ Common mistake

Sorting by a non-unique field and using only that field as the cursor (price: { $gt: lastPrice } alone) — any two products tied on price get a nondeterministic order across page boundaries, and it's possible for a product to be silently skipped entirely if it lands exactly on a page boundary with a price tie. The compound $or tie-break above is what makes the pagination deterministic.

💻 Code example

// Page 1 db.products.find().sort({ _id: -1 }).limit(20); // Page 2+ db.products.find({ _id: { $lt: lastSeenId } }).sort({ _id: -1 }).limit(20); // tie-broken keyset pagination on a non-unique sort field db.products.find({ $or: [ { price: { $gt: lastPrice } }, { price: lastPrice, _id: { $gt: lastId } } ] }).sort({ price: 1, _id: 1 }).limit(20);
ScenarioRecommended approachWhy
Public infinite-scroll feed, deep paging expectedKeyset/range-basedskip() cost grows unboundedly with page depth
Admin table with a small, bounded row count (hundreds, not millions)skip() is fineThe walk skip() pays for is small regardless
"Jump to page 47 of 300" UI requirementskip() (keyset can't jump to an arbitrary page number, only "next"/"previous")Keyset pagination is fundamentally sequential — it has no equivalent of an arbitrary page-number jump

Keyset pagination trades away one real capability: it only supports "next page" and "previous page" relative to a cursor, not "jump to page 47." If a product genuinely needs arbitrary page-number jumping at real depth, that's a harder problem (usually solved by capping how deep users can actually jump, or paginating a pre-computed, cached result set) rather than something either pagination style solves cleanly on its own. Verify any pagination fix the same way Chapter 02 taught — with explain("executionStats") on the actual query at a realistic page depth, not just on page 1.

Want a visual for this concept?

Generate a diagram tailored to “MongoDB Performance — Explain Plans, Covered Queries & Pagination” — the AI picks whichever visual (flowchart, comparison, sequence, etc.) best fits.

Sign in to generate a visual →

Practice quiz

Next Step

Continue to Spring Boot with MongoDB — Spring Data MongoDB & MongoRepository← Back to all MongoDB chapters