advanced~3.5h

Resource Management: Requests, Limits, QoS Classes & OOMKilled

Requests and limits look like two simple numbers in a Pod spec, but they drive scheduling decisions, eviction priority, and two of the most common production incidents in Kubernetes: OOMKilled containers and invisible CPU throttling. This chapter is what those numbers actually do.

A resource REQUEST is what the scheduler guarantees a container will get — it's the number the Scheduler (covered in Core Components) uses to decide which node has enough capacity to place a Pod on. A resource LIMIT is the maximum a container is allowed to consume — exceeding a CPU limit gets throttled; exceeding a MEMORY limit gets the container killed outright.

These two numbers, combined across every container in a Pod, determine the Pod's QoS (Quality of Service) class, which in turn determines eviction priority when a node runs low on resources — a distinction that's invisible until the exact moment a node is under pressure and it matters enormously.

The Scheduler places a Pod only on a node with enough UNRESERVED capacity to satisfy every container's resource REQUESTS — limits play no role in the scheduling decision at all, only in runtime enforcement after the Pod is already running.
CPU limits are enforced via the Linux kernel's CFS (Completely Fair Scheduler) bandwidth control — a container hitting its CPU limit isn't killed, it's THROTTLED: its process is paused for the remainder of each scheduling period once it's used its allotted CPU-time slice, which shows up as increased latency, not an error.
Memory limits are enforced by the kernel's cgroup memory controller — a container that tries to allocate memory beyond its limit gets its process killed immediately by the OOM (Out-Of-Memory) killer, reported by Kubernetes as the container status OOMKilled, with no graceful shutdown opportunity.
QoS class is computed automatically from requests/limits: Guaranteed (every container has requests == limits for both CPU and memory), Burstable (at least one container has a request, but not meeting Guaranteed's exact-match rule), or BestEffort (no requests or limits set at all).

Set requests to a realistic baseline usage measured under normal load (not a guess) — check actual usage via kubectl top pod after running representative traffic before setting a request value.
Set a CPU limit ABOVE the request (allowing legitimate bursting) unless there's a specific reason to hard-cap it — an overly tight CPU limit throttles a container even when the NODE has plenty of spare CPU sitting idle.
Set a memory limit close to the request for anything latency-sensitive, since memory limit violations are fatal (OOMKilled) rather than merely throttled — there's no 'graceful degradation' equivalent to CPU throttling for memory.
Check kubectl describe pod for a OOMKilled reason and Last State: Terminated after a suspicious restart, and cross-reference against kubectl top pod history to confirm memory pressure was the actual cause before assuming it's a code bug.

A JVM-based Spring Boot service (from the Spring Boot on Kubernetes chapter) gets OOMKilled repeatedly in production despite 'plenty of memory' on the node — the actual cause is the JVM heap max-size flag being set HIGHER than the Pod's memory limit, so the JVM itself never sees a reason to garbage-collect aggressively before the KERNEL kills the whole container outright.

A batch-processing service with a tight CPU limit shows unexplained latency spikes under load, invisible in application-level logs — kubectl top and cgroup throttling metrics reveal the container is being CPU-throttled well before the node itself is under any real CPU pressure, purely because its own limit was set too conservatively.

A cluster under real memory pressure evicts BestEffort Pods (no requests/limits set at all) FIRST, then Burstable Pods exceeding their requests, and evicts Guaranteed Pods only as an absolute last resort — a Pod's QoS class, decided purely by how its manifest was written, silently determines whether it survives a node-pressure event.

Always set BOTH requests and limits explicitly — never rely on defaults (which are often none at all, silently landing a Pod in the lowest-priority BestEffort QoS class).
Align application-level memory settings (JVM heap max, Node.js --max-old-space-size) to sit comfortably BELOW the Pod's memory limit, with headroom for non-heap memory (thread stacks, native buffers) — this is the single most common cause of avoidable OOMKilled incidents.
Use Guaranteed QoS (requests == limits) for genuinely critical, latency-sensitive workloads that must never be evicted before less-important Pods on the same node.
Monitor CPU throttling metrics (container_cpu_cfs_throttled_periods_total from cAdvisor/Prometheus) directly, not just raw CPU usage — a container can show LOW average CPU usage while still being frequently throttled in short bursts that raw averages completely hide.

Setting a memory limit without checking what the application itself is configured to use

— a JVM/Node/Python process with an internal memory ceiling ABOVE the container's Kubernetes memory limit gets killed by the kernel before the application's own memory management ever kicks in.

Copy-pasting the same requests/limits across every service regardless of actual usage

— a value that's correct for a lightweight API service is wildly wrong (too small, causing OOMKilled; or too large, wasting scheduling capacity cluster-wide) for a memory-intensive batch job.

Treating CPU throttling as 'the container is just slow'

— without checking throttling-specific metrics, a throttled container LOOKS like generic slowness in application logs, sending an on-call engineer investigating the wrong layer entirely (application code) instead of the actual cause (a too-tight CPU limit).

Leaving requests/limits unset 'to keep things simple' in a shared cluster

— every unset Pod lands in BestEffort, meaning it's first in line for eviction under any resource pressure, and contributes to unpredictable scheduling for every OTHER Pod on the same nodes too.

Right-sizing requests (neither too high, wasting reserved-but-unused capacity across the cluster, nor too low, risking eviction/throttling) is the single highest-leverage resource-management action — use actual measured usage (via kubectl top or a VPA in recommendation-only mode, from the Scaling chapter) rather than guessing.
CPU limits that are too tight silently cap throughput even on an otherwise-idle node — for latency-sensitive services, consider a generous CPU limit (or none) paired with a well-tuned request, rather than defaulting to a conservative limit 'just in case.'
Bin-packing efficiency across a cluster improves when requests closely match REAL usage — over-requested Pods reserve capacity that then sits unused, forcing the cluster autoscaler (Scaling chapter) to provision more nodes than actual load requires.

Make setting BOTH requests and limits a mandatory admission-controlled policy (via a LimitRange or an OPA/Kyverno policy) cluster-wide, rather than trusting every team to remember on every deployment.
Alert on OOMKilled restart reasons and on sustained CPU-throttling metrics as first-class production signals, not something only investigated after a user-visible incident already occurred.
Periodically review actual usage vs. configured requests/limits (many teams do this quarterly) and adjust — resource needs drift as code and traffic patterns change, and a request/limit set correctly a year ago is not guaranteed to still be correct today.
Reserve Guaranteed QoS deliberately for the small set of Pods that genuinely need eviction priority — making everything Guaranteed defeats the purpose of having QoS tiers at all, since a cluster under real pressure then has no lower-priority Pods left to evict first.

Deploy a Pod with no resource requests/limits set at all, then use kubectl describe pod to confirm it lands in the BestEffort QoS class.
Deploy a container with a deliberately-too-low memory limit and a workload that allocates more than that limit, then observe the OOMKilled status in kubectl describe pod's container state.
Deploy a container with a tight CPU limit under sustained CPU load, and check container_cpu_cfs_throttled_periods_total (via kubectl top or a Prometheus query) to see throttling directly, correlating it with observed request latency.

Requests drive scheduling (where a Pod can go); limits drive runtime enforcement (CPU throttling, memory OOMKilled) — the two numbers together determine a Pod's QoS class and, ultimately, its eviction priority under node pressure. The two most common resource-management incidents — an application-level memory setting exceeding the container's Kubernetes limit (OOMKilled), and a too-tight CPU limit silently throttling a service that LOOKS merely slow in application logs — are both invisible until someone specifically checks the right metric, which is exactly why they're among the most common real production surprises in Kubernetes.

Want a visual for this concept?

Generate a diagram tailored to “Resource Management: Requests, Limits, QoS Classes & OOMKilled” — the AI picks whichever visual (flowchart, comparison, sequence, etc.) best fits.

Practice quiz

Next Step

Continue to High Availability: Multi-Node Control Plane & Leader Election →← Back to all Kubernetes chapters