intermediate~4h

Scheduling: Affinity, Taints & Topology Spread

Real clusters use four mechanisms to influence Pod placement: Node Affinity, Pod Affinity/Anti-Affinity, Taints and Tolerations, and Topology Spread Constraints. Each solves a distinct placement probl

Node Affinity: requiredDuringSchedulingIgnoredDuringExecution is a hard constraint; preferredDuringScheduling is a soft preference.
IgnoredDuringExecution: rules are only evaluated at scheduling time. If node labels change later, running Pods are NOT evicted.
Taints have three effects: NoSchedule, PreferNoSchedule, NoExecute (evict running Pods, optionally after tolerationSeconds).
A toleration cancels the repelling effect of a matching taint. It does NOT attract the Pod to that node.
Topology spread constraints express 'spread these Pods across this topology domain with at most this skew'.

Label nodes with the attributes you want to schedule against (e.g. disktype=ssd, topology.kubernetes.io/zone -- usually already set by the cloud provider).
Add nodeAffinity to the Pod spec with requiredDuringSchedulingIgnoredDuringExecution for a hard constraint (e.g. must run on GPU nodes) or preferredDuringScheduling for a soft preference.
For co-locating or spreading Pods relative to each other, add podAffinity or podAntiAffinity with a labelSelector matching the other Pods and a topologyKey defining the domain (e.g. kubernetes.io/hostname, topology.kubernetes.io/zone).
Taint nodes that should repel Pods by default (kubectl taint nodes ... key=value:NoSchedule) -- e.g. dedicating nodes to a specific team or workload.
Add a matching toleration to the Pods that ARE allowed on those tainted nodes -- a toleration only cancels the repulsion, it doesn't attract the Pod there (pair with nodeAffinity if you also want to attract it).
For even distribution across zones/nodes, add topologySpreadConstraints specifying topologyKey, maxSkew, and whenUnsatisfiable (DoNotSchedule vs ScheduleAnyway).

Requiring GPU-heavy ML training Pods to schedule only on GPU-equipped nodes via nodeAffinity plus a matching taint/toleration pair so non-GPU workloads never land there wastefully.
Using podAntiAffinity to spread replicas of a critical service across different nodes (and zones) so a single node or zone failure doesn't take down every replica at once.
Using podAffinity to co-locate a companion service on the same node as the Pods it serves, minimizing network hops.
Dedicating a pool of nodes to a specific tenant or team via taints, with only that team's Pods carrying the matching toleration.
Using topologySpreadConstraints across availability zones for a quorum-based service so it doesn't lose majority when one zone goes down.

Prefer topologySpreadConstraints over manual podAntiAffinity for even distribution goals -- it expresses 'spread evenly with this much skew' directly instead of approximating it with anti-affinity rules.
Use preferredDuringScheduling (soft) affinity by default and reserve requiredDuringScheduling (hard) for genuine hard requirements -- hard constraints can leave Pods permanently unschedulable if matching nodes are ever unavailable.
Combine taints/tolerations (repel everyone except X) with nodeAffinity (attract X specifically) when dedicating nodes -- a toleration alone doesn't prevent other tolerating workloads from also landing there.
Set whenUnsatisfiable: ScheduleAnyway for best-effort spreading, and DoNotSchedule only when even distribution is a hard production requirement.
Remember affinity/taint rules are evaluated only at scheduling time -- plan for node-label drift by re-evaluating placement rather than assuming running Pods react to later label changes.

Assuming a toleration attracts a Pod to a tainted node -- it only allows scheduling there among other eligible nodes; without matching nodeAffinity too, the Pod could land anywhere else it's also eligible.
Using requiredDuringSchedulingIgnoredDuringExecution too liberally, causing Pods to go permanently Pending when no node currently matches, with no automatic fallback.
Expecting a Pod already running on a node to be evicted when that node's labels change -- IgnoredDuringExecution means affinity rules are one-time, at scheduling only.
Setting topologySpreadConstraints with maxSkew too tight for the actual node/zone count available, causing Pods to go unschedulable rather than 'close enough' distribution.
Using NoExecute taints without understanding they actively evict already-running Pods (optionally after tolerationSeconds) -- different from NoSchedule, which only blocks new Pods.
Forgetting that control-plane nodes carry a taint by default -- Pods needing to run there need an explicit toleration or they're silently excluded.

Overly complex affinity rules (many required terms, large clusters) increase scheduler decision latency per Pod -- keep rule sets as simple as the actual placement requirement allows.
Hard node/pod affinity constraints reduce the scheduler's effective placement options, which can worsen overall cluster bin-packing efficiency even if each individual Pod's placement is technically satisfied.
Topology spread constraints with DoNotSchedule can cause Pods to sit Pending rather than start slightly unevenly distributed -- weigh strict evenness against actual startup-time requirements.
Pod anti-affinity with topologyKey: kubernetes.io/hostname on large Deployments can make scheduling progressively harder as the cluster fills, since fewer eligible hosts remain for each additional replica.

Combine topologySpreadConstraints across zones with PodDisruptionBudgets so both scheduling-time distribution and runtime disruption tolerance protect the same availability goal.
Reserve hard (required) affinity for genuine hardware/compliance requirements (GPU nodes, data-residency zones) -- use soft (preferred) for everything else to avoid unschedulable-Pod incidents.
Document and label dedicated node pools (taints + labels) clearly so on-call engineers understand why certain Pods won't schedule on certain nodes during an incident.
Monitor for Pods stuck Pending due to unsatisfiable affinity/topology-spread rules as a first-class alert -- this failure mode is silent and easy to miss.

Label two nodes differently (e.g. disktype=ssd on one) and use requiredDuringSchedulingIgnoredDuringExecution nodeAffinity to force a Pod onto the labeled node only.
Taint one node and deploy a Pod without a toleration, confirming it's never scheduled there, then add the toleration and confirm it now can be.
Deploy a Deployment with podAntiAffinity (topologyKey: kubernetes.io/hostname) across a 3-node cluster and confirm via kubectl get pods -o wide that no two replicas share a node.
Apply a topologySpreadConstraint across zones on a multi-zone cluster and confirm even Pod distribution with kubectl get pods -o wide.

Node affinity and Pod affinity/anti-affinity let Pods express required or preferred placement constraints.
Affinity rules are evaluated only at scheduling time -- already-placed Pods are not retroactively moved.
Taints repel Pods from a node; tolerations cancel that repulsion but create no positive attraction.
NoExecute is the only taint effect that actively evicts already-running, non-tolerating Pods.

Want a visual for this concept?

Generate a diagram tailored to “Scheduling: Affinity, Taints & Topology Spread” — the AI picks whichever visual (flowchart, comparison, sequence, etc.) best fits.

Practice quiz

Next Step

Continue to Security: RBAC, Service Accounts & Pod Security Standards →← Back to all Kubernetes chapters