🕸 Kubernetes Taints and Tolerations, Node Selectors, and Node Affinity

4 min readOct 8, 2024

This guide provides an in-depth explanation of key Kubernetes concepts used to control Pod scheduling on Nodes: Taints, Tolerations, Node Selectors, and Node Affinity. Each concept is explained with full details and practical examples.

1. Taints and Tolerations in Kubernetes

Taints: Mark Nodes as unsuitable for some Pods by adding a taint. Taints stop Pods from being scheduled on a Node unless they tolerate the taint.

Usage: Taints are typically used to reserve certain Nodes for specific workloads or to prevent certain workloads from being scheduled on a Node.
Taint Format: A taint consists of three parts:
Key: The label or attribute name.
Value: The specific value of the key.
Effect: The action to take when a Pod does not tolerate the taint. Three possible effects:

NoSchedule: Do not schedule the Pod on the Node unless the Pod tolerates the taint.
PreferNoSchedule: Prefer to avoid scheduling the Pod on the Node, but it's not a strict rule.
NoExecute: Evict any already-running Pods that do not tolerate the taint.

Example: Tainting a Node

Let’s taint a Node to prevent Pods from being scheduled on it unless they have a matching toleration.

kubectl taint nodes <node-name> environment=prod:NoSchedule

This command adds a taint to node1 with environment=prod and the NoSchedule effect.

Tolerations: Allow Pods to ignore taints and be scheduled on Nodes that have them.

Usage: Add tolerations to Pods to allow them to run on Nodes with specific taints.

Example: Pod with Toleration

This Pod has a toleration for the taint "environment=prod:NoSchedule", allowing it to run on tainted Nodes.

apiVersion: v1
kind: Pod
metadata:
  name: toleration-example
spec:
  containers:
  - name: nginx
    image: nginx
  tolerations:
  - key: "environment"
    operator: "Equal"
    value: "prod"
    effect: "NoSchedule"

Here, the toleration allows the Pod to be scheduled on Nodes with the environment=prod taint.

2. Node Selector in Kubernetes

Node Selector: This is the simplest way to constrain a Pod to run only on Nodes with specific labels.

Usage: You can label your Nodes and then use nodeSelector in your Pod spec to ensure Pods are scheduled only on Nodes that match the specified labels.
Limitation: Node selector only supports exact matching (key and value must match exactly).

Example: Labeling a Node

Label a Node with disk=ssd to mark it as a Node with SSD storage.

kubectl label nodes <node-name> disk=ssd

Example: Pod with Node Selector

This Pod will be scheduled only on Nodes that have the label disk=ssd.

apiVersion: v1
kind: Pod
metadata:
  name: node-selector-example
spec:
  containers:
  - name: nginx
    image: nginx
  nodeSelector:
    disk: ssd

In this case, the Pod will only be scheduled on Nodes labeled disk=ssd. If no such Nodes are available, the Pod will not be scheduled.

3. Node Affinity in Kubernetes

Node Affinity: Provides more advanced and flexible control over where Pods are scheduled, compared to node selectors.

Types of Node Affinity:

RequiredDuringSchedulingIgnoredDuringExecution: The Pod must be scheduled on Nodes matching the affinity rules. This is strict and similar to node selectors.
PreferredDuringSchedulingIgnoredDuringExecution: The Pod prefers to be scheduled on Nodes matching the affinity rules, but if no such Nodes are available, the Pod can be scheduled elsewhere.

Example: Labeling Nodes for Affinity

Label two Nodes with different zones.

kubectl label nodes node1 zone=us-east-1
kubectl label nodes node2 zone=us-west-2

Example: Pod with Preferred Node Affinity

This Pod prefers to be scheduled on Nodes labeled zone=us-east-1 but can still run on other Nodes if no matching Node is available.

apiVersion: v1
kind: Pod
metadata:
  name: preferred-affinity-example
spec:
  containers:
  - name: nginx
    image: nginx
  affinity:
    nodeAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 1
        preference:
          matchExpressions:
          - key: zone
            operator: In
            values:
            - us-east-1

In this case, the Pod prefers Nodes in the us-east-1 zone but can be scheduled elsewhere if needed.

Example: Pod with Required Node Affinity

This Pod will only be scheduled on Nodes labeled zone=us-east-1. If no such Node is available, the Pod will not be scheduled.

apiVersion: v1
kind: Pod
metadata:
  name: strict-affinity-example
spec:
  containers:
  - name: nginx
    image: nginx
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: zone
            operator: In
            values:
            - us-east-1

This is a strict rule, so the Pod can only run on Nodes in the us-east-1 zone.

4. Multiple Affinity Rules

You can combine node affinity with other constraints, such as pod affinity and anti-affinity, for even more granular control.

Example: Combining Node Affinity and Tolerations

You can use both tolerations and node affinity in the same Pod spec to control both Node taints and Node labels.

apiVersion: v1
kind: Pod
metadata:
  name: combined-affinity-toleration
spec:
  containers:
  - name: nginx
    image: nginx
  tolerations:
  - key: "environment"
    operator: "Equal"
    value: "prod"
    effect: "NoSchedule"
  affinity:
    nodeAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 1
        preference:
          matchExpressions:
          - key: zone
            operator: In
            values:
            - us-east-1

This Pod will prefer to run in us-east-1 zone and will tolerate the environment=prod taint, allowing it to be scheduled on a Node with that taint.

Conclusion

Taints and Tolerations: Control which Pods can be scheduled on certain Nodes by marking Nodes as unschedulable unless the Pod has a matching toleration.
Node Selector: A simple, exact match mechanism for ensuring Pods are scheduled on Nodes with specific labels.
Node Affinity: Provides more flexible control, allowing for both preferred and required scheduling based on Node labels.

Each of these mechanisms can be combined to fine-tune the scheduling behavior of Pods in your Kubernetes cluster, ensuring Pods run on the most appropriate Nodes for your workload.