[SOLVED] Node had taints, that the pod didn’t tolerate

Kubernetes (K8s) has a feature, that allows to mark (taint) a Node, so that no Pods can be scheduled on it, unless a Pod explicitly tolerates the taint.

If you try to deploy the Pod that doesn’t tolerate any of the K8s Nodes’ taints, you will get an error as follows:

0/5 nodes are available: 1 node(s) had taint {node-role.node-role.kubernetes.io/master: }, that the pod didn’t tolerate, 2 node(s) had taint {dedicated: agents}, that the pod didn’t tolerate, 2 node(s) had taint {dedicated: controllers}, that the pod didn’t tolerate.

To tolerate the K8s Nodes’ taints, it is required to specify a toleration for the Pod in the PodSpec, and below I will show the examples of how to do this.

Cool Tip: Get Pod’s logs using the kubectl command! Read more →

Tolerate K8s Nodes’ Taints

To list taints on the Nodes, execute:

$ kubectl get nodes -o custom-columns=NAME:.metadata.name,TAINTS:.spec.taints --no-headers
- sample output -
node-0   [map[effect:NoSchedule key:node-role.kubernetes.io/master]]
node-1   [map[effect:NoSchedule key:dedicated value:agents]]
node-2   [map[effect:NoSchedule key:dedicated value:controllers]]

The toleration is a key-value pair that matches the taint on the Node.

For example, if you have a Node with the taint as highlighted above, you can create a Pod that can be scheduled on that Node by adding the toleration as follows:

# nginx-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:latest
        ports:
        - containerPort: 80
    tolerations:
      operator: "Exists"
      key: "dedicated"
      value: "agents"
      effect: "NoSchedule"

You can use the operator “Exists” to match any value or effect for a given key:

tolerations:
  operator: "Exists"
  key: "<taintKey>"

For example, to allow the Pod to be scheduled on the K8s master Nodes, add:

tolerations:
  operator: "Exists"
  key: "node-role.kubernetes.io/master"

You can also match any taint by simply defining the “Exists” operator, as follows:

tolerations:
  operator: "Exists"

To ensure that the taint toleration has been applied to the Pod, execute:

$ kubectl get pod <podName> -o jsopath='{.spec.tolerations}'