Bootstrap FreeKB - OpenShift - Resolve "Cannot evict pod as it would violate the pod's disruption budget"
OpenShift - Resolve "Cannot evict pod as it would violate the pod's disruption budget"

Updated:   |  OpenShift articles

Let's say you are using the oc adm drain command to evict pods from a node.

oc adm drain worker-hsjrp

 

And something like this is being returned.

error when evicting pods/"my-pod-gqfs2" -n "my-namespace" (will retry after 5s): Cannot evict pod as it would violate the pod's disruption budget.

 

This means a Pod Disruption Budget is preventing the pod from being evicted from the node. Notice in this example that the pod is in my-namespace and min available is 1 meaning there must always be 1 running pod. The oc get PodDisruptionBudgets command can be used to the the Pod Disruption Budgets in the namespace the pod is in.

~]$ oc get PodDisruptionBudgets --namespace my-namespace
NAME                       MIN AVAILABLE   MAX UNAVAILABLE   ALLOWED DISRUPTIONS   AGE
my-pod-disruption-budget   1               N/A               0                     26s

 

This often occurs when the deployment / replica set / pod is configured with 1 replica and the Pod Disruption Budget is configured with 1 minimum, meaning there must always be 1 running pod.

~]# oc get deployment my-deployment --output yaml
spec:
  replicas: 1

 

Probably the easiest solution is to update the deployment to have 1 more replica than the Pod Disruption Budget minimum available. For example, if the Pod Disruption Budget is configured with 1 minimum, then configure the deployment to have 2 (or more) replicas.

~]# oc get deployment my-deployment --output yaml
spec:
  replicas: 2

 

Which should spawn a second pod running on a different node.

~]$ oc get pods --output wide
NAME           READY   STATUS    RESTARTS   AGE   IP             NODE           NOMINATED NODE   READINESS GATES
my-pod-7vj6h   1/1     Running   0          21s   10.131.1.131   worker-2xqdt   <none>           <none>
my-pod-gqfs2   1/1     Running   0          24h   10.128.2.67    worker-4nd2v   <none>           <none>

 

Now when you drain the node the pod is gracefully evicted from the pod. Cool.

~]$ oc adm drain worker-4nd2v --pod-selector app=my-deployment
node/worker-4nd2v cordoned
evicting pod my-namespace/my-pod-gqfs2
pod/my-pod-gqfs2 evicted
node/worker-4nd2v drained

 

And a new pod was spawned. Perfect.

~]$ oc get pods --output wide
NAME           READY   STATUS    RESTARTS   AGE   IP             NODE                  NOMINATED NODE   READINESS GATES
my-pod-42zq4   1/1     Running   0          32s   10.131.1.132   lab001-worker-2xqdt   <none>           <none>
my-pod-7vj6h   1/1     Running   0          4m    10.131.1.131   lab001-worker-2xqdt   <none>           <none>

 

 




Did you find this article helpful?

If so, consider buying me a coffee over at Buy Me A Coffee



Comments


Add a Comment


Please enter f66191 in the box below so that we can be sure you are a human.