Bootstrap FreeKB - OpenShift - Evicting pods from a node (oc adm drain)
OpenShift - Evicting pods from a node (oc adm drain)

Updated:   |  OpenShift articles

A node (sometimes also referred to as a "machine") contains one or more pods, and each pod contains one or more containers.

 

Evicting basically means a pod will be terminated and removed from a node.

The oc get nodes command will return the list of nodes.

~]$ oc get nodes
NAME                  STATUS   ROLES            AGE    VERSION
my-node-edge-lm6wz     Ready    infra,worker     519d   v1.23.5+012e945
my-node-edge-pmlls     Ready    infra,worker     519d   v1.23.5+012e945
my-node-infra-c4v5h    Ready    infra,worker     519d   v1.23.5+012e945
my-node-infra-mc8rc    Ready    infra,worker     519d   v1.23.5+012e945
my-node-infra-p9cjv    Ready    infra,worker     519d   v1.23.5+012e945
my-node-master-0       Ready    master           522d   v1.23.5+012e945
my-node-master-1       Ready    master           522d   v1.23.5+012e945
my-node-master-2       Ready    master           522d   v1.23.5+012e945
my-node-worker-lk5vm   Ready    compute,worker   61d    v1.23.5+012e945
my-node-worker-pj4r4   Ready    compute,worker   61d    v1.23.5+012e945

 

The oc adm drain command can be used to evict the pods on the node. In this example, the pods on my-node-worker-lk5vm will be evicted, and the pods should be recreated on one of the other worker node. It's definitely a good idea to include the --dry-run client option to see what this command should do.

~]# oc adm drain my-node-worker-lk5vm --dry-run client
node/my-node-worker-lk5vm cordoned (dry run)
evicting pod my-pod-9mzm2 (dry run)
evicting pod my-pod-vmzmz (dry run)
evicting pod my-pod-pflxc (dry run)
pod/my-pod-9mzm2 evicted (dry run)
pod/my-pod-vmzmz evicted (dry run)
pod/my-pod-pflxc evicted (dry run)
node/my-node-worker-lk5vm drained (dry run)

 

Let's say you have a deployment / pod that has label app: my-app.

metadata:
  labels:
    app: my-deployment

 

The --pod-selector option can be used to only drain pods from the node that match a certain label.

~]$ oc adm drain lab001-worker-4nd2v --pod-selector='app=my-app' --dry-run=client
node/lab001-worker-4nd2v cordoned (dry run)
evicting pod my-project/y-pod (dry run)
node/lab001-worker-4nd2v drained (dry run

 

I often include the following command line options and flags.

oc adm drain my-node-worker-lk5vm --ignore-daemonsets --delete-emptydir-data --force --grace-period=0 --timeout=180s

 

Or, the oc adm manage-node command can be used to mark a node as unschedulable. Unlike the oc admin drain node command, the pods will not be evicted from the node. The pods running on the node will continue to remain running on the node.

oc adm manage-node my-node-worker-lk5vm --schedulable=false

 

After the oc admin drain node command has been issued, the status of the node should include Scheduling Disabled so that no new pods are created on the node. This marks the node as unscheduable.

~]$ oc get nodes
NAME                  STATUS                       ROLES            AGE    VERSION
my-node-edge-lm6wz     Ready                       infra,worker     519d   v1.23.5+012e945
my-node-edge-pmlls     Ready                       infra,worker     519d   v1.23.5+012e945
my-node-infra-c4v5h    Ready                       infra,worker     519d   v1.23.5+012e945
my-node-infra-mc8rc    Ready                       infra,worker     519d   v1.23.5+012e945
my-node-infra-p9cjv    Ready                       infra,worker     519d   v1.23.5+012e945
my-node-master-0       Ready                       master           522d   v1.23.5+012e945
my-node-master-1       Ready                       master           522d   v1.23.5+012e945
my-node-master-2       Ready                       master           522d   v1.23.5+012e945
my-node-worker-lk5vm   Ready                       compute,worker   61d    v1.23.5+012e945
my-node-worker-pj4r4   Ready,SchedulingDisabled    compute,worker   61d    v1.23.5+012e945

 

The oc adm uncordon command can be used to mark a node as schedulable so that pods can again be created on the node. 

~]$ oc adm uncordon my-node-worker-lk5vm
node/my-node-worker-lk5vm uncordoned

 

Or, the oc adm manage-node command can be used.

oc adm manage-node my-node-worker-lk5vm --schedulable=true

 




Did you find this article helpful?

If so, consider buying me a coffee over at Buy Me A Coffee



Comments


Add a Comment


Please enter 0773c4 in the box below so that we can be sure you are a human.