A node (sometimes also referred to as a "machine") contains one or more pods, and each pod contains one or more containers.

The scheduler is responsible for determining which worker node a resource should get created on. For example, when deploying a new application to OpenShift, the scheduler determines which worker node the pod should be created on, typically the worker node with the most available memory and CPU. Check out my article on the default scheduler.

By default, a node with a Ready status will be marked as schedulable. The oc get nodes command can be used to determine if a node is Ready. In this example, all of the nodes are ready.

~]# oc get nodes
NAME                  STATUS   ROLES            AGE    VERSION
my-node-edge-lm6wz     Ready    infra,worker     519d   v1.23.5+012e945
my-node-edge-pmlls     Ready    infra,worker     519d   v1.23.5+012e945
my-node-infra-c4v5h    Ready    infra,worker     519d   v1.23.5+012e945
my-node-infra-mc8rc    Ready    infra,worker     519d   v1.23.5+012e945
my-node-infra-p9cjv    Ready    infra,worker     519d   v1.23.5+012e945
my-node-master-0       Ready    master           522d   v1.23.5+012e945
my-node-master-1       Ready    master           522d   v1.23.5+012e945
my-node-master-2       Ready    master           522d   v1.23.5+012e945
my-node-worker-lk5vm   Ready    compute,worker   61d    v1.23.5+012e945
my-node-worker-pj4r4   Ready    compute,worker   61d    v1.23.5+012e945

When schedulable, new pods can be placed on the node. When unschedulable, new pods will not be placed on the node. If there are already pods on the node, and you mark the pod as unschedulable, the pods will remain on the node and will not be impacted. The oc adm manage-node command can be used to mark a node as unschedulable.

oc adm manage-node my-node-worker-lk5vm --schedulable=false

Or the oc adm cordon command can be used to set the node as unschedulable.

oc adm cordon my-node-worker-lk5vm

The oc adm drain node command can be used to evict the pods on the node. In this example, the pods on my-node-worker-lk5vm will be evicted. If there is a replica set or replication controller for the pods being evicted, the pods will be created on one of the other available nodes, assuming the replicas set or replications controller is set with one or more replicas, and there isn't a nodeSelector or nodeAffinity or podAffinity or taint and tolerations preventing the pods from being scheduled on one of the other nodes.

The oc adm drain node command will also cordon the node, which set the node as unschedulable.

~]# oc adm drain node my-node-worker-lk5vm
node/my-node-worker-lk5vm cordoned
evicting pod my-pod-9mzm2
evicting pod my-pod-vmzmz
evicting pod my-pod-pflxc
pod/my-pod-9mzm2 evicted
pod/my-pod-vmzmz evicted
pod/my-pod-pflxc evicted
node/my-node-worker-lk5vm drained

I often include the following command line options and flags.

oc adm drain my-node-worker-lk5vm --ignore-daemonsets --delete-emptydir-data --force --grace-period=0 --timeout=180s

The status of the node should now include Scheduling Disabled so that no new pods are created on the node. 

~]$ oc get nodes
NAME                  STATUS                       ROLES            AGE    VERSION
my-node-edge-lm6wz     Ready                       infra,worker     519d   v1.23.5+012e945
my-node-edge-pmlls     Ready                       infra,worker     519d   v1.23.5+012e945
my-node-infra-c4v5h    Ready                       infra,worker     519d   v1.23.5+012e945
my-node-infra-mc8rc    Ready                       infra,worker     519d   v1.23.5+012e945
my-node-infra-p9cjv    Ready                       infra,worker     519d   v1.23.5+012e945
my-node-master-0       Ready                       master           522d   v1.23.5+012e945
my-node-master-1       Ready                       master           522d   v1.23.5+012e945
my-node-master-2       Ready                       master           522d   v1.23.5+012e945
my-node-worker-lk5vm   Ready                       compute,worker   61d    v1.23.5+012e945
my-node-worker-pj4r4   Ready,SchedulingDisabled    compute,worker   61d    v1.23.5+012e945

Often, after cordon and draining a node, you may want to restart the node. The oc debug command can be used to start a node in debug mode. This will provide you with a command line prompt in a "debug" pod for the node.

~]# oc debug node/my-node-5n4fj
Starting pod/my-node-5n4fj-debug ...
sh-4.4#

This one liner command can be used.

oc debug node/<node name> -- chroot /host systemctl reboot

The oc adm manage-node command can be used to mark a node as schedulable so that pods can again be created on the node. 

oc adm manage-node my-node-worker-lk5vm --schedulable=true

Or, the oc admin uncordon command can be used.

~]$ oc adm uncordon my-node-worker-lk5vm
node/my-node-worker-lk5vm uncordoned