OpenShift - Resolve "DaemonSet openshift-sdn is not available awaiting nodes"

Let's say the oc get clusteroperator network command returns the following. Notice network is degraded.

~]$ oc get clusteroperator network
NAME      VERSION   AVAILABLE   PROGRESSING   DEGRADED   SINCE
network   4.6.41    True        True          True       335d

 

The oc describe clusteroperator network command can be used to display details for the cluster operator network. Notice in this example that the deploy is progressing, waiting on nodes in the openshift-sdn namespace.

~]$ oc describe clusteroperator network
Status:
  Condition:
    Last Transaction Time: 2022-02-10T04:02:41Z
    Message:               DaemonSet "openshift-sdn/ovs" is not available (awaiting 5 nodes)
                           DaemonSet "openshift-sdn/sdn" is not available (awaiting 6 nodes)
    Reason:                Deploying
    Status:                True
    Type:                  Progressing

 

The oc get daemonset command can be used to show the state of the daemon sets in the openshift-sdn namespace. Notice in this example that the READY and AVAILABLE counts (25) are less than the DESIRED, CURRENT and UP-TO-DATE counts for the ovs and sdn deamon sets.

A daemonset is used to run a pod on specific nodes or all nodes.

~]# oc get daemonset --namespace openshift-sdn
NAME            DESIRED  CURRENT  READY  UP-TO-DATE  AVAILABLE  NODE SELECTOR                   AGE
ovs             30       30       25     30          25         kubernetes.io/os=linux          483d
sdn             30       30       25     30          25         kubernetes.io/os=linux          483d
sdn-controller  3        3        3      3           3          node-role.kubernetes.io/master  483d

 

This can occur as identified in the following Red Hat Bug when changes are made to the OVS or SDN deamon sets.

https://bugzilla.redhat.com/show_bug.cgi?id=1718375

And the fix for this bug is included in RHBA-2019:1382.

If you reissue the oc get daemonset command about once every 30 minutes, hopefully you will see that the READY and AVAILABLE counts slowly increasing until these counts eventually match the DESIRED, CURRENT and UP-TO-DATE counts.

~]# oc get daemonset --namespace openshift-sdn
NAME            DESIRED  CURRENT  READY  UP-TO-DATE  AVAILABLE  NODE SELECTOR                   AGE
ovs             30       30       30     30          30         kubernetes.io/os=linux          483d
sdn             30       30       30     30          30         kubernetes.io/os=linux          483d
sdn-controller  3        3        3      3           3          node-role.kubernetes.io/master  483d

 

Once the daemon set READY and AVAILABLE counts match the DESIRED, CURRENT and UP-TO-DATE count, reissue the oc get clusteroperator network command, and degraded should be false.

~]$ oc get clusteroperator network
NAME      VERSION   AVAILABLE   PROGRESSING   DEGRADED   SINCE
network   4.6.41    True        False         False      335d

 

And the oc describe clusteroperator network command should hopefully return something like this.

~]$ oc describe clusteroperator network
Status:
  Condition:
    Last Transaction Time: 2022-02-10T06:12:44Z
    Status:                False
    Type:                  Degraded



Did you find this article helpful?

If so, consider buying me a coffee over at Buy Me A Coffee

Add a Comment




We will never share your name or email with anyone. Enter your email if you would like to be notified when we respond to your comment.





Please enter 0cd81 in the box below so that we can be sure you are a human.




Comments

Web design by yours truely - me, myself, and I   |   jeremy.canfield@freekb.net   |