Bootstrap FreeKB - OpenShift - Resolve "AttachVolume.Attach failed for volume"
OpenShift - Resolve "AttachVolume.Attach failed for volume"

Updated:   |  OpenShift articles

Let's say something like this is being returned.

Warning  FailedAttachVolume  10s   attachdetach-controller  AttachVolume.Attach failed for volume "my-volume" : rpc error: code = NotFound desc = volume my-volume was not found

 

This typically occurs when attempting to Mount a Persistent Volume in a container. For example, perhaps your deployment YAML has the following to mount my-persistent-volume-claim in the container.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-deployment
spec:
  template:
    spec:
      containers:
      - name: my-container
        volumeMounts:
        - mountPath: /var/data
          name: my-volume
      volumes:
      - name: my-volume
        persistentVolumeClaim:
          claimName: my-persistent-volume-claim

 

First and foremost, the oc get pvc command can be used to list the Persistent Volume Claims in the currently selected project / namespace to ensure that the Persistent Volume Claim exists and that the status of the Persistent Volume Cliam is BOUND meaning the Persistent Volume Claim is bound to a Persistent Volume.

~]$ oc get pvc
NAME                           STATUS   VOLUME                    CAPACITY   ACCESS MODES   STORAGECLASS   AGE
my-persistent-volume-claim     Bound    my-persistent-volume      1Gi        RWO            file-storage   256d

 

Assuming the Persistent Volume Claim exists and is bound, the oc adm policy scc-subject-review command can be used to return the Security Context Constraint the pod is using (my-security-context-constraint in this example).

~]# oc get pod my-pod --output yaml | oc adm policy scc-subject-review -f -
RESOURCE      ALLOWED BY   
Pod/my-pod    my-security-context-constraint

 

And then the oc describe scc command can be used to show more details about the Security Context Constraint. Notice in this example that my-security-context-constraint has downwardAPI, emptyDir, and projected for Allowed Volume Types, but does not have persistentVolumeClaim. This can cause "Attach failed for volume" to be returned.

~]$ oc describe scc my-security-context-constraint
Name:                                           anyuid
Priority:                                       10
Access:
  Users:                                        <none>
  Groups:                                       system:cluster-admins
Settings:
  Allow Privileged:                             false
  Allow Privilege Escalation:                   true
  Default Add Capabilities:                     <none>
  Required Drop Capabilities:                   MKNOD
  Allowed Capabilities:                         <none>
  Allowed Seccomp Profiles:                     <none>
  Allowed Volume Types:                         downwardAPI,emptyDir,projected
  Allowed Flexvolumes:                          <all>
  Allowed Unsafe Sysctls:                       <none>
  Forbidden Sysctls:                            <none>
  Allow Host Network:                           false
  Allow Host Ports:                             false
  Allow Host PID:                               false
  Allow Host IPC:                               false
  Read Only Root Filesystem:                    false
  Run As User Strategy: RunAsAny
    UID:                                        <none>
    UID Range Min:                              <none>
    UID Range Max:                              <none>
  SELinux Context Strategy: MustRunAs
    User:                                       <none>
    Role:                                       <none>
    Type:                                       <none>
    Level:                                      <none>
  FSGroup Strategy: RunAsAny
    Ranges:                                     <none>
  Supplemental Groups Strategy: RunAsAny
    Ranges:                                     <none>

 

In this scenario, let's create a Service Account.

~]$ oc create serviceaccount my-service-account
serviceaccount/my-service-account created

 

Then associate the Service Account with a Security Context Contraint (SCC) that has persistentVolumeClaim for Allowed Volume Types, such as anyuid or nonroot or restricted or privileged.

~]$ oc adm policy add-scc-to-user anyuid -z my-service-account
clusterrole.rbac.authorization.k8s.io/system:openshift:scc:anyuid added: "my-service-account"

 

The oc describe clusterrolebinding command can be used to validate that the service account is bound to the anyuid Security Context Constraint.

~]$ oc describe clusterrolebinding system:openshift:scc:anyuid
Name:         system:openshift:scc:anyuid
Labels:       <none>
Annotations:  <none>
Role:
  Kind:  ClusterRole
  Name:  system:openshift:scc:anyuid
Subjects:
  Kind            Name                Namespace
  ----            ----                ---------
  ServiceAccount  my-service-account  my-project

 

Then the oc set serviceaccount command can be used to have a deployment / pod run by the Service Account which will apply the Security Context Contraint (SCC) to the container / deployment / pod.

~]$ oc set serviceaccount deployment/my-deployment my-service-account
deployment.apps/my-deployment serviceaccount updated

 

the oc set serviceaccount command will update the deployment YAML to have serviceAccount and serviceAccountName

spec:
  template:
    spec:
      serviceAccount: my-service-account
      serviceAccountName: my-service-account

 

Now the oc adm policy scc-review command shows that the deployment has the Service Account and the anyuid Security Context Contraint (SCC).

~]$ oc get deployment/my-deployment --output yaml | oc adm policy scc-review -f -
RESOURCE                   SERVICE ACCOUNT      ALLOWED BY
Deployment/my-deployment   my-service-account   anyuid

 

And likewise the pod should also have the Security Context Constraint.

~]$ oc get pod/my-pod --output yaml | oc adm policy scc-review -f -
RESOURCE     SERVICE ACCOUNT      ALLOWED BY
Pod/my-pod   my-service-account   anyuid

 

And the oc describe pod command should show that the pod has the anyuid Security Context Contraint.

~]$ oc describe pod my-app-mkfjk
Annotations:  openshift.io/scc: anyuid

 




Did you find this article helpful?

If so, consider buying me a coffee over at Buy Me A Coffee



Comments


Add a Comment


Please enter a19c6d in the box below so that we can be sure you are a human.