
Let's say you use the oc delete project command to delete a project.
oc delete project my-project
But the project status remains Terminating indefinately.
~]$ oc get project my-project
NAME DISPLAY NAME STATUS
my-project Terminating
Let's take a look at the project YAML. Notice in this example the status has NamespaceFinalizersRemaining. In this scenario, you don't want to try to force delete the project. Instead, you want to determine why the project has remaining finalizers and then take appropriate action with the finalizers.
~]$ oc get project my-project --output yaml
apiVersion: project.openshift.io/v1
kind: Project
metadata:
annotations:
openshift.io/description: ""
openshift.io/display-name: ""
openshift.io/sa.scc.mcs: s0:c58,c17
openshift.io/sa.scc.supplemental-groups: 1003340000/10000
openshift.io/sa.scc.uid-range: 1003340000/10000
creationTimestamp: "2021-04-15T20:03:59Z"
deletionTimestamp: "2022-09-16T15:27:51Z"
labels:
kubernetes.io/metadata.name: my-project
maistra.io/member-of: my-project
kiali.io/member-of: my-project
name: my-project
resourceVersion: "1273963823"
uid: f112402c-cbdf-4805-a66e-66228d245de3
spec:
finalizers:
- kubernetes
status:
conditions:
- lastTransitionTime: "2022-09-16T15:27:58Z"
message: 'Some content in the namespace has finalizers remaining: finalizer.kiali
in 1 resource instances, maistra.io/my-project in 2 resource instances'
reason: SomeFinalizersRemain
status: "True"
type: NamespaceFinalizersRemaining
It is noteworthy that I had a situation where the istio-system project was stuck in a Terminating state, and I had to open a case with Red Hat. I first got the name of the master pods in the openshift-etcd project.
~]$ oc get pods --namespace openshift-etcd
NAME READY STATUS RESTARTS AGE
etcd-master-0 5/5 Running 0 27d
etcd-master-1 5/5 Running 0 27d
etcd-master-2 5/5 Running 0 27d
I then issued the etcdclt del command in one of the master pods to delete the maistra Service Mesh control planes and finally the istio-system namespace.
oc exec --tty true --stdin true etcd-master-0 --namespace openshift-etcd -- etcdctl del /kubernetes.io/maistra.io/servicemeshcontrolplanes/istio-system/basic
oc exec --tty true --stdin true etcd-master-0 --namespace openshift-etcd -- etcdctl del /kubernetes.io/maistra.io/servicemeshmemberrolls/istio-system/default
oc exec --tty true --stdin true etcd-master-0 --namespace openshift-etcd -- etcdctl del /kubernetes.io/namespaces/istio-system
Remove the Custom Resource Definitions
I then had to use the oc get customresourcedefinitions command to return the istio, Jaegar, Kiali, Maistra, Service Mesh Custom Resource Definitions.
~]# oc get customresourcedefinitions | egrep -i 'istio|jaegar|kiali|maistra|servicemesh'
kialis.kiali.io 2021-04-15T20:01:30Z
monitoringdashboards.monitoring.kiali.io 2021-04-15T20:01:30Z
servicemeshextensions.maistra.io 2021-04-15T20:09:13Z
servicemeshmemberrolls.maistra.io 2021-04-15T20:02:49Z
servicemeshmembers.maistra.io 2021-04-15T20:02:49Z
servicemeshpolicies.authentication.maistra.io 2021-04-15T20:09:13Z
servicemeshrbacconfigs.rbac.maistra.io 2021-04-15T20:09:13Z
And then use the oc delete customresourcedefinition command to delete the Custom Resource Definitions.
oc delete crd kialis.kiali.io
oc delete crd monitoringdashboards.monitoring.kiali.io
oc delete crd servicemeshextensions.maistra.io
oc delete crd servicemeshmemberrolls.maistra.io
oc delete crd servicemeshmembers.maistra.io
oc delete crd servicemeshpolicies.authentication.maistra.io
oc delete crd servicemeshrbacconfigs.rbac.maistra.io
And then re-issue the oc get customresourcedefinitions command to ensure the Custom Resource Definitions were deleted. No results should be returned.
oc get customresourcedefinitions | egrep -i 'istio|kiali|maistra|servicemesh'
If the Custom Resource Definition remains, you may need to remove finalizers from the Custom Resource Definition. For example, to delete the kialis.kiali.io Custom Resource Definition, I had to use the oc edit command and remove the customresourcecleanup.apiextensions.k8s.io finalizer. Once the finalizer was removed, the Custom Resource Definition was no longer listed.
~]$ oc get customresourcedefinition kialis.kiali.io --output yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
creationTimestamp: "2021-04-15T20:01:30Z"
deletionGracePeriodSeconds: 0
deletionTimestamp: "2022-09-16T15:42:35Z"
finalizers:
- customresourcecleanup.apiextensions.k8s.io
Remove the Operators
I then had to use the oc get operators command to return the istio, Jaegar, Kiali, Maistra, Service Mesh Operators.
~]$ oc get operators
NAME AGE
cluster-logging.openshift-logging 2y151d
elasticsearch-operator.openshift-operators-redhat 2y151d
jaeger-product.openshift-operators 2y72d
kiali-ossm.openshift-operators 3d18h
servicemeshoperator.openshift-operators 3d18h
And then I used the oc delete operator command to uninstall the Jaegar, Kiali, and Service Mesh Operators.
oc delete operator jaeger-product.openshift-operators
oc delete operator kiali-ossm.openshift-operators
oc delete operator servicemeshoperator.openshift-operators
And then I re-issued the oc get operators command to ensure the Jaegar, Kiali, and Service Mesh Operators were uninstalled.
~]$ oc get operators
NAME AGE
cluster-logging.openshift-logging 2y151d
elasticsearch-operator.openshift-operators-redhat 2y151d
Did you find this article helpful?
If so, consider buying me a coffee over at