Bootstrap FreeKB - OpenShift - Elastic Search logs (ELK EFK stack)
OpenShift - Elastic Search logs (ELK EFK stack)

Updated:   |  OpenShift articles

This diagram illustrates the systems that are typically used to parse log data on OpenShift. Fluentd collects the log data from the containers and passes the log data onto Elastic Search. Optionally, Kibana can be used as a tool that may make it easier to visualize the logs.

This is similar to the ELK stack (Elastic Search, Logstash, Kibana), but would actually be the EFK stack (Elastic Search, Fluentd, Kibana).

 

The oc get pods command can be used list the Elastic Search pods in the openshift-logging project.

oc get pods -n openshift-logging

 

Something like this should be returned.

NAME                                            READY   STATUS      RESTARTS   AGE
cluster-logging-operator-7f65964859-gtlvv       1/1     Running     0          50d
curator-1622518200-qpnfc                        0/1     Error       0          9d
curator-1623295800-h658p                        0/1     Completed   0          22h
elasticsearch-cdm-dm8dl5ki-1-5d4d54988f-qzz4x   2/2     Running     0          50d
elasticsearch-cdm-dm8dl5ki-2-674f9db4c6-k4r2x   2/2     Running     0          37d
elasticsearch-cdm-dm8dl5ki-3-7d55fbfbff-8ssnk   2/2     Running     0          50d
elasticsearch-im-app-1623377700-2xnxv           0/1     Completed   0          7m57s
elasticsearch-im-audit-1623377700-bsrcl         0/1     Completed   0          7m57s
elasticsearch-im-infra-1623377700-5ltdm         0/1     Completed   0          7m57s
fluentd-22pbq                                   1/1     Running     0          50d
fluentd-44v9v                                   1/1     Running     0          50d
fluentd-6lpwh                                   1/1     Running     0          50d
fluentd-89xsl                                   1/1     Running     0          50d
fluentd-995zv                                   1/1     Running     0          50d
fluentd-b5vj7                                   1/1     Running     0          50d
fluentd-bc4zg                                   1/1     Running     0          50d
fluentd-br7ft                                   1/1     Running     0          50d
fluentd-cmnqr                                   1/1     Running     0          50d
fluentd-gc6zv                                   1/1     Running     0          50d
fluentd-gl68p                                   1/1     Running     0          50d
fluentd-gplgt                                   1/1     Running     0          50d
fluentd-kbvx8                                   1/1     Running     0          50d
fluentd-kgzvm                                   1/1     Running     0          50d
fluentd-kzpjk                                   1/1     Running     0          50d
fluentd-nbm9v                                   1/1     Running     0          50d
fluentd-pd287                                   1/1     Running     0          50d
fluentd-rml9r                                   1/1     Running     0          50d
fluentd-vj7mw                                   1/1     Running     0          50d
fluentd-vp5jq                                   1/1     Running     0          50d
fluentd-x5j5g                                   1/1     Running     1          50d
fluentd-xl257                                   1/1     Running     0          50d
fluentd-xpw7s                                   1/1     Running     0          50d
fluentd-xttg7                                   1/1     Running     0          50d
fluentd-zdn6j                                   1/1     Running     0          50d
fluentd-zh2vc                                   1/1     Running     0          50d
kibana-7b676c4bf8-d9t6w                         2/2     Running     0          50d

 

The oc logs command can be used to view the logs in the Elastic Search pods.

oc logs elasticsearch-cdm-dm8dl5ki-1-5d4d54988f-qzz4x --container elasticsearch -n openshift-logging

 

Elastic Search organized the log data from Fluentd into datastores called indices. The following command can be used to list the indices in each Elastic Search pod.

oc exec elasticsearch-cdm-dm8dl5ki-1-5d4d54988f-qzz4x --container elasticsearch -- es_util --query=_cat/indices?v 

 

Something like this should be returned.

health status index                            uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   .kibana_-911233311_c007137_7139  0CTseHnsQFKw6WxzFsKOug   1   1          2            0     28.4kb         14.2kb
green  open   .kibana_-905684497_c067937_67938 554X5_58QfSRSd3YtxufHQ   1   1          1            0      7.4kb          3.7kb
green  open   infra-001509                     Nv1IDefPSImOYnaZxuN45A   3   1   13511117            0     16.6gb          8.3gb
green  open   audit-000195                     V4_7fuCHSZ6Nzh3Do6Ol9w   3   1          0            0      1.5kb           783b
green  open   app-001529                       VGu-dSpJT2mF23ELVg05WQ   3   1     916929            0        1gb        524.8mb

 

Or the only list the infra indices.

oc exec elasticsearch-cdm-dm8dl5ki-1-5d4d54988f-qzz4x --container elasticsearch -- es_util --query=_cat/indices/infra

 

 




Did you find this article helpful?

If so, consider buying me a coffee over at Buy Me A Coffee



Comments


Add a Comment


Please enter c8e14e in the box below so that we can be sure you are a human.