OpenShift - Fluentd logs (ELK EFK stack)

This diagram illustrates the systems that are typically used to parse log data on OpenShift. Fluentd collects the log data from the containers and passes the log data onto Elastic Search. Optionally, Kibana can be used as a tool that may make it easier to visualize the logs.

This is similar to the ELK stack (Elastic Search, Logstash, Kibana), but would actually be the EFK stack (Elastic Search, Fluentd, Kibana).


The oc get pods command can be used list the Fluentd pods in the openshift-logging project.

~]# oc get pods --namespace openshift-logging
NAME                                            READY   STATUS      RESTARTS   AGE
cluster-logging-operator-7f65964859-gtlvv       1/1     Running     0          50d
curator-1622518200-qpnfc                        0/1     Error       0          9d
curator-1623295800-h658p                        0/1     Completed   0          22h
elasticsearch-cdm-dm8dl5ki-1-5d4d54988f-qzz4x   2/2     Running     0          50d
elasticsearch-cdm-dm8dl5ki-2-674f9db4c6-k4r2x   2/2     Running     0          37d
elasticsearch-cdm-dm8dl5ki-3-7d55fbfbff-8ssnk   2/2     Running     0          50d
elasticsearch-im-app-1623377700-2xnxv           0/1     Completed   0          7m57s
elasticsearch-im-audit-1623377700-bsrcl         0/1     Completed   0          7m57s
elasticsearch-im-infra-1623377700-5ltdm         0/1     Completed   0          7m57s
fluentd-22pbq                                   1/1     Running     0          50d
fluentd-44v9v                                   1/1     Running     0          50d
fluentd-6lpwh                                   1/1     Running     0          50d
fluentd-89xsl                                   1/1     Running     0          50d
fluentd-995zv                                   1/1     Running     0          50d
fluentd-b5vj7                                   1/1     Running     0          50d
fluentd-bc4zg                                   1/1     Running     0          50d
fluentd-br7ft                                   1/1     Running     0          50d
fluentd-cmnqr                                   1/1     Running     0          50d
fluentd-gc6zv                                   1/1     Running     0          50d
fluentd-gl68p                                   1/1     Running     0          50d
fluentd-gplgt                                   1/1     Running     0          50d
fluentd-kbvx8                                   1/1     Running     0          50d
fluentd-kgzvm                                   1/1     Running     0          50d
fluentd-kzpjk                                   1/1     Running     0          50d
fluentd-nbm9v                                   1/1     Running     0          50d
fluentd-pd287                                   1/1     Running     0          50d
fluentd-rml9r                                   1/1     Running     0          50d
fluentd-vj7mw                                   1/1     Running     0          50d
fluentd-vp5jq                                   1/1     Running     0          50d
fluentd-x5j5g                                   1/1     Running     1          50d
fluentd-xl257                                   1/1     Running     0          50d
fluentd-xpw7s                                   1/1     Running     0          50d
fluentd-xttg7                                   1/1     Running     0          50d
fluentd-zdn6j                                   1/1     Running     0          50d
fluentd-zh2vc                                   1/1     Running     0          50d
kibana-7b676c4bf8-d9t6w                         2/2     Running     0          50d


The oc logs command can be used to view the logs in the fluentd pods. These are not the logs that fluentd collects from your OpenShfit pods.

oc logs fluentd-22pbq -n openshift-logging


Fluentd uses the Linux journal daemon (journald) to collect log data. The following command can be used to list the journal log files. Of course, replace b03724ec5eda4cc3b6f845a1a1d7cd41 with whatever string is displayed.

oc exec fluentd-22pbq -- tail /var/log/journal/b03724ec5eda4cc3b6f845a1a1d7cd41


Something like this should be returned. The cat command could then be used to read each .journal file.

-rw-r-----+ 1 root systemd-journal  58720256 Jun 11 11:58 system.journal
-rw-r-----+ 1 root systemd-journal 134217728 Jun 11 04:09 system@c622def313224131b5b2a9733c20a557-000000001e9f33cd-0005c47575fb8abb.journal
-rw-r-----+ 1 root systemd-journal 134217728 Jun 11 04:24 system@c622def313224131b5b2a9733c20a557-000000001ea1136a-0005c475ac6da6b6.journal
-rw-r-----+ 1 root systemd-journal 134217728 Jun 11 04:38 system@c622def313224131b5b2a9733c20a557-000000001ea2f2f5-0005c475e4028a83.journal



Did you find this article helpful?

If so, consider buying me a coffee over at Buy Me A Coffee

Add a Comment

We will never share your name or email with anyone. Enter your email if you would like to be notified when we respond to your comment.

Please enter 49541 in the box below so that we can be sure you are a human.


Web design by yours truely - me, myself, and I   |   |