Loki is often used with Vector, where Vector is the collector that collects data and Loki stores the data is some sort of storage, such as an Amazon Web Services (AWS) S3 Bucket. Loki is meant for short-term storage of log data and is optimized for performance. Prior to Vector, Fluentd was often used as the log collector.
You configure logging by first installing the Loki Operator to manage your log storage followed by the OpenShift Logging Operator to manage the components of logging.
Create the openshift-operators-redhat namespace
For example, let's say you have a file named openshift-operators-redhat.yml with the following YAML.
apiVersion: v1
kind: Namespace
metadata:
name: openshift-operators-redhat
annotations:
openshift.io/node-selector: ""
labels:
openshift.io/cluster-monitoring: "true"
The oc apply command can be used to create the openshift-operators-redhat namespace.
oc apply -f openshift-operators-redhat.yml
Create the loki-operator Subscription
For example, let's say you have a file named loki-subscription.yml with the following YAML.
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
name: loki-operator
namespace: openshift-operators-redhat
spec:
channel: stable
name: loki-operator
source: redhat-operators
sourceNamespace: openshift-marketplace
The oc apply command can be used to create the Subscription.
oc apply -f loki-subscription.yml
Create the openshift-logging namespace
For example, let's say you have a file named openshift-logging.yml with the following YAML.
apiVersion: v1
kind: Namespace
metadata:
name: openshift-logging
annotations:
openshift.io/node-selector: ""
labels:
openshift.io/cluster-logging: "true"
openshift.io/cluster-monitoring: "true"
The oc apply command can be used to create the openshift-operators-redhat namespace.
oc apply -f openshift-logging.yml
Create the cluster-logging OperatorGroup
For example, let's say you have a file named cluster-logging.yml with the following YAML.
apiVersion: v1
kind: Namespace
metadata:
name: openshift-logging
annotations:
openshift.io/node-selector: ""
labels:
openshift.io/cluster-logging: "true"
openshift.io/cluster-monitoring: "true"
The oc apply command can be used to create the cluster-logging OperatorGroup.
oc apply -f cluster-logging.yml
Create the cluster-logging Subscription
For example, let's say you have a file named loki-operator-subscription.yml with the following YAML.
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
name: loki-operator
namespace: openshift-operators-redhat
spec:
channel: stable
name: loki-operator
source: redhat-operators
sourceNamespace: openshift-marketplace
The oc apply command can be used to create the loki-operator Subscription.
oc apply -f loki-operator-subscription.yml
Create the loki storage secret
Create a secret that will be used by the LokiStack to know where to store log data. For example, let's say you have the following YAML in a file named loki-s3.yml.
apiVersion: v1
stringData:
access_key_id: AKIA2MABCD6GDQ1234RY
access_key_secret: 4FGkm30sdf-0m234dfAVMAD2340-dsfaADV324df
bucketnames: my-bucket-abc123
endpoint: https://s3.us-east-1.amazonaws.com
region: us-east-1
kind: Secret
metadata:
creationTimestamp: "2024-09-17T21:28:09Z"
name: loki-s3
namespace: openshift-logging
type: Opaque
The oc apply command can be used to create the secret.
oc apply -f loki-s3.yml
Create the LokiStack Custom Resource
For example, let's say you have the following YAML in a file named loki-stack.yml. Notice in this example that the LokiStack contains the loki-s3 secret so that the LokiStack knows to store log data in the Amazon Web Service S3 Bucket.
apiVersion: loki.grafana.com/v1
kind: LokiStack
metadata:
name: loki
namespace: openshift-logging
spec:
limits:
global:
ingestion:
ingestionBurstSize: 40
ingestionRate: 20
maxGlobalStreamsPerTenant: 25000
queries:
maxChunksPerQuery: 2000000
maxEntriesLimitPerQuery: 10000
maxQuerySeries: 3000
queryTimeout: 3m
retention:
days: 7
managementState: Managed
size: 1x.extra-small
storage:
schemas:
- effectiveDate: "2024-07-01"
version: v13
secret:
name: loki-s3
type: s3
storageClassName: thin-csi
template:
compactor:
nodeSelector:
node-type: openshift-logging
distributor:
nodeSelector:
node-type: openshift-logging
gateway:
nodeSelector:
node-type: openshift-logging
indexGateway:
nodeSelector:
node-type: openshift-logging
ingester:
nodeSelector:
node-type: openshift-logging
querier:
nodeSelector:
node-type: openshift-logging
queryFrontend:
nodeSelector:
node-type: openshift-logging
ruler:
nodeSelector:
node-type: openshift-logging
tenants:
mode: openshift-network
The oc apply command can be used to create the LokiStack Custom Resource.
oc apply -f loki-stack.yml
Create the ClusterLogging Custom Resource
For example, let's say you have the following YAML in a file named cluster-logging.yml.
apiVersion: logging.openshift.io/v1
kind: ClusterLogging
metadata:
name: instance
namespace: openshift-logging
spec:
collection:
type: vector
logStore:
lokistack:
name: logging-loki
type: lokistack
visualization:
type: ocp-console
ocpConsole:
logsLimit: 15
managementState: Managed
The oc apply command can be used to create the ClusterLogging Custom Resource.
oc apply -f cluster-logging.yml
List the Operators
The oc get operators command can be used to list the installed Operators. In this example, the Loki and OpenShift Cluster Logging Operators are listed. Awesome!
~]$ oc get operators
NAME AGE
cluster-logging.openshift-logging 181d
loki-operator.openshift-operators-redhat 181d
Assuming the Loki Operator was installed in the openshift-logging namespace, the oc get all command can be used list the Loki pods, services, daemon set, deployments, replica sets, stateful sets and route.
~]$ oc get all --namespace openshift-logging
NAME READY STATUS RESTARTS AGE
pod/flowlogs-pipeline-2vzbx 1/1 Running 10 61d
pod/flowlogs-pipeline-4w4mj 1/1 Running 8 75d
pod/flowlogs-pipeline-fcpx6 1/1 Running 10 75d
pod/flowlogs-pipeline-g599l 1/1 Running 9 125d
pod/flowlogs-pipeline-hn2jx 1/1 Running 8 125d
pod/flowlogs-pipeline-kfn77 1/1 Running 10 125d
pod/flowlogs-pipeline-p75r5 1/1 Running 11 61d
pod/flowlogs-pipeline-phbc2 1/1 Running 11 61d
pod/flowlogs-pipeline-qpsb7 1/1 Running 12 125d
pod/flowlogs-pipeline-rw76z 1/1 Running 1 23d
pod/flowlogs-pipeline-xjnn4 1/1 Running 10 75d
pod/flowlogs-pipeline-xtp62 1/1 Running 1 23d
pod/flowlogs-pipeline-z4r99 1/1 Running 12 75d
pod/loki-compactor-0 1/1 Running 0 18d
pod/loki-distributor-65fdc984f6-7hwc5 1/1 Running 0 18d
pod/loki-distributor-65fdc984f6-dqwcc 1/1 Running 0 18d
pod/loki-gateway-775f67b766-8tx2d 2/2 Running 0 18d
pod/loki-gateway-775f67b766-gzvgs 2/2 Running 0 18d
pod/loki-index-gateway-0 1/1 Running 0 18d
pod/loki-index-gateway-1 1/1 Running 0 18d
pod/loki-ingester-0 1/1 Running 0 18d
pod/loki-ingester-1 1/1 Running 0 18d
pod/loki-querier-c7f847457-5sp9h 1/1 Running 0 18d
pod/loki-querier-c7f847457-l7kgz 1/1 Running 0 18d
pod/loki-query-frontend-7d6dbbd7d9-d4snw 1/1 Running 0 18d
pod/loki-query-frontend-7d6dbbd7d9-klqv5 1/1 Running 0 18d
pod/foo-plugin-55f86487c4-d4n7b 1/1 Running 0 23d
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/flowlogs-pipeline-prom ClusterIP 172.30.163.134 <none> 9401/TCP 167d
service/loki-compactor-grpc ClusterIP None <none> 9095/TCP 162d
service/loki-compactor-http ClusterIP 172.30.125.165 <none> 3100/TCP 162d
service/loki-distributor-grpc ClusterIP None <none> 9095/TCP 162d
service/loki-distributor-http ClusterIP 172.30.234.132 <none> 3100/TCP 162d
service/loki-gateway-http ClusterIP 172.30.21.134 <none> 8080/TCP,8081/TCP,8083/TCP 162d
service/loki-gossip-ring ClusterIP None <none> 7946/TCP 162d
service/loki-index-gateway-grpc ClusterIP None <none> 9095/TCP 162d
service/loki-index-gateway-http ClusterIP 172.30.88.235 <none> 3100/TCP 162d
service/loki-ingester-grpc ClusterIP None <none> 9095/TCP 162d
service/loki-ingester-http ClusterIP 172.30.226.16 <none> 3100/TCP 162d
service/loki-querier-grpc ClusterIP None <none> 9095/TCP 162d
service/loki-querier-http ClusterIP 172.30.108.85 <none> 3100/TCP 162d
service/loki-query-frontend-grpc ClusterIP None <none> 9095/TCP 162d
service/loki-query-frontend-http ClusterIP 172.30.181.211 <none> 3100/TCP 162d
service/foo-plugin ClusterIP 172.30.90.11 <none> 9001/TCP 167d
service/foo-plugin-metrics ClusterIP 172.30.144.186 <none> 9002/TCP 167d
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/flowlogs-pipeline 13 13 13 13 13 <none> 167d
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/loki-distributor 2/2 2 2 162d
deployment.apps/loki-gateway 2/2 2 2 162d
deployment.apps/loki-querier 2/2 2 2 162d
deployment.apps/loki-query-frontend 2/2 2 2 162d
deployment.apps/foo-plugin 1/1 1 1 167d
NAME DESIRED CURRENT READY AGE
replicaset.apps/loki-distributor-65fdc984f6 2 2 2 18d
replicaset.apps/loki-gateway-775f67b766 2 2 2 18d
replicaset.apps/loki-querier-c7f847457 2 2 2 18d
replicaset.apps/loki-query-frontend-7d6dbbd7d9 2 2 2 18d
replicaset.apps/foo-plugin-55f86487c4 1 1 1 133d
NAME READY AGE
statefulset.apps/loki-compactor 1/1 162d
statefulset.apps/loki-index-gateway 2/2 162d
statefulset.apps/loki-ingester 2/2 162d
NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD
route.route.openshift.io/loki loki-openshift-logging.apps.openshift.example.com loki-gateway-http public reencrypt None
Viewing Logs
The oc logs command can be used to view logs for a particular resource, such as a pod, a service, a replica set, and so on
The oc node-logs command can be used to view node logs
Did you find this article helpful?
If so, consider buying me a coffee over at 