# Configure EMQX Logs Collection
# Task target
- How to collect EMQX cluster logs through Telegraf and export them to the standard output of the container
# Deploy telegraf-operator
Telegraf is a server-based agent for collecting and sending all metrics and events from databases, systems, and IoT sensors. It supports four types of plugins, including input, output, aggregator and processor. More articles about Telegraf can be found at: telegraf (opens new window), The documentation for telegraf-operator can be found in: telegraf-operator (opens new window)
Execute the following command to deploy telegraf-operator
helm repo add influxdata https://helm.influxdata.com/
helm upgrade --install telegraf-operator influxdata/telegraf-operator
2
# Global Configuration - classes
The global configuration is mounted via secret, specifying the class name as logs
agent
Used to configure telegraf agent, refer to the detailed definition: telegraf agent (opens new window)
inputs.tail
The tail plug-in used to configure the input is defined in detail in: tail (opens new window)
outputs.file
The file plug-in used to configure the output is defined in detail in: file (opens new window)
apiVersion: v1
kind: Secret
metadata:
name: telegraf-operator-classes
namespace: default
stringData:
logs: |+
[agent]
interval = "60s"
flush_jitter = "5s"
flush_interval = "15s"
debug = true
quiet = false
metric_batch_size = 128
metric_buffer_limit = 256
[[inputs.tail]]
files = ["/opt/emqx/log/emqx.log.[1-9]"]
from_beginning = false
max_undelivered_lines = 64
character_encoding = "utf-8"
data_format = "grok"
grok_patterns = ['^%{TIMESTAMP_ISO8601:timestamp:ts-"2006-01-02T15:04:05.999999999-07:00"} \[%{LOGLEVEL:level}\] (?m)%{GREEDYDATA:messages}$']
[inputs.tail.tags]
collection = "log"
[[outputs.file]]
files = ["stdout"]
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
Save the above as: classes.yaml
- Create secret
kubectl apply -f classes.yaml
- Check the created secret
kubectl get secret telegraf-operator-classes
The output is similar to:
NAME TYPE DATA AGE
telegraf-operator-classes Opaque 1 11h
2
# Deploy EMQX cluster
Telegraf uses annotations to inject sidecar for Pod log collection, for a detailed definition of annotations refer to the documentation: telegraf annotations (opens new window)
Save the above as: emqx-telegraf.yaml
- Deploy EMQX cluster
kubectl apply -f emqx-telegraf.yaml
- Checking EMQX Cluster Status
- Check the collected logs
The collected EMQX logs are output to the standard output.