# Common Configuration

# Replicas

Specify the number of EMQX instances

Example:

spec:
  replicas: 3
1
2

Field Description:

FieldDefaultDescription
replicas3Number of EMQX instances

# Environment Variables

Used to set the environment variables of the instance

Example:

spec:
  env:
    - name: Foo
      value: Bar
1
2
3
4

Field Description:

FieldDescription
.spec.env.namevariable name
.spec.env.valuevariable value

# Image pull secret

Example:

spec:
  imagePullSecrets: [fake-secrets]
1
2

# Node Configuration

  • nodeName

nodeName is a more direct form of node selection than affinity or nodeSelector.

If the nodeName field is not empty, the scheduler ignores the Pod and the kubelet on the named node tries to place the Pod on that node. Using nodeName overrules using nodeSelector or affinity and anti-affinity rules.

Example:

spec:
  nodeName: kube-01
1
2

Schedule to node kube-01

  • nodeSelector

nodeSelector is the simplest recommended form of node selection constraint. Kubernetes only schedules the Pod onto nodes that have each of the labels you specify.

Example:

spec:
  nodeSelector:
    key: value
1
2
3

Schedule to the node which is labeled with key=value

# Node Affinity

Node affinity is conceptually similar to nodeSelector, allowing you to constrain which nodes your Pod can be scheduled on based on node labels.

Example:

spec:
  affinity: [config of affinity]
1
2

Please refer to Kubernetes Docs (opens new window)

# Tolerations

Tolerations are applied to pods. Tolerations allow the scheduler to schedule pods with matching taints.

Taints and tolerations work together to ensure that pods are not scheduled onto inappropriate nodes.

Example:

spec:
  toleRations:
    - key: "key"
      operator: "Equal"
      value: "value"
      effect: "NoSchedule"
1
2
3
4
5
6

Please refer to Kubernetes Docs (opens new window)

# Persistence

pvc configuration

Example:

spec:
  persistent:
    storageClassName: standard
    resources:
      requests:
        storage: 20Mi
    accessModes:
      - ReadWriteOnce
1
2
3
4
5
6
7
8
FieldDefaultDescription
.spec.persistent.storageClassNamestandardthe name of storage class
.spec.persistent.resources.requests.storage20Mistorage size
.spec.persistent.accessModesReadWriteOnceaccess mode,only support ReadWriteOnce

# Init Containers

Init containers can contain utilities or setup scripts not present in an app image.

A Pod can have multiple containers running apps within it, but it can also have one or more init containers, which are run before the app containers are started.

Example:

spec:
  initContainers:
    name: busybox
    image: busybox:stable
    securityContext:
      runAsUser: 0
      runAsGroup: 0
      capabilities:
        add:
        - SYS_ADMIN
        drop:
        - ALL
    command:
      - /bin/sh
      - -c
      - |
        mount -o remount rw /proc/sys
        sysctl -w net.core.somaxconn=65535
        sysctl -w net.ipv4.ip_local_port_range="1024 65535"
        sysctl -w kernel.core_uses_pid=0
        sysctl -w net.ipv4.tcp_tw_reuse=1
        sysctl -w fs.nr_open=1000000000
        sysctl -w fs.file-max=1000000000
        sysctl -w net.ipv4.ip_local_port_range='1025 65534'
        sysctl -w net.ipv4.udp_mem='74583000 499445000 749166000'
        sysctl -w net.ipv4.tcp_max_sync_backlog=163840
        sysctl -w net.core.netdev_max_backlog=163840
        sysctl -w net.core.optmem_max=16777216
        sysctl -w net.ipv4.tcp_rmem='1024 4096 16777216'
        sysctl -w net.ipv4.tcp_wmem='1024 4096 16777216'
        sysctl -w net.ipv4.tcp_max_tw_buckets=1048576
        sysctl -w net.ipv4.tcp_fin_timeout=15
        sysctl -w net.core.rmem_default=262144000
        sysctl -w net.core.wmem_default=262144000
        sysctl -w net.core.rmem_max=262144000
        sysctl -w net.core.wmem_max=262144000
        sysctl -w net.ipv4.tcp_mem='378150000  504200000  756300000'
        sysctl -w net.netfilter.nf_conntrack_max=1000000
        sysctl -w net.netfilter.nf_conntrack_tcp_timeout_time_wait=30
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

This configuration above is used to perform kernel and network optimizations for the EMQX container

# Extra Containers

Similar to the side-car container, it can run simultaneously with the EMQX container and be used to process the user-defined routines.

Example:

spec:
  extraContainers:
    - name: extra
      image: busybox:stable
      command:
        - /bin/sh
        - -c
        - |
          tail -f /dev/null
1
2
3
4
5
6
7
8
9

# EMQX Template

# EMQX Dashboard

Dashboard Account configurations

Example:

spec:
  emqxTemplate:
    username: "admin"
    password: "public"
1
2
3
4

Field Description:

FieldDefaultDescription
.spec.emqxTemplate.imageusernameadminusername
.spec.emqxTemplate.imagepasswordpublicpassword

# Image configuration

specify image and pull policy

Example:

spec:
  emqxTemplate:
    image: emqx/emqx-ee:4.4.8
    imagePullPolicy: IfNotPresent
1
2
3
4

Field Description:

FieldDefaultDescription
.spec.emqxTemplate.imageimage address
.spec.emqxTemplate.imagePullPolicyIfNotPresentIfNotPresent: the image is pulled only if it is not already present locally.

Please refer to Kubernetes Docs (opens new window)

# Security Context

A security context defines privilege and access control settings for a Pod or Container.

Example:

spec:
  emqxTemplate:
    securityContext:
      runAsUser: 1000
      runAsGroup: 1000
      fsGroup: 1000
      fsGroupChangePolicy: Always
1
2
3
4
5
6
7

Please refer to Kubernetes Docs (opens new window)

# Extra Volumes

Mount extra volumes, eg: secret or configmap

Example:

spec:
  emqxTemplate:
    extraVolumes:
      - name: fake-volume
        emptyDir: {}
    extraVolumeMounts:
      - name: fake-volume
        mountPath: /tmp/fake
1
2
3
4
5
6
7
8

Please refer to issue (opens new window)

# EMQX Entrypoint

The entrypoint for the EMQX container, or if not provided, then the CMD for the EMQX image is used.

Example:

spec:
  emqxTemplate:
    args:
      - bash
      - -c
      - |
        printenv | grep "^EMQX_"
        emqx foreground
1
2
3
4
5
6
7
8

Please refer to Kubernetes Docs (opens new window)

# EMQX Configurations

Example:

spec:
  emqxTemplate:
    config:
      name: emqx-ee
      cluster.discovery: dns
      cluster.dns.type: srv
      cluster.dns.app: emqx-ee
      cluster.dns.name: emqx-ee-headless.default.svc.cluster.local
      listener.tcp.external: "1883"
1
2
3
4
5
6
7
8
9

Note: spec.env override spec.emqxTemplate.config

Please refer to EMQX Docs (opens new window)

# ACL

EMQX ACL configuration

Example:

spec:
  emqxTemplate:
    - "{allow, all}."
1
2
3

Please refer to EMQX Docs (opens new window)

# EMQX Modules

Example:

spec:
  emqxTemplate:
    modules:
      - name: "emqx_mod_acl_internal"
        enable: true
      - name: "emqx_mod_presence"
        enable: true
1
2
3
4
5
6
7

Please refer to EMQX Docs (opens new window)

# Probes

  • readinessProbe

Periodically check the readiness of the EMQX container

Example:

spec:
  emqxTemplate:
    readinessProbe:
      httpGet:
        path: /status
        port: 8081
      initialDelaySeconds: 10
      periodSeconds: 5
      failureThreshold: 12
1
2
3
4
5
6
7
8
9

Field Description:

FieldDefaultDescription
.spec.emqxTemplate.readinessProbe.httpGet.pathPath to access on the HTTP serve
.spec.emqxTemplate.readinessProbe.httpGet.portName or number of the port to access on the container
.spec.emqxTemplate.readinessProbe.initialDelaySeconds0The initialDelaySeconds field tells the kubelet that it should wait 10 seconds before performing the first probe
.spec.emqxTemplate.readinessProbe.periodSeconds10The periodSeconds field specifies that the kubelet should perform a liveness probe every 5 seconds.
.spec.emqxTemplate.readinessProbe.failureThreshold3When a probe fails, Kubernetes will try failureThreshold times before giving up.
  • livenessProbe

Periodically check the liveness of the EMQX container

Example:

spec:
  emqxTemplate:
    livenessProbe:
      httpGet:
        path: /status
        port: 8081
      initialDelaySeconds: 60
      periodSeconds: 30
      failureThreshold: 3
1
2
3
4
5
6
7
8
9

Field Description:

FieldDefaultDescription
.spec.emqxTemplate.livenessProbe.httpGet.pathPath to access on the HTTP serve
.spec.emqxTemplate.livenessProbe.httpGet.portName or number of the port to access on the container
.spec.emqxTemplate.livenessProbe.initialDelaySeconds0The initialDelaySeconds field tells the kubelet that it should wait 60 seconds before performing the first probe
.spec.emqxTemplate.livenessProbe.periodSeconds10The periodSeconds field specifies that the kubelet should perform a liveness probe every 30 seconds.
.spec.emqxTemplate.livenessProbe.failureThreshold3When a probe fails, Kubernetes will try failureThreshold times before giving up.
  • startupProbe

Check if the EMQX container started successfully

Example:

spec:
  emqxTemplate:
    startupProbe:
      httpGet:
        path: /status
        port: 8081
      initialDelaySeconds: 10
      periodSeconds: 5
      failureThreshold: 12
1
2
3
4
5
6
7
8
9

Field Description:

FieldDefaultDescription
.spec.emqxTemplate.startupProbe.httpGet.pathPath to access on the HTTP serve
.spec.emqxTemplate.startupProbe.httpGet.portName or number of the port to access on the container
.spec.emqxTemplate.startupProbe.initialDelaySeconds0The initialDelaySeconds field tells the kubelet that it should wait 10 seconds before performing the first probe
.spec.emqxTemplate.startupProbe.periodSeconds10The periodSeconds field specifies that the kubelet should perform a liveness probe every 5 seconds.
.spec.emqxTemplate.startupProbe.failureThreshold3When a probe fails, Kubernetes will try failureThreshold times before giving up.

Please refer to Kubernetes Docs (opens new window)

# Resource Configurations

Cpu and memory of EMQX pod configurations

Example:

spec:
  emqxTemplate:
    resources:
      requests:
        memory: "64Mi"
        cpu: "125m"
      limits:
        memory: "1024Mi"
        cpu: "500m"
1
2
3
4
5
6
7
8
9

Field Description:

FieldDescription
.spec.emqxTemplate.resources.requestsresource requests
.spec.emqxTemplate.resources.requests.limitsresource limits

Please refer to Kubernetes Docs (opens new window)

# Service Template

EMQX service template configurations

Example:

spec:
  emqxTemplate:
    metadata:
      name: emqx-ee
      namespace: default
      labels:
        "apps.emqx.io/instance": "emqx-ee"
    spec:
      type: ClusterIP
      selector:
        "apps.emqx.io/instance": "emqx-ee"
      ports:
        - name: "http-management-8081"
          port: 8081
          protocol: "TCP"
          targetPort: 8081
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16