EMQX Backup Sync

This plugin periodically synchronizes selected backup data from a primary EMQX cluster to a secondary EMQX cluster by using the existing Data Backup APIs.

The secondary cluster calls the primary cluster to export a backup file, then downloads that file, uploads it locally, and imports it. Selected configuration roots are imported with EMQX's existing configuration import semantics. Selected Mnesia table sets are imported as a snapshot, so records in those table sets that only exist on the secondary cluster are removed. Roots and table sets that are not selected are left unchanged.

Configuration

Install and start the plugin on each secondary cluster. The primary cluster does not need this plugin installed; it only needs Dashboard Data Backup APIs reachable from the secondary cluster.

primary {
  base_url = "https://primary.example.com:18083/api/v5"
  api_key = "sync-key"
  api_secret = "sync-secret"
  ssl {
    enable = true
    server_name_indication = "primary.example.com"
    verify = "verify_peer"
    cacertfile = "/etc/emqx/certs/primary-ca.pem"
    certfile = ""
    keyfile = ""
  }
}

sync {
  interval = "5m"
  root_keys = [
    "connectors",
    "actions",
    "sources",
    "rule_engine",
    "listeners",
    "schema_registry"
  ]
  table_sets = [
    "banned",
    "builtin_authn",
    "builtin_authz"
  ]
  timeout = "30s"
  retain_remote_backup = false
  retain_backup_after_import = true
}

The configured API key must be allowed to access Data Backup endpoints on the primary cluster. primary.api_key and primary.api_secret can be set directly or as file:// paths, for example file:///etc/emqx/backup-sync-api-key.

Supported sync.root_keys values are connectors, actions, sources, rule_engine, listeners, schema_registry, authentication, and authorization.

Rules commonly depend on connectors, actions, sources, and schema registry objects. If rule_engine is synchronized without its dependencies, imports may fail or create incomplete runtime behavior. Include those dependent roots in sync.root_keys unless they already exist on the secondary cluster.

By default, synchronization also includes the banned, builtin_authn, and builtin_authz table sets. These selected table sets are replaced on the secondary cluster. Set sync.table_sets = [] when configuration-only synchronization is required. Supported sync.table_sets values are banned, builtin_authn, builtin_authz, builtin_retainer, psk, and mt. The primary Data Backup API does not include dashboard_users or api_keys when called with an API key.

CLI

Use the following command on a secondary node to inspect the local sync worker:

emqx ctl backup_sync status

The command prints the local node, health, worker state, selected core node, next scheduled sync, and the non-sensitive sync configuration.

Download

Tarballs for each EMQX release:

EMQX Version	Plugin Version	Package
6.3.0	0.1.3	emqx_backup_sync-0.1.3.tar.gz (sha256)
6.3.1	0.1.3	emqx_backup_sync-0.1.3.tar.gz (sha256)