Skip to content

Integrate with Prometheus

EMQX supports integration with third-party monitoring systems, such as Prometheus. It is the monitoring solution open-sourced by SoundCloud. It offers a versatile set of features including support for multidimensional data models, flexible query language, and powerful alarm management.

Using a third-party monitoring system can bring the following advantages:

  • A complete monitoring system, where the monitoring data of EMQX will be integrated with that of the other systems. For example, you can get the monitoring information of the server host;
  • More intuitive monitoring report with figures and charts, such as using Grafana dashboard to visualize the EMQX metrics;
  • Various alarm notification options, such as using Prometheus Alertmanager to set up alarm rules and notification methods.

EMQX supports two methods for integrating Prometheus metrics monitoring:

  • Pull Mode: Prometheus directly collects metrics through EMQX's REST API.
  • Push Mode: EMQX pushes metrics to the Pushgateway service, from which Prometheus collects the metrics.

This page introduces the configuration steps for both methods. You can click Management -> Monitoring in the left navigation menu of the EMQX Dashboard, and in the Integration tab, select Prometheus to perform the configuration. You can also click the Help button on the page to view specific configuration steps for each mode.

Configure Pull Mode Integration

EMQX provides the following REST APIs for Prometheus to collect system metrics:

  • /api/v5/prometheus/stats: Basic metrics and counters of EMQX.
  • /api/v5/prometheus/auth: Key metrics and counters in access control, including authentication and authorization.
  • /api/v5/prometheus/data_integration: Metrics and counters related to the rule engine, connectors, actions, Sink/Source, and encoding/decoding.

When calling the above APIs to obtain metrics, you can use the URL query parameter mode to get different types of metric data. The meanings of different parameters are as follows:

For more information about Prometheus pull endpoints, refer to the EMQX Open Source API documentation and EMQX Enterprise API documentation.

TIP

By default, the pull mode API does not require authentication. You can configure the Enable Basic Auth switch on the page to enable basic authentication for the interface. Once enabled, you need to create an API key on EMQX and apply it to the Prometheus configuration to obtain metric data.

Prometheus Configuration for Reference

yaml
# prometheus.yaml
global:
  scrape_interval:     10s # The default scrape interval is every 10 seconds.
  evaluation_interval: 10s # The default evaluation interval is every 10 seconds.
  # On this machine, every time series will be exported by default.
  external_labels:
    monitor: 'emqx-monitor'
scrape_configs:
  - job_name: 'emqx_stats'
    static_configs:
      - targets: ['127.0.0.1:18083']
    metrics_path: '/api/v5/prometheus/stats'
    scheme: 'http'
    basic_auth:
      username: ''
      password: ''

  - job_name: 'emqx_auth'
    static_configs:
      - targets: ['127.0.0.1:18083']
    metrics_path: '/api/v5/prometheus/auth'
    scheme: 'http'
    basic_auth:
      username: ''
      password: ''

  - job_name: 'emqx_data_integration'
    static_configs:
      - targets: ['127.0.0.1:18083']
    metrics_path: '/api/v5/prometheus/data_integration'
    scheme: 'http'
    basic_auth:
      username: ''
      password: ''

Configure Push Mode Integration

EMQX supports pushing metrics to Pushgateway, from which Prometheus can then collect these metrics. EMQX supports pushing metrics to Pushgateway, which is disabled by default. To enable the Pushgateway service, you can click the Enable Pushgateway toggle switch on the Prometheus configuration page in the Dashboard.

enable-push-gateway

Configure the following fields according to your business needs, and then click Save Changes.

  • Interval: Specify the time interval for reporting the monitoring metrics data to Pushgateway. The default value it 15 seconds.
  • Pushgateway Server: Type the URL of Prometheus server. It is http://127.0.0.1:9091 by default.
  • Job Name: Specify variables that include the EMQX node name and hostname. The default value is ${name}/instance/${name}~${host}. For example, when the EMQX node name is emqx@127.0.0.1, the name variable takes the value emqx and the host variable takes the value 127.0.0.1.
  • Headers: Type the key and value of the HTTP headers for the monitoring metrics that are pushed to Pushgateway. You can add a list of headers by clicking the Add button. The type is string, for example, { Authorization = "some-authz-tokens"}.

At the same time, you can click the Help button and refer to the steps on the Use Pushgateway tab for configuration.

TIP

The Push mode currently only includes EMQX's basic metrics and counters from the /api/v5/prometheus/stats endpoint, so the Pull mode is more recommended.

You can also enable and configure the Pushgateway by adding the following configurations to etc/emqx.conf. For more information on configuration items, see Configuration - Prometheus.

bash
prometheus {
  push_gateway_server = "http://127.0.0.1:9091"
  interval = 15s
  headers {}
  job_name = "${name}/instance/${name}~${host}"
}

Use Grafana to Visualize EMQX Metrics

You can also use Grafana with Prometheus to visualize EMQX metrics, which can be achieved by importing the EMQX template files into Grafana. To download the template, click EMQX | Grafana Dashboard or click the Help button at the bottom of the Integration tab of the Monitoring page.

TIP

For detailed operating steps, see Monitoring MQTT broker with Prometheus and Grafana