# Configuration Files

Users can configure EMQX with configuration files or environment variables. This section mainly introduces the EMQX configuration files and provides the basic configuration instructions for the most commonly used functions in EMQX. For comprehensive configuration items with detailed explanations, see [EMQX Enterprise Configuration Manual](https://docs.emqx.com/en/enterprise/v6.3.1/hocon/).

## Config Directories

After EMQX is installed, it creates a set of directories to manage its configuration and runtime data. These directories are separated into two main categories:

- **Static Configuration Directory (`etc`)**: Ready-only and contains immutable or static configuration files.
- **Dynamic Configuration Directory (`data/configs`)**: Writable and stores runtime-generated or dynamically updated configuration files.

### Static Configuration Directory (`etc`)

The `etc` directory holds the configuration files that define EMQX's initial setup. These files are typically modified during deployment or upgrades and are read-only at runtime to ensure stability. The location of the `etc` directory depends on the installation method:

| Installation                               | Path            |
| ------------------------------------------ | --------------- |
| Installed with RPM or DEB package          | `/etc/emqx`     |
| Running in docker container                | `/opt/emqx/etc` |
| Extracted from portable compressed package | `./etc`         |

Starting from EMQX 6.3.0, RPM and DEB installations include an `/opt/emqx/etc` symlink to `/etc/emqx`.

### Dynamic Configuration Directory (`data/configs`)

At runtime, EMQX allows dynamic reconfiguration through the Dashboard, REST API, or CLI. Changes made using these tools are stored in the `data/configs` directory to ensure persistence across sessions. The location of this directory also depends on the installation method:

| Installation                               | Path                     |
| ------------------------------------------ | ------------------------ |
| Installed with RPM or DEB package          | `/var/lib/emqx/configs`  |
| Running in docker container                | `/opt/emqx/data/configs` |
| Extracted from portable compressed package | `./data/configs`         |

Starting from EMQX 6.3.0, RPM and DEB installations include an `/opt/emqx/data` symlink to `/var/lib/emqx`. Therefore, `/opt/emqx/data/configs` resolves to `/var/lib/emqx/configs`. Configuring a custom data directory does not update the symlink.

::: tip
It is possible to change the data directory by modifying the `node.data_dir` setting in the configuration or the `EMQX_NODE__DATA_DIR` environment variable. However, when running a cluster, all nodes must use the same directory path.
:::

Although not encouraged, the content of the configuration files can overlap. In case of overlapping, the conflict is resolved by a predefined override rule, see [Config Override Rules](#config-override-rules).

## Config Examples

While the [Schema](#schema) section provides a detailed reference, configuration examples can be helpful for understanding and applying settings in EMQX.

- If you installed EMQX using RPM or DEB packages, you can find configuration examples in the `etc/emqx/examples` directory.
- If you are running EMQX in a Docker container, you can find configuration examples in the `opt/emqx/etc/examples` directory.

## Base Configuration File

Starting from EMQX 5.8.4, there is a base configuration file named `base.hocon` in the `etc` directory. This file contains default settings that can be overridden by higher-level configuration files at runtime.

For example, you may want to start the deployment with a basic authentication configuration, and then override it with a more complex configuration at runtime from the Dashboard UI.

For immutable configurations such as `node` and `cluster` configs, you can also use environment variables when the values are deployment-specific and should not be changed at runtime. See [Environment Variables](#environment-variables) and [Config Override Rules](#config-override-rules) for more details.

::: tip
The `base.hocon` file is not synchronized across the cluster and only applies to the node where it is located.
:::

## Configuration Rewrite File

In `data/configs` directory, the `cluster.hocon` file contains configuration items for the entire cluster.
Configuration changes made from the Dashboard, REST API, and CLI will be persisted to this file.

If a node in the cluster is restarted or if new nodes are added, the node will automatically copy and apply the `cluster.hocon` file from another node in the cluster. For this reason, it is not recommended to modify the file manually.

Configurations in this file are applied on top of those in the `base.hocon` file. For details on the configuration override hierarchy, see [Config Override Rules](#config-override-rules).

Since EMQX version 5.1, any changes to the cluster configuration will trigger a backup of the `cluster.hocon` file before it is overwritten. These backups are timestamped with the node's local time, and up to 10 backup files can be retained.

## Immutable Configuration File

For backward compatibility, the `emqx.conf` file remains available for critical system settings, including `node` and `cluster` configurations. This file has a higher priority than both `base.hocon` and `cluster.hocon`, but with a lower priority than environment variables. Avoid changing it unless you intentionally want this priority and understand that package upgrades may update the shipped defaults in this file.

For more details on configuration overrides, refer to the [Config Override Rules](#config-override-rules) section.

## Configuration Paths

In EMQX, configuration values can be referenced using dot-separated paths, similar to a tree structure. Starting from the root (always a Struct), each segment in the path refers to a field name or a Map key. For array elements, a 1-based index is used.

Here are some examples of configuration paths:

```bash
node.name = "emqx.127.0.0.1"
zone.zone1.max_packet_size = "10M"
authentication.1.enable = true
```

## HOCON Configuration Format

From EMQX v5.0, EMQX uses [Human-Optimized Config Object Notation (HOCON)](https://github.com/emqx/hocon) as the configuration file format.

HOCON is a format for human-readable data and a superset of JSON. With features like inheritance, combined, and quotes, HOCON further simplifies the configuration work.

**HOCON syntax：**

HOCON values can be represented as JSON-like objects, for example:

```bash
node {
  name = "emqx@127.0.0.1"
  cookie = "mysecret"
  cluster_call {
    retry_interval  =  1m
  }
}
```

or in flattening:

```bash
node.name = "127.0.0.1"
node.cookie = "mysecret"
node.cluster_call.retry_interval = "1m"
```

This cuttlefish-like flattening format is backward compatible with the previous EMQX versions, but it is used differently:

HOCON recommends adding quotes at both ends of the string. Strings without special characters can also be unquoted, for example, `foo`, `foo_bar`, while cuttlefish regards all characters to the right of `=` as values.

For more information about HOCON syntax, please refer to [HOCON Documentation](https://github.com/lightbend/config/blob/main/HOCON.md).

## Environment Variables

Besides configuration files, you can also use environment variables to configure EMQX.

For example, environment variable `EMQX_NODE__NAME=emqx2@127.0.0.1` will override the following configuration:

```bash
# emqx.conf
node {
  name = "emqx@127.0.0.1"
}
```

Configuration items and environment variables can be converted by the following rules:

1. Since the `.` separator in the configuration file cannot be used in environment variables, EMQX uses double underscores `__` as the configuration separator;
2. To distinguish the converted configuration items from other environment variables, EMQX also adds a prefix `EMQX_` to the environment variable;
3. The value of the environment variable is parsed as a HOCON value, making it possible to pass complex data types through environment variables. Values that contain HOCON special characters such as `:`, `=`, or `#` must be wrapped in double quotes `"` so the parser treats them as a literal string. In particular, `#` starts a HOCON line comment — without quoting, the parser silently drops everything from `#` to the end of the line.

Conversion example:

```bash
# Environment variables

## localhost:1883 will be parsed into a struct `{"localhost": 1883}`, so it needs to be wrapped in double quotes
export EMQX_LISTENERS__SSL__DEFAULT__BIND='"127.0.0.1:8883"'

## Pass the HOCON array directly by character
export EMQX_LISTENERS__SSL__DEFAULT__SSL_OPTIONS__CIPHERS='["TLS_AES_256_GCM_SHA384"]'


# Configuration file
listeners.ssl.default {
    ...
    bind = "127.0.0.1:8883"
    ssl_options {
      ciphers = ["TLS_AES_256_GCM_SHA384"]
    }
  }
}
```

::: warning Values that contain `#`, `:`, or `=`

A common gotcha is passing a password (or any string) that contains a `#`. Because `#` starts a HOCON line comment, this:

```bash
export EMQX_DASHBOARD__DEFAULT_PASSWORD="MQtt#123"
```

results in the password being parsed as `MQtt` — the `#123` is dropped as a comment. To pass the literal value through, wrap it in **HOCON-level** double quotes (not just shell quotes), so the parser sees `"MQtt#123"` including the quotes:

```bash
# Correct — value seen by the HOCON parser is: "MQtt#123"
export EMQX_DASHBOARD__DEFAULT_PASSWORD='"MQtt#123"'

# Same effect, with the inner quotes escaped for the shell
export EMQX_DASHBOARD__DEFAULT_PASSWORD="\"MQtt#123\""
```

The same applies to values containing `:` or `=`. URL-encoding (e.g. `%23` for `#`) does not work — EMQX does not URL-decode environment variable values.

:::

::: tip Why some unquoted values pass through and others don't

Internally, EMQX wraps each environment variable value as `fake_key=<value>` and tries to parse it as HOCON. If that succeeds, the parsed value is used; if it fails because the value is not valid HOCON syntax, EMQX falls back to the raw string. That is why `EMQX_..._PASSWORD="abc#def"` becomes `abc` (valid HOCON, `#def` is a comment) while `EMQX_..._PASSWORD=".abc#def"` is kept as the literal `.abc#def` (invalid HOCON syntax, fallback to raw). Wrapping the value in HOCON quotes makes the behavior deterministic.

:::

::: tip

EMQX will ignore undefined root paths, for example, `EMQX_UNKNOWN_ROOT__FOOBAR` , because `UNKNOWN_ROOT` is not a pre-defined root path.

When a known root path is set with an unknown field name, EMQX will output a `warning` log at startup. For example, when `enable` is incorrectly configured as `enabled`, it will output:

```bash
[warning] unknown_env_vars: ["EMQX_AUTHENTICATION__ENABLED"]
```

:::

::: tip
Starting from EMQX 6.3.0, `EMQX_FEATURES` is a special startup environment variable for [feature gates](../../get-started/deploy/feature-gates.md). It does not map to a HOCON configuration path, is not stored in `cluster.hocon`, and is resolved only when EMQX starts.
:::


### Boot-Time Environment Variables

Most `EMQX_`-prefixed environment variables override `emqx.conf` settings, following the conversion rules above. A few variables configure EMQX itself before the configuration files are parsed, so they have no `emqx.conf` equivalent:

- `EMQX_FEATURES`: Selects the set of applications the node boots, for example `FULL` or `ESSENTIAL`.
- `EMQX_SECURITY_PROFILE`: Selects the node-wide security profile, `legacy` or `hardened`.

Starting from EMQX 6.3.0, the `emqx` command loads environment variables from `etc/emqx.env` whenever it runs, including during a service start, a foreground start, and `emqx ctl`. On RPM and DEB installations, the file is located at `/etc/emqx/emqx.env`. Use this file to set boot-time environment variables instead of editing the systemd unit.

- Values set in the file override variables inherited from the environment.
- Package upgrades keep your edits to the file.
- The shipped file lists boot-time variables as commented lines, for example, `#KEY="${KEY:-default}"`. If you uncomment a line without changing the expression, it keeps a nonempty environment value or uses the default when the variable is unset or empty. To override an existing value, replace the expression with `KEY=value`.
- Restart the EMQX node after changing a boot-time environment variable.
- Regular `EMQX_`-prefixed overrides, for example `EMQX_NODE__COOKIE`, can also be set in the file.

## Config Override Rules

In EMQX, configuration values are applied hierarchically, with the following override rules:

- Within the same file, values defined later will override earlier ones.
- Higher-level configurations will replace lower-level ones.

The order of configuration priority is as follows:

`base.hocon < cluster.hocon < emqx.conf < environment variables`.  

This means that the settings in `base.hocon` have the lowest priority and can be overridden by settings in higher-priority files. Environment variables that start with `EMQX_` have the highest priority.

::: tip
Before version 5.8.4, the `base.hocon` file did not exist. The priority order remains the same, but without `base.hocon`.
:::

Changes made through EMQX Dashboard UI, HTTP API, or CLI are persisted in `cluster.hocon` at runtime and will take effect immediately. However, changes may get reverted after a node restart if the same configuration items are set differently in `emqx.conf` or environment variables.  

To avoid confusion, **do not overlap configuration settings** between `emqx.conf` and `cluster.hocon`.

::: tip
1. If you're using an older version of EMQX (e.g., 5.0.2/v5.0.22 or earlier, where the `cluster-override.conf` file still exists), the priority order for configuration settings is: `emqx.conf < ENV < HTTP API (cluster-override.conf)`.
2. When upgrading from version 5.0.2/v5.0.22 or earlier to the latest version, the priority order remains unchanged, and `cluster.hocon` will not be created to maintain compatibility.
3. The `cluster-override.conf` mechanism is removed in version 5.1.
   :::

### Override

In the following configuration, the `debug` value of `level` defined in the last line will overwrite the previously defined `error`, but the `enable` field remains unchanged:

```bash
log {
  console {
    enable = true
    level = error
  }
}

## Set the console log printing level to debug, and keep the other configurations
log.console.level = debug
```

The packet size limit was first set to 1MB, then overridden to 10MB:

```bash
zones {
  zone1 {
    mqtt.max_packet_size = 1M
  }
}
zones.zone1.mqtt.max_packet_size = 10M
```

### List Element Override

EMQX array has two expression ways:

- List, for example, `[1, 2, 3]`
- Map (subscribing), for example: `{"1"=1, "2"=2, "3"=3}`

The following 3 formats are equivalent:

```bash
authentication.1 = {...}
authentication = {"1": {...}}
authentication = [{...}]
```

Based on this feature, we can easily override the value of an element in an array, for example:

```bash
authentication  = [
  {
    enable = true,
    backend = "built_in_database",
    mechanism = "password_based"
  }
]

# The `enable` field of the first element can be overridden in the following way:
authentication.1.enable = false
```

::: tip

Arrays (in list format) will be fully overwritten and the original value cannot be kept, for example:

```bash
authentication = [
  {
    enable = true
    backend = "built_in_database"
    mechanism="password_based"
  }
]

## With the following method, all fields except `enable` of the first element will be lost.
authentication = [{ enable = true }]
```

:::

### Zone Override

A zone in EMQX is a concept for grouping configurations. Zones can be associated with listeners by setting the `zone` field to the name of the desired zone. MQTT clients connected to listeners associated with a zone will inherit the configurations from that zone, which may override global settings.

::: tip
By default, listeners are linked to a zone named `default`. The `default` zone is a logical grouping and does not exist in the configuration files.
:::

The following configuration items can be overridden at the zone level:

- `mqtt`: MQTT connection and session settings, such as allowing a greater maximum packet size for MQTT messages in a specific zone.
- `force_shutdown`: Policies for forced shutdowns.
- `force_gc`: Fine-tuning for Erlang process garbage collection.
- `flapping_detect`: Detection of client flapping.
- `durable_sessions`: Session persistence settings, such as enabling durable storage for MQTT sessions in a specific zone.

In EMQX version 5, the default configuration file does not include any zones, which differs from version 4, where there are two default zones: `internal` and `external`.

To create a zone, you need to define it in the config file, for example:

```bash
zones {
  # Multiple zones can be defined
  my_zone1 {
    # Zones share the same configuration schema as the global configurations
    mqtt {
      # Allow a larger packet size for connections in this zone
      max_packet_size = 10M
    }
    force_shutdown {
      # Configuration specific to this zone
      ...
    }
    durable_sessions {
      # Enable durable storage for sessions in this zone
      enable = true
      ...
    }
  }
  my_zone2 {
    ...
  }
}
```

In a listener, set the `zone` field to associate it with a zone that has been created.

```bash
listeners.tcp.default {
    bind = 1883
    zone = my_zone1
    ...
}
```

## Configuration-as-Code Best Practices

When you manage EMQX configuration from source control or an automation system, use the following rule of thumb:

- Put configuration-as-code settings in `base.hocon`.
- Do not edit `cluster.hocon` manually or mount your own `cluster.hocon` file.
- Avoid changing `emqx.conf` unless you understand its higher priority in the configuration layers and the upgrade consequences.
- Use environment variables for simple overrides that you do not expect to change from the Dashboard, API, or CLI at runtime.

The recommended source of truth for configuration-as-code is `base.hocon`. It is read from the static configuration directory during node startup and can be managed by your packaging, image build, configuration management, or GitOps workflow. Runtime changes made from the Dashboard, REST API, or CLI are persisted to `cluster.hocon` and layered on top of `base.hocon`.

For example, a deployment can keep its listener, log, authentication, authorization, and data integration baseline in `base.hocon`:

```bash
# base.hocon
listeners.tcp.default {
  bind = "0.0.0.0:1883"
  max_connections = 1024000
}

log.console {
  enable = true
  level = warning
}

authentication = [
  {
    mechanism = password_based
    backend = built_in_database
    user_id_type = username
  }
]
```

Do not use `cluster.hocon` as the source of truth for configuration-as-code. EMQX owns this file at runtime: the Dashboard, REST API, and CLI rewrite it, EMQX creates backups before overwriting it, and cluster nodes can copy it from each other. Manually editing or mounting this file can cause your changes to conflict with runtime updates or be overwritten.

`emqx.conf` is shipped with distribution packages as a baseline configuration file. Keeping it unchanged makes upgrades easier because your installation can pick up conservative default changes delivered by new EMQX versions. If you set a configurable item in `emqx.conf`, it has higher priority than both `base.hocon` and `cluster.hocon`, so runtime changes to the same item may appear to work but be reverted after a node restart. Use `emqx.conf` only when you intentionally need that behavior.

Environment variables have the highest priority. They are useful for simple deployment-specific values, especially values already provided by the runtime environment and values that should not be mutated at runtime:

```bash
export EMQX_NODE__NAME='emqx@node1.example.net'
export EMQX_NODE__COOKIE='mysecret'
export EMQX_CLUSTER__DISCOVERY_STRATEGY='static'
export EMQX_CLUSTER__STATIC__SEEDS='["emqx@node1.example.net", "emqx@node2.example.net"]'
```

Because environment variables override all configuration files, avoid using them for settings that operators are expected to tune later from the Dashboard, API, or CLI.

## Schema

To make the HOCON objects type-safe, EMQX introduced a schema for it. This schema defines data types, field names, and metadata, allowing for configuration value validation and more.

The [EMQX Enterprise Configuration Manual](https://docs.emqx.com/en/enterprise/v6.3.1/hocon/) are generated from the schema.

::: tip
The zone configuration schema is not included in the configuration manual because it is identical for each group. For example, `zones.my_zone1.mqtt {...}` has the same schema as `mqtt {...}`.
:::

### Primitive Data Types

Primitive data types in the configuration manual are largely self-explanatory, requiring minimal documentation. Below is a comprehensive list of all primitive types you will encounter.

#### Integer

Represents a whole number. Examples include `42`, `-3`, `0`.

#### Integer(Min..Max)

An integer that falls within a specified range. For example, `1..+inf` means from `1` to positive infinity (`+inf`), indicating that only positive integers are acceptable.

#### Enum(symbol1, symbol2, ...)

Defines an enumerated type that can only take one of the predefined symbols. For instance, `Enum(debug,info,warning,error)` defines acceptable logging levels.

#### String

The **String** data type represents a sequence of characters and supports several formats for diverse use cases:

- **Unquoted**: Ideal for simple identifiers or names that avoid special characters (see below for details).

- **Quoted Strings**: For strings that include special characters or whitespace,
  use double quotes (`"`), utilizing the backslash (`\`) to escape characters as needed. Example: `"line1\nline2"`.

- **Triple-quoted String**: Enclosed with triple quotes (`"""`), these strings do not require escapes (except for `\`),
  simplifying the inclusion of complex content. Note that quotes adjacent to the triple-quotes must be escaped to be considered part of the string.

- **Triple-quoted String with Indentation**: Surrounded by `"""~` and `~"""`,
  introduced since EMQX 5.6, this format allows for indentation within the string for better layout in the configuration file, ideal for multi-line or formatted text.

**Special Considerations for Unquoted Strings:**
- Avoid "forbidden characters": `$`, `"`, `{`, `}`, `[`, `]`, `:`, `=`, `,`, `+`, `#`, `` ` ``, `^`, `?`, `!`, `*`, `&`, `\`, or whitespace.
- Do not start with `//` (which introduces a comment).
- Do not begin with `true`, `false`, or `null` in a way that could be misinterpreted as boolean or null values.

**Guidelines for Triple-quoted Strings:**
- To include a quote character adjacent to the triple-quotes, escape it or use the `~` delimiter for clarity.
- Multiline strings support indentation with spaces (not tabs) for readability.
  The indentation level is determined by the smallest number of leading spaces on any line.

For example:

```
rule_xlu4 {
  sql = """~
    SELECT
      *
    FROM
      "t/#"
  ~"""
}
```

For additional details on string quoting conventions in HOCON, consult [the HOCON specification](https://github.com/lightbend/config/blob/main/HOCON.md#unquoted-strings).

For insights into EMQ's specialized adaptations of the triple-quoted string with indentation, refer to the [emqx/hocon.git README](https://github.com/emqx/hocon?tab=readme-ov-file#divergence-from-spec-and-caveats).

#### String("constant")

A constant string value, effectively acting as a single-value enumeration (`Enum`). This could be used to define a static value that doesn't change, such as a specific setting or mode.

#### Boolean

Either `true` or `false`, which is case sensitive.

#### Float

A floating-point number, supporting decimals. Examples include `3.14`, `-0.001`.

#### Duration

Represents a span of time in a human-readable format. Examples and explanation of format.

#### Duration(s)

Specifies a `Duration` type with a precision level of seconds. Further details and examples.

#### Secret

A type intended for sensitive information, such as passwords and tokens. Explanation of its usage and importance.


### Complex Data Types

Complex data types in EMQX's HOCON configuration are designed to encapsulate data structures that can include both other complex types and primitive values.
These data types enable flexible and hierarchical data representation.

#### Struct `Struct(name)`

Represents a structure with fields enclosed between curly braces `{}`.
The `name` parameter is a reference to a schema that specifies the structure's fields and their respective types.

#### Map `Map($name-\>Type)`

Similar to `Struct`, a `Map` holds key-value pairs without predefined names for the fields.

The `$name` variable indicates that the keys can be any string (except for a string with dot `.` in it),
representing the name of an entity or attribute.
The `Type` specifies that all values in the map must be of the same data type, allowing for uniform collections of data items.

#### OneOf `OneOf(Type1, Type2, ...)`

Defines a union type that can include two or more types to indicate that one struct field can be any one of the member types.
For example, this allows a configuration entry to be either `String(infinity)` or a `Duration`.

#### Array `Array(Type)`

Defines an array consisting of elements that adhere to the specified `Type`.


::: tip

If a Map field name is a positive integer number, it is interpreted as an alternative representation of an `Array`. For example:

```bash
myarray.1 = 74
myarray.2 = 75
```

will be interpreted as `myarray = [74, 75]`, which is handy when trying to override array elements.

:::

### Variform Expressions

Variform is a lightweight, expressive language designed for string manipulation and runtime evaluation.
It is not a full-fledged programming language but a specialized tool that can be embedded within
configurations for EMQX to perform string operations dynamically.

::: tip
Variform expressions are only applicable to certain configuration items. Do not use them unless specifically stated.
:::

::: tip NULL value:
In Variform expressions, a value-binding reference or sub-expression evaluation may result in an undefined value, which is represented as an empty string (`''`).

It is important to note that if a JSON-decoded field is `null`, it is treated as undefined value (hence `''`), but not string value `"null"`.
:::

#### Syntax

To illustrate:

```js
function_call(clientid, another_function_call(username))
```

This expression combines or manipulates clientid and username to generate a new string value.

Variform supports below literals:

- Boolean: `true` or `false`.
- Integer: For example, `42`.
- Float: For example, `3.14`.
- String: ASCII characters between single quotes `'` or double quotes `"`.
- Array: Elements between `[` and `]`, separated by a comma `,`.
- Variable: Referencing to predefined values, for example `clientid`.
- Function: Predefined functions, for example, `concat([...])`.

Variform does not support the following:

- Arithmetic operations
- Loops
- User-defined variables
- User-defined functions
- Exception handling and error recovery
- Escape sequence in string literals. Call the `unescape` function to unescape special characters.

Below is a configuration example with a Variform expression embedded.

```js
mqtt {
    client_attrs_init = [
        {
            # Extract the prefix of client ID before the first -
            expression = "nth(1, tokens(clientid, '-'))"
            # And set as client_attrs.group
            set_as_attr = group
        }
    ]
}
```

::: tip
When an unescape function is required in the expression, it's a good idea to use triple quote (`"""`) strings in HOCON config so there is no need to perform double escaping.

For example

```
#### For multi-line client ID, take the first line.
expression = """nth(1, tokens(clientid, unescape('\n')))"""
```
:::

#### Pre-defined Functions

EMQX includes a rich set of string, array, random, and hashing functions similar to those available in rule engine string functions.
These functions can be used to manipulate and format the extracted data. For instance, `lower()`, `upper()`,
and `concat()` help in adjusting the format of extracted strings, while `hash()` and `hash_to_range()` allow for creating hashed or ranged outputs based on the data.

Below are the functions that can be used in the expressions:

- **String functions**:
  - [String Operation Functions](../../develop/data-integration/rule-sql-builtin-functions.md#string-operation-functions)
  - A new function `any_to_string/1` is also added to convert any intermediate non-string value to a string.
- **Array functions**: [nth/2](../../develop/data-integration/rule-sql-builtin-functions.md#nth-n-integer-array-array-any)
- **Topic functions**:
  - `topic_join(Words)`: Join topic levels in an array with `/` into an MQTT topic or topic filter. For example, `topic_join(['devices', clientid, '#'])` generates `devices/<clientid>/#`.
  - `topic_join(Parent, Word)`: Append `Word` to the `Parent` topic. If `Parent` already ends with `/`, no extra separator is added.
  - `topic_match(Topic, Filter)`: Check whether an MQTT topic matches a topic filter. It returns `true` or `false`. For example, `topic_match(topic, topic_join(['devices', clientid, '#']))` can check whether the current topic matches a client-specific topic filter.
  - `topic_split(Topic)`: Split an MQTT topic into an array of topic levels by `/`.
- **Random functions**: rand_str, rand_int
- **Schema-less encode/decode functions**:
  - [bin2hexstr(Data)](../../develop/data-integration/rule-sql-builtin-functions.md#bin2hexstr-data-binary-string)
  - [hexstr2bin(Data)](../../develop/data-integration/rule-sql-builtin-functions.md#hexstr2bin-data-string-binary)
  - [base64_decode(Data)](../../develop/data-integration/rule-sql-builtin-functions.md#base64-decode-data-string-bytes-string)
  - [base64_decode(Data, 'no_padding')](../../develop/data-integration/rule-sql-builtin-functions.md#base64-decode-data-string-bytes-string)  (since 6.0.2)
  - [base64_decode(Data, 'no_padding', 'urlsafe')](../../develop/data-integration/rule-sql-builtin-functions.md#base64-decode-data-string-bytes-string)  (since 6.0.2)
  - [base64_encode(Data)](../../develop/data-integration/rule-sql-builtin-functions.md#base64-encode-data-string-bytes-string)
  - [base64_encode(Data, 'no_padding')](../../develop/data-integration/rule-sql-builtin-functions.md#base64-encode-data-string-bytes-string) (since 6.0.2)
  - [base64_encode(Data, 'no_padding', 'urlsafe')](../../develop/data-integration/rule-sql-builtin-functions.md#base64-encode-data-string-bytes-string) (since 6.0.2)
  - `int2hexstr(Integer)`: Encode an integer to hex string. e.g. 15 as 'F' (uppercase).
- **Hash functions**:
  - `hash(Algorithm, Data)`: Algorithm can be one of: md4 | md5, sha (or sha1) | sha224 | sha256 | sha384 | sha512 | sha3_224 | sha3_256 | sha3_384 | sha3_512 | shake128 | shake256 | blake2b | blake2s
  - `hash_to_range(Input, Min, Max)`: Use sha256 to hash the Input data and map the hash to an integer between Min and Max inclusive (Min <= X <= Max)
  - `map_to_range(Input, Min, Max)`: Map the input to an integer between Min and Max inclusive (Min <= X <= Max)
- **Compare functions**:
  - `num_eq(A, B)`: Return 'true' if two numbers are the same, otherwise 'false'.
  - `num_neq(A, B)`: Return 'true' if two numbers are NOT the same, otherwise 'false'.
  - `num_gt(A, B)`: Return 'true' if A is greater than B, otherwise 'false'.
  - `num_gte(A, B)`: Return 'true' if A is not less than B, otherwise 'false'.
  - `num_lt(A, B)`: Return 'true' if A is less than B, otherwise 'false'.
  - `num_lte(A, B)`: Return 'true' if A is not greater than B, otherwise 'false'.
  - `str_eq(A, B)`: Return 'true' if two strings are the same, otherwise 'false'.
  - `str_neq(A, B)`: Return 'true' if two strings are NOT the same, otherwise 'false'
  - `str_gt(A, B)`: Return 'true' if A is behind B in lexicographic order, otherwise 'false'.
  - `str_gte(A, B)`: Return 'true' if A is not before B in lexicographic order, otherwise 'false'.
  - `str_lt(A, B)`: Return 'true' if A is before B in lexicographic order, otherwise 'false'.
  - `str_lte(A, B)`: Return 'true' if A is not after B in lexicographic order, otherwise 'false'.
  - `is_empty_var(V)`: Check if a variable is empty. Empty in Variform means the value is not present (`undefined`), JSON's `null` (but not string `"null"`), or an empty string `""`.
  - `not(Bool)`: Return `true` if `Bool` is `false`, and return `false` if the condition is `true`. It also accepts string parameters. If the input is a string, the output is also a string.

- **System functions**:
  - `getenv(Name)`: Return the value of the environment variable `Name` with the following constraints:
    - Prefix `EMQXVAR_` is added before reading from OS environment variables. For example, `getenv('FOO_BAR')` is to read `EMQXVAR_FOO_BAR`.
    - Values are immutable once loaded from the OS environment.

- **Data extraction functions**:
  - `json_value(Data, Path)`: Extract values from JSON strings using a dot-separated path to navigate nested structures. For example, if `username` is a JSON object, you can access a field with `json_value(username, 'shop.floor')`.
  - `jwt_value(Data, Path)`: Decode JWT token payloads and extract claim values using a dot-separated path. For example, if `password` is a JWT with a customized claim, you can access the nested value with `jwt_value(password, 'client_attrs.unitid')`.
  - `is_jwt(Data)` (since 6.2.3): Check whether `Data` has the structure of a JWT in JWS compact form. The function returns `true` only for values with 3 dot-separated Base64URL-decodable segments and a decoded header JSON object that contains an `alg` field. It does not verify the signature or inspect the payload, and returns `false` for `undefined`, `null`, empty strings, JWE tokens with 5 segments, and malformed values.

#### Conditions

The variform expression so far has no comprehensive control flows.
Below functions can help to preform basic control of which value to return from the expression.

- `iif(Condition, ThenExpression, ElseExpression)`: returns `ThenExpression` if `Condition` yields `true` or a non-empty string value, otherwise returns `ElseExpression`.
- `coalesce(Arg1, Arg2, ...)`: returns the first non-empty argument.
- `coalesce([Element1, Element2, ...])`: returns the first non-empty element.

#### Error Handling

As the default behavior of scripting environments like Bash, Variform expression is designed to yield an empty string ("") in scenarios where errors occur, such as unbound variables or exceptions during runtime.

- Unbound Variables: If an expression references a variable that has not been defined or is out of scope (unbound), the expression will be evaluated as an empty string.
- Runtime Exceptions: Any exceptions that occur during the execution of an expression, whether due to incorrect function usage, invalid data types, or other unforeseen issues, will result in the expression yielding an empty string. For example, the array index is out of range.

#### Example Expressions

- `nth(1, tokens(clientid, '.'))`:  Extract the prefix of a dot-separated client ID.
- `strlen(username, 0, 5)`: Extract a partial username.
- `coalesce(regex_extract(clientid,'[0-9]+'),'vin-1000')`: Extract digits from client ID using a regular expression. If the regular expression yields empty string, then return `'000'`.
- `iif(true, "Value if true", "Value if false")`: Returns `Value if true`
- `iif("", "Value if true", "Value if false")`: Returns `Value if false`
- `iif("hello", "Value if true", "Value if false")`: Returns `Value if true`
- `iif(regex_match(clientid,'^foo\.+*'),'foo','bar')`: Returns `foo` if `clientid` starts with `foo.`, otherwise `bar`.