Ingest Data into TDengine

TDengine
TaosData(https://www.taosdata.com) Enables efficient, real-time data ingestion, processing and monitoring of TB and even PB scale data per day, generated by billions of sensors and data collectors. TDengine can be widely applied to IoT, Industrial Internet, Connected Vehicles, DevOps, Energy , Finance and many other use-cases.

Deploy TDengine by Huawei Cloud or Docker:

TIP

To use TDengine v3.x, please use EMQX v4.4.12 and later.

docker run --name TDengine -d -p 6030:6030 -p 6041:6041 -p 6043-6049:6043-6049 -p 6043-6049:6043-6049/udp tdengine/tdengine

Execute cmd in docker:

docker exec -it TDengine bash
taos

Create database test

create database test;

Create table t_mqtt_msg. For more details on TDengine data structures and SQL, see TDengine SQL:

USE test;
CREATE TABLE t_mqtt_msg (
  ts timestamp,
  msgid NCHAR(64),
  mqtt_topic NCHAR(255),
  qos TINYINT,
  payload BINARY(1024),
  arrived timestamp
);

Create Rule:

Open EMQX Dashboard. Click Rule.

Write SQL:

SELECT

  *,
  now_timestamp('millisecond')  as ts

FROM

  "#"

The subsequent action creation operation can be selected flexibly depending on your EMQX version.

Add Action:

Find Action, click Add Action.

Click Data persist, Choice Data to TDengine.

Requires EMQX Enterprise 4.1.1 and later only.

Action parameters:

1. SQL template. In this example we insert a data to TDengine, note that we should specify the database name in the SQL and the character type should be enclosed in single quotes, the SQL template:

insert into test.t_mqtt_msg(ts, msgid, mqtt_topic, qos, payload, arrived) values (${ts}, '${id}', '${topic}', ${qos}, '${payload}', ${timestamp})

Before data is inserted into the table, placeholders like ${id} will be replaced by the corresponding values.

If a placeholder variable is undefined, you can use the Insert undefined value as Null option to define the rule engine behavior:

• false (default): The rule engine can insert the string undefined into the database.
• true: Allow the rule engine to insert NULL into the database when a variable is undefined.

2. Now that the resource drop-down box is empty, you can create a TDengine resource by clicking on Create in the upper right corner:

Create new TDengine Resource:
TDEngine Username: root
TDEngine password: taosdata

Click Confirm.

Return to the action screen and click "Confirm".

Return to the rule screen and click "Create".

Test:

In the rule list, click on the Rule ID link to preview the rule you just created.

The rule has been created, now send a data:

Topic: "t/a"
QoS: 1
Payload: {"msg": "hello"}

Query and check:

select * from t_mqtt_msg;