Memory

Stream Scan table  Lookup table

The memory source obtains the data stream events held in the memory of the NeuronEX processing module. The typical use of memory actions is to form Rule Pipeline and put the results of the previous rule processing in Memory Sink , and then obtain the data through the memory source to form a regular pipeline.

The memory source consumes events generated by the memory Sink through data source (topic). Topic consumption is similar to a pubsub topic, such as MQTT, so there may be multiple memory targets publishing to the same topic, and there may be multiple memory sources subscribing to Same theme. For details, see [topic wildcard](#topic wildcard)

Create stream

Log in to NeuronEX and click Data Processing -> Source. On the Stream tab, click Create Stream.

In the pop-up Source/Create page, enter the following configuration:

• Stream Name: Enter the stream name
• Whether the schema stream: Check to confirm whether it is a structured stream. If it is a structured stream, you need to add further stream fields. It can be unchecked by default.
• Stream type: select memory
• Data source (topic): The memory topic to be subscribed to, such as topic1. For details, see topic wildcard

Topic wildcard

Memory sources also support topic wildcards, similar to MQTT topics. Currently, two wildcard characters are supported.

+ : Single level wildcard replaces one topic level.

#: Multi-level wildcards cover multiple topic levels and can only be used at the end.

Example:

1. home/device1/+/sensor1
2. home/device1/#

Create scan table

Memory sources support scan tables. Log in to NeuronEX and click Data Processing -> Source. On the Scan Table tab, click Create Scan Table.

• Table Name: Enter the table name
• Whether the schema stream: Check to confirm whether it is a structured stream. If it is a structured stream, you need to add further stream fields. It can be unchecked by default.
• Table type: select memory
• Data source (topic): The memory topic to be subscribed to, such as topic1. Similar to MQTT themes. For details, see topic wildcard
• Retain Size: Specify the Retain size.

Create lookup table

The memory source supports being used as a lookup table. At this time, it mainly has the following advantages:

• INDEPENDENCE: The memory lookup table is independent of any rule operations. Even if the rules are modified or deleted, the data in the memory lookup table is not affected. Data sharing: If multiple rules reference the same table, or there are multiple memory tables with the same subject/key pairs, they all share the same data set, ensuring consistency between different rules and simplifying data access.
• Integration with Memory Sink: The memory lookup table can be integrated with the updateable Memory Sink to ensure the real-time nature of the content.
• Rule Pipeline: The memory lookup table can serve as a bridge between multiple rules, similar to the concept of rule pipeline. It enables one stream to store historical data in memory, which can be accessed and utilized by other streams, making it suitable for scenarios that require combining historical and real-time data for decision-making.

Log in to NeuronEX and click Data Processing -> Source. On the lookup Table tab, click Create lookup Table.

• Table Name: Enter the table name
• Whether the schema stream: Check to confirm whether it is a structured table. If it is a structured table, you need to add further table fields. It can be unchecked by default.
• Table type: select memory
• Data source (topic): The memory topic to be subscribed to, such as topic1. Similar to MQTT themes. For details, see topic wildcard
• Key: Specify the primary key.

Tips

When used as a lookup table, the KEY attribute should also be configured, which will serve as the primary key of the virtual table to speed up queries. Once created, the memory lookup table will begin accumulating data from the specified memory topic and indexed by the KEY field, allowing fast retrieval.