Processing Data Before Forwarding
Collected data can be published as it is, or processed by the rules engine first. This page covers how that step fits into the configuration flow; for the full capability of the rules engine, see Data Processing.
Whether this step is needed
| Situation | How to handle it |
|---|---|
| The cloud or upper-level system needs the complete raw data | Skip this step and go to Create a Northbound Application |
| The polling rate far exceeds what the business needs | Aggregate over a time window before publishing — volume drops by two orders of magnitude |
| Values stay unchanged for long periods | Filter conditionally, publishing only when a value moves beyond a threshold |
| Alarms need sub-second response | Keep the decision at the edge, avoiding a cloud round trip and remaining effective while the link is down |
| Units or field names must be normalized before publishing | Do it once at the edge rather than in every downstream system |
Both paths can run at once: some collection groups publish directly while others go through the rules engine.
TIP
A linear conversion (decimal, bias) or a decimal-place setting does not need the rules engine — configure it on the tag. See Groups and Tags · Shaping the data.
Three steps to connect it
1. Feed collected data into the rules engine
The rules engine receives southbound data as a northbound application. On Data Collection → North Apps, find the Rules Engine Application that exists by default (node name DataProcessing), click Add Subscription, and select the southbound driver and collection group to process.
Once subscribed, the group's tags flow into the engine's neuronStream stream. See Rules Engine Application.
2. Create a rule
On Data Processing → Rules, create a rule and describe the processing in SQL. For example, keeping only changes beyond a threshold:
SELECT * FROM neuronStream WHERE abs(pressure - lag(pressure)) > 0.5For SQL capabilities see the SQL Reference, and for window aggregation see Windows.
3. Choose where the result goes
Add a sink to the rule to decide where the processed data is written:
| Destination | What to use |
|---|---|
| Straight to an external system | MQTT, Kafka, MySQL, InfluxDB, Redis, AWS S3, and more — see Sink |
| Back to the device | Neuron sink, closing a collect → decide → control loop at the edge |
| On to the next rule | Memory sink, forming a rule pipeline |
WARNING
A rule's sinks and the northbound applications are two independent exits. Processed data leaves through the sink and does not pass through a northbound application again; if both are configured, the same collected data is published twice.
Verify
Enable rule testing while creating the rule to see live whether the SQL produces what you expect, instead of diagnosing it after the data reaches a downstream system.
For a hands-on walkthrough, see Your First Rule.
Next steps
- To publish without a rule, or to send rule results out through a northbound application — Create a Northbound Application
- Sources, SQL, windows, sinks, and extensions of the rules engine — Data Processing