Systems run a local MQTT broker to distribute data both internally and to enable external subscriptions.
The external network identifier, used for all data published to external MQTT brokers, is stored under global.networkId
In the real world, devices and networks are nested - a building may have a network ID, but this may be fed from a plantroom, also with its own identity. To accommodate nested networks, a dash is used to prefix topics (and hence the network ID) as they pass through to a Cloud Server.
Adding a prefix to sub-networks ensures networks remain unique on a higher level server. E.g. There may be two unconnected networks with sub-networks called "block1".
Data from within a node or from connected devices is assigned a local network identifier of "local" (stored under global.localNetworkId) so that the node's external network identifier can be changed without affecting internal software. Separate internal and external identifiers also allows for separate message rates, where local traffic can be high speed. Node-RED is used to apply Reporting By Exception rules, to limit traffic to significant values (e.g. 0.25C), and save bandwidth.
Handling Data
Within Node-RED a Message Handler Node is used to handle data generated locally.
The Node has two outputs - one for local traffic, and one for external traffic.
Report by exception rules are applied to external traffic.
The Node formats topics to the five level standard, adding group and device if missing.
The Node stores data into global.readings, along with any attributes within the incoming message, as well as timestamps.
Additional routing can be performed within Node-RED, including selection of retained vs fire & forget routes.
Handler Nodes use a configuration node to store setup information including network and node identities.
Where a number of nodes sit on a Local Area Network (LAN) it is common to use a central local MQTT broker.
Traffic can be routed to a Cloud MQTT broker by the use of MQTT Bridges. These run on an MQTT broker, and can re-publish messages filtered by topic (networkId) to another broker. Bridges are managed outside of Node-RED.
Additional Report By Exception rules can be applied inline.
On larger networks where data is shared between local nodes, Reporting By Exception can be centrally managed using a Node-RED instance, along with any additional MQTT broker functions such as providing backup routes.
Cloud based instances of Node-RED, as used on this Wiki, are often used to pull together data from various MQTT brokers into one place.
Data Groups & Attributes
The topic group is used to define the type of data. Values include:
dat (operational data)
stat (statistics and status)
settings (persistent settings)
cmd (a command)
set (command to change a setting)
system (software and networking)
meter (meter data)
alarm (alarms and warnings)
ana (analogue readings)
pcdb (pcdb data)
design (preloaded design data)
Attributes for data may be loaded from a central index (GitHub hosted), providing additional information such as units and descriptions. This is to avoid the need to locally describe data, and to assist in compatibility.
Node-RED Global Objects
Within Node-RED, data and attributes are stored within the global.readings object.
The current data can be viewed in the data explorer.
Historic data is saved into the global.readingsHistory object, in the same structure as global.readings, but containing an array of timestamped values.
Persistent settings for a node (or application) are held on disk and in the global.settings object. This is for local operational settings.
Settings have a type, may have maximum and minimum values, units and a title. Certain types require additional fields such as select from a list settings.
The snapshot below shows a variety of setting types including numeric/text values, html, or JSON objects and arrays.
Persistent settings for local devices are held on disk and in the global.deviceSettings object, and are specific to a device (or node).
Data within the readings object is nested according to the topic (excluding the node as more than one node may publish data on a device).
e.g. a message with a topic myNetwork/node123/myDevice/dat/outputTemperature would be stored in (global).readings.myNetwork.myDevice.dat.outputTemperature.value
Internally, message topics can be truncated and the network and node will be filled in. The data group will be assumed to be "dat" if not supplied, representing operational data.
e.g. a message with a topic setpoint would be stored in (global).readings.local.myNode.dat.setpoint.value
The following code could be used to read data and its attributes for output.
var localNetworkId = global.get("localNetworkId");
var node = global.get("node");
var reading = global.get("readings." + localNetworkId + "." + node + ".dat.outputTemperature");
var output = "The value of " + reading.title + " is " + reading.value + reading.units;
// "The value of Output Temperature is 35°C