Open Control Architecture

Basic Architecture

Features:

• Real-time global commutation (<0.5s latency) of any payload type between millions of devices
• Open source, leveraging the best open technologies
• Battle hardened, standard and secure technologies
• Simple to use for plumbers and electricians
• Can connect to anything, physical or web based

Controls software is written in Node-RED.

We have chosen to base the Open Controls Architecture on Node-RED for a number of reasons:

• Open source and free to use at will
• Very stable, having been actively developed for over 10 years with the likes of IBM
• Simple to use with a visual interface that compares to electrical wiring
• Software can be copied and pasted from one system to another, dropped in and wired up
• Infinitely versatile with standard and custom components available from an online library to achieve almost any function or connect to any system
• Complex software can be built in a hurry, dragging components in and wiring up, then refined and packaged into neat and updateable components
• Built on Node.js and JavaScript, possible the world's most active software environment as used on all web pages
• Used by almost everyone - from hobbyists through to mobile phone companies - with considerable community support
• Provides a means to easily implement new and existing technologies and online services, including machine learning, peer-to-peer communications, databases, and encryption

Software works through packets of data (messages) passing along virtual (and real) wires, from one node to another.

A node can be:

• a function block within Node-RED
• an instance of Node-RED
• any point (server or device) in the communications network

Each instance of Node-RED has an identifier, stored under global.node

Communications

Communications between systems is performed using MQTT using a standardised 5 level topic structure. network / node / device / group / key.

See https://github.com/heatweb/heat-network

Systems run a local MQTT broker to distribute data both internally and to enable external subscriptions.

The external network identifier, used for all data published to external MQTT brokers, is stored under global.networkId

In the real world, devices and networks are nested - a building may have a network ID, but this may be fed from a plantroom, also with its own identity. To accommodate nested networks, a dash is used to prefix topics (and hence the network ID) and thereby maintain a five level topic structure - important for wildcard subscriptions to work..

myNetwork/plantroomNode/plantroomNode/settings/title = "Main Plantroom Controller"

block1/blk1node/blk1node/settings/title = "Block 1 Controller"  !!! DO NOT USE COMMON NETWORK IDs

Adding a prefix to sub-networks ensures networks remain unique on a higher level server. E.g. There may be two unconnected networks with sub-networks called "block1".

myNetwork-block1/blk1node/blk1node/settings/title = "Block 1 Controller"

anotherNetwork-block1/blk1node/blk1node/settings/title = "Block 1 Controller"

Where managing multiple networks, it is advised to use long and non-descriptive unique identities. When creating new configurations a random 8-16 character ID should be generated.

e.g. Ea5lQXfF2tmBlzYk

Nodes should also be given unique identities. Nodes Id's can be shorter as they only need to be unique within a network, rather than globally (in the whole world).

e.g. jxt15Ewk

The titles of networks, as well as nodes, can be saved and found under its device setting.

Ea5lQXfF2tmBlzYk/jxt15Ewk/network/settings/title = "My Heat Network"

Ea5lQXfF2tmBlzYk/jxt15Ewk/jxt15Ewk/settings/title = "Main Plantroom Controller"

And to simplify for sub-nodes it is possible to use a node's id as the node's network identifier.

a5lQXfF2tmBlzYk-fg4jS8s7/fg4jS8s7/network/settings/title = "Block 1 Heat Network"

a5lQXfF2tmBlzYk-fg4jS8s7/fg4jS8s7/fg4jS8s7/settings/title = "Block 1 Controller"

a5lQXfF2tmBlzYk-fg4jS8s7-nnf555dE/nnf555dE/nnf555dE/settings/title = "Block 1, Flat 17 Controller"

Data from within a node or from connected devices is assigned a local network identifier of "local" (stored under global.localNetworkId) so that the node's external network identifier can be changed without affecting internal software. Separate internal and external identifiers also allows for separate message rates, where local traffic can be high speed. Node-RED is used to apply Reporting By Exception rules, to limit traffic to significant values (e.g. 0.25C), and save bandwidth.

Handling Data

Within Node-RED a Message Handler Node is used to handle data generated locally.

• The Node has two outputs - one for local traffic, and one for external traffic.
• Report by exception rules are applied to external traffic.
• The Node formats topics to the five level standard, adding group and device if missing.
• The Node stores data into global.readings, along with any attributes within the incoming message, as well as timestamps.
• Additional routing can be performed within Node-RED, including selection of retained vs unretained routes.

Handler Nodes use a configuration node to store setup information including network and node identities.

Where a number of nodes sit on a Local Area Network (LAN) it is common to use a central local MQTT broker.

Traffic can be routed to a Cloud MQTT broker by the use of MQTT Bridges. These run on an MQTT broker, and can re-publish messages filtered by topic (networkId) to another broker. Bridges are managed outside of Node-RED.

Additional Report By Exception rules can be applied inline.

On larger networks where data is shared between local nodes, Reporting By Exception can be centrally managed using a Node-RED instance, along with any additional MQTT broker functions such as providing backup routes.

Cloud based instances of Node-RED, as used on this Wiki, are often used to pull together data from various MQTT brokers into one place.

Data Groups

The topic group is used to define the type of data. Values include:

• dat (operational data)
• stat (statistics and status)
• settings (persistent settings)
• cmd (a command)
• set (command to change a setting)
• system (software and networking)
• meter (meter data)
• alarm (alarms and warnings)
• ana (analogue readings)
• json (JSON formatted data)
• modbus (Modbus serial data)
• pcdb (pcdb data)
• design (preloaded design data)

Data Attributes

Attributes (metadata) provide additional information about data, such as units and a description. They include:

• units (units for data)
• title (short description)
• points (maximum data points to store in history)
• deadband (the deadband used for external communications)
• timestamp (unix time data is generated or handled)
• changed (unix time data last changed)
• attributes (an object containing custom attributes)

Attributes are generally loaded from a central index (GitHub or LAN hosted), providing additional information such as units and descriptions. This is to avoid the need to locally describe data, and to assist in compatibility. Custom attributes may be added inline to messages and will override defaults.

Loaded attributes are saved in a global Node-RED object,and will be added to messages as they are handled, and stored into readings alongside the value (payload). They are however not transmitted through MQTT - only topics and payloads are transmitted by default.

Attributes can be transmitted through MQTT in one of two ways:

• A special command cmd/sync can be sent to a nodes message handler asking it to publish a full JSON object for a device (or devices), including all data groups, values, and attributes. The command payload is an MQTT topic including wildcards, e.g. local/# (all), or +/+/boiler1/#. The reply for each device that matches is sent on the corresponding topic, e.g. myNetwork/node123/boiler1/json/sync and can be handled by a receiving system.
• Packaging a message as a JSON object, value and attributes, and send this instead of just the value. A receiving system can work out the payload is JSON and handle appropriately.

Node-RED Global Objects

Within Node-RED, data and attributes are stored within the global.readings object.

The current data can be viewed in the data explorer.

Historic data is saved into the global.readingsHistory object, in the same structure as global.readings, but containing an array of timestamped values.

Persistent settings for a node (or application) are held on disk and in the global.settings object. This is for local operational settings.

Settings have a type, may have maximum and minimum values, units and a title. Certain types require additional fields such as select from a list settings.

The snapshot below shows a variety of setting types including numeric/text values, html, or JSON objects and arrays.

Persistent settings for local devices are held on disk and in the global.deviceSettings object, and are specific to a device (or node).

Data within the readings object is nested according to the topic (excluding the node as more than one node may publish data on a device).

e.g. a message with a topic myNetwork/node123/myDevice/dat/outputTemperature  would be stored in (global).readings.myNetwork.myDevice.dat.outputTemperature.value

Internally, message topics can be truncated and the network and node will be filled in. The data group will be assumed to be "dat" if not supplied, representing operational data.

e.g. a message with a topic setpoint  would be stored in (global).readings.local.myNode.dat.setpoint.value

The following code could be used to read data and its attributes for output.

 var localNetworkId = global.get("localNetworkId");
 var node = global.get("node");

 var reading = global.get("readings." + localNetworkId + "." + node + ".dat.outputTemperature");

 var output = "The value of " + reading.title + " is " + reading.value + reading.units;  
 // "The value of Output Temperature is 35°C