Adapter Protocol (sMQTT) v0.35

1.1 Adapters

Adapters link OT-Device protocols and the MQTT-based messaging within the SICON.OS. Adapters can either run as software modules within the SICON.OS (“internal adapters”) or as distributed modules in the OT-devices themselves (“external adapters”).

1.2 Devices

Every adapter represents one or more devices in a hierarchical structure. The root device is called “main”.

Device Descriptions are used in 2 different formats:

• IODD: for IO-Link devices

• SDD: the proprietary SICON.OS device description format, can be used to

  • describe a non-IO-Link device completely

  • describe additional information for IO-Link devices that is not contained in the IODD

2. MQTT Protocol

2.1 General recommendations for MQTT clients

Since the development of this specification is still ongoing and requirements are expected to change somewhat, all MQTT clients should be implemented with a good level of tolerance towards incoming data. In particular:

• Messages on unknown or unsupported subtopics should be ignored without further reaction

• Messages with zero-length payload should be ignored without further reaction

• Missing or incorrectly specified data in a message: whenever possible, an internal default value for that data should be used and the message processed anyway. If no reasonable default value can be defined for a required parameter, the message should be ignored.

• When waiting for a certain type of message, there should be a way to exit this wait condition in case the message never arrives, usually a timeout with fallback to a previous or idle state.

2.2 Connection Settings

Unless otherwise noted, the following methods and settings will be used by all mqtt clients:

• Connections to the broker are started with QoS = 0 and cleanSession = true

• The keepAlive feature shall be used with the broker with a timeout of 10 seconds. This feature allows both broker and client to detect disconnections and react accordingly. Specifically, upon client disconnect the Last Will and Testament (LWT) shall be used as well.

2.3 Topic Trees

Overview of the topic trees used in SICON.OS

• mqtt/status/+ – Status information of internal services

• services/[recipient]/[sender]/# - Certain commands between services

• adapterConfig/[recipient]/[sender]/+ - Management of adapters

• adapter/[recipient]/[sender]/+ - Management of devices

• device/[DevID]/+ - Public device data

2.4 JSON

All MQTT messages in this protocol use JSON in the message body.

2.5 Data Types

The values of parameter objects are always represented as strings in JSON. However within these strings, we use a set of data types derived from the IO-Link specification:

The complex data types “ArrayT” and “RecordT” from the IODD specification are not used in this protocol. When they occur in a device, the data are represented by the individual members as subindices or the whole object as a HexString.

2.6 Data Formatting

In addition to a data type, data objects can have additional attributes that govern how the raw data from the OT devices are to be represented within SICON.OS and to the user:

• Gradient, Offset: If present they are represented as Float32T and are used to scale the device data in the following way:

<Data format in gateway> = Gradient x <Raw data from device> + Offset

• Resolution: Dec|Dec.1|Dec.2|Dec.3|Hex|Bin

• Unit: to represent the physical unit of a value, given as a string. E.g. “unit”:”mbar”.

Data Formatting: allowed combinations of DataType, DisplayFormat, Gradient and Offset

2.7 Acknowledgment of message receipt

A number of mqtt messages, mainly the commands to control adapters, shall be acknowledged by the recipient. This is also used as a handshake to control sequential message flow.

Acknowledgment messages are formed using the following scheme when the mapper sends a request to an adapter:

In the remainder of this document, the term “requires an acknowledgment message” shall refer to using this scheme.

2.8 Return codes

The error codes for reading and writing stat data are defined based on the “additional code” values of ISDU responses in IO-Link. That means in most cases an adapter for an IO-Link-Master can just forward the value it has received from the IO-Link device.

2.9 Security

The broker requires a username:password from every client except for read/write to device/#-topics.

2.10 Status of internal mqtt clients

The internal mqtt clients in SICON.OS (mapper, mqtt2db, etc.) will post their status on a specific topic tree and use LWT; so a watchdog-type service can monitor for crashes and restart them.

Topic tree will look like this:

• “mqtt/status/[clientName]”, message {“Status”:”on|off|disconnected”}

The mapper (clientName = Mapper) will – as an exception – set its status with retail=true.

3. Adapter Management

• Every instance of an adapter is assigned a unique identification called “InterfaceID” within SICON.OS (known as AdapterInstanceID in the database). The InterfaceID is an integer value.

• After an adapter has received its InterfaceID, it gets started by the backend. This causes the adapter to initiate the resetting of old device entries in the database as well as preregistering its “main” device.

• The adapter then remains dormant until it is chosen by the user for full device registration.

The overall sequence with all possible steps is shown here:

3.1 Assigning the InterfaceID

3.1.1 Internal / External Adapters

The term “internal adapters” refers to a set of software modules that share the same mqtt client and run as services in SICON.OS.

The term “external adapters” refers to adapters that come with their own mqtt client and are separated from the internal set of adapters. They may be running on SICON.OS as well, but still communicate like external adapters. External adapters need to be preconfigured with the broker IP-address and port. A useful default value could be defined (e.g. 192.168.0.250).

An adapter that has not been assigned an InterfaceID yet shall make its presence known upon startup by publishing the regAdapter message.

(* “Mult” = multiplicity of the attribute)

According to chapter 2, this message is published with the mqtt option “retainFlag = true” so the broker saves this message as the current state in case the mapper comes online after the adapter. An LWT is not to be used here.

The ID-Assigner responds to this by sending an assignID message.

Upon reception, the adapter shall

1. Store the InterfaceID persistently so the next system startup can be done without assignment.

2. Clear the retained status-message on ”adapterConfig/idAssigner/[adapter MAC address]/regAdapter” (this is done by sending an empty message (size 0) with retainFlag = true to this topic)

3. Disconnect from the broker so a proper LWT can be set on the status topic (see next chapter).

3.2 Persistent adapter status

External adapters are in one of 4 different connection states and publish their current state persistently using the status message.

3.3 Starting and stopping adapters

Adapters are in the state “off” upon startup, i.e. they do not register devices with the mapper.

To switch the adapter between on and off states, the commands startAdapter and stopAdapter are used.

The adapter switches to state “reset” (without publishing this status) and sends out the “resetDevices” message as described in in 3.4.

The adapter shall unregister all its attached devices by sending disDevice for each of them. It also sets its status to “off” and publishes this as described in 3.2. While in the off-state, it shall not register any new devices.

3.4 Resetting devices

After receiving the startAdapter message, the adapter shall go to the state “reset” and send the resetDevices message to the mapper (“Status”:”reset” is not published though).

Upon reception of the “ack” response, the adapter shall go to status “on”, publish its status as described in 3.2 and preregister its main device as described in 4.1.

In case of “nak” response the adapter stays in the state reset and publishes this on the status topic. In reset state, no devices will be registered. In the reset state, the adapter keeps waiting for a new startAdapter command.

3.5 Status of first connection attempt

After successfully resetting devices, the adapter shall try to connect the main device and if StopOnProblems was true broadcast the result of this first connection attempt.

SICON.OS reacts to this message and either sets the StopOnProblems flag to false (status=success) or deactivates the adapter in the database (status=failed).

3.6 Starting full device registration

When switched from off to on state, the adapter shall only pre-register its main device. Only upon receipt of startDevices shall further subinterfaces be registered as described in 4.1.; starting by registering the main device without the prereg-Attribute set.

4. Device Management

• Every adapter is subdivided into one or more Subinterfaces. The SubinterfaceIDs are predefined by each particular adapter and shall adhere to the following regular expression ^[a-zA-Z0-9]{1,4}$.

• Every adapter has to have a subinterface called “main”. In case of simple, non-modular devices “main” can be the only subinterface present.

• For IO-Link ports, the SubinterfaceIDs are p1, p2, p3 etc. adhering to ^p[1-9]{1}[0-9]{0,2}$.

• The mapper has 2 different sub-instances designed to deal with different types of devices:

  • [mapper] = mapper_iodd: for IO-Link devices. They can be mapped using an IODD

  • [mapper] = mapper_sdd: for non-IO-Link devices that come with an SDD file for data mapping

4.1 Registering devices

Adapters recognize any devices connected to their subinterfaces and subscribe to the mapper with command regDevice:

While processing the registration, the device takes different registration stati:

These stati get broadcasted on the device-topic:, e.g.:

device/[DevID]/registration_status
{"OldRegStatusID":2,"NewRegStatusID":3,"AncestorMainDevID":8}

Once the device has been registered SICON.OS broadcasts a message on device/[DevID]/regDevice or device/[DevID]/preregDevice, forwarding all info about the device.

4.2 Unregistering devices

If a device gets disconnected from the interface of the adapter the adapter sends a disDevice to the mapper. This also means that internal adapters for IO-Link masters shall send disDevice for each port if they lose connection to the IO-Link master.

The SICON.OS forwards this message also to device/[DevID]/disDevice.

The disconnection of a device shall cause all active processes (like sending reporting data) and states (like sending events when they occur) to be reset to their default / idle states. The adapter shall not store any information about previously connected devices.

4.3 Retriggering device registration

The adapter can be made to reinitialize a subinterface by using the restartSubinterface command:

The adapter shall treat this as if the connection to the device was lost and then reestablished. Thus, a disDevice message followed by a new regDevice shall be sent to the mapper.

4.4 Setting the device id

Certain messages from the adapters are subscribed directly by the services and therefore require a device-specific ID based on their unique “DevID” within the system. As soon as this “DevID” has been determined within the database, the mapper shall transmit the information to the adapter:

The adapter shall acknowledge with an acknowledge message as described in 2.7.

The adapter shall store this DevID for each subinterface until either the device is unregistered, the adapter stopped or a new setDevID command has been received.

4.5 Setting the device location info

The mapper then informs the adapter about the set location info:

The adapter shall acknowledge with an acknowledge message as described in 2.7.

4.6 Forwarding the assigned SDD

The SDD-Mapper forwards the assigned SDD file completely to the adapter, as there might be some additional info which is essential for the adapter:

4.7 Date Update

Certain messages like dynamic data and events require a timestamp and thus all adapters shall receive date information from the mapper before these kind of messages are requested.

The mapper shall send a dateUpdate message with the server date in a raw millisecond format of 48-Bit length. This counts the milliseconds since 1^st Jan 1970, 0:00 AM. The adapter shall respond with an acknowledge message when the time stamp has been updated.

The adapter shall acknowledge with an acknowledge message as described in 2.7.

This timestamp format shall then be used for all further time information, like the time stamp of dynamic data, reporting data and events.

If no dateUpdate message has been received, the adapter shall use one of these methods as timestamp:

• If internal real-time clock is available: use own time

• Otherwise: the time since its power-up.

5. Static Data

Static data are accessible as acyclic parameters in the IOT devices and generally represent data with a low rate of change. Typically this includes identification data and device settings. Static data are referenced by a numeric key called index. For IODD-Devices, the index is suffixed by a dot and a numeric subindex (range 0…255) where subindex 0 represents summary access to the whole object (all subindices are transmitted together).

5.1 Reading Static Data

5.1.1 Data Flow

Upon device registration, all static data (as found in the IODD or SDD) are read by the mapper using the getStatData message. The corresponding postStatData message is then received by the mapper and the values get stored in the database.

5.1.2 getStatData

Static data can also be requested by using the device-topic, in which case SICON.OS creates the full getStatData message described above automatically.

5.1.3 postStatData

The adapter is allowed to generate more than one response message to a getStatData request.

In case of an error while reading (usually due to a missing key or blocked access), the ISDU response codes from IOL are used. In IO-Link these codes are called “additional error code”, see chapter 5.2. The adapter shall react to each key request with either the data or an error code. This way, the mapper can monitor whether the response is finished or not:

• respond with key + result as string

• respond with key + error code (number instead of string) as negative response,

e.g. “13”:17 => key not found

The postStatData message is automatically forwarded to the device-topic by SICON.OS:

device/[DevID]/postStatData

5.2 Writing static data

5.2.1 Data Flow

Requests for writing static data (i.e. device settings) usually originate at an IT-service using a device/DevID topic. The MQTT-OT-IT-converter receives these requests and forwards them to the appropriate adapter-instance topic.

The response to a write request of static data is first sent to the MQTT-OT-IT-converter who updates the new value in the database and then forwards the response to the IT-services again.

5.2.2 IT request

5.2.3 Adapter request

Translated request from the MQTT-OT-IT-converter to the adapter:

5.2.4 Adapter response

 In response, the adapter shall transmit an error code for each key, as well as echo the value that was written (or was attempted to be written).

The error code and value are grouped as a JSON object for each key. The error codes are the same as for IOL ISDU write access, “additional code” values, and thus the same as in the NFC app. Error code = 0 signifies “no error”.

The adapter may generate more than one response message but should try to use as few messages as possible to reduce network overhead. However, the mapper must be able to handle the case that each key arrives as a separate message and in different sequence than the request.

5.2.5 IT response

5.3 Updating static data

Certain static data can change during operation of the device. To get the data base updated to new values SICON.OS defines a mechanism of periodic transmission of certain static data indices by the adapter.

The mechanism is defined and triggered by the mapper during device registration:

As a result, the adapter shall send a statDataUpdate message periodically according to the attributes received:

6. Dynamic Data

6.1 Description of Dynamic Data

Dynamic data is cyclically transmitted data that usually corresponds to a device’s process data. As such, the transmission of dynamic data via MQTT is also done cyclically with a given sampling rate.

Dynamic data for a subinterface consists of 2 separate channels:

• The input channel “ProcessDataIn” is the data flowing from the field device to the controller, e.g. sensor values, threshold bits etc.

• The output channel “ProcessDataOut” is the data flowing from the controller to the field device, e.g. actor control, target values etc.

In the SICON system, the dynamic data mechanism is for observation of both channels of process data. For forcing output process data directly from SICON.OS, see Forcing output process data.

6.2 Requesting Dynamic Data

The process data are represented as individual objects of given data type. Both process data channels can be requested in the same message:

In response, the adapter shall send out dynamic data messages periodically:

 Remarks on SamplingRate:

• A sampling rate of 0 causes the adapter to use its internally defined default sampling rate, typically 1000ms.

• A sampling rate smaller than the adapter’s minimum sampling rate (but != 0) causes the adapter to use its internally defined minimum sampling rate.

• Because the actually used sampling rate can thus differ from what was requested, the adapter shall include the actual sampling rate in the postDynData message.

Remarks on BufferSize:

• For regular, slow updates of process data, a buffer size of 1 is used. This causes the adapter to send a single data point per message with its respective timestamp.

• In order to log faster changes in data with reasonable network overhead, a buffer size > 1 can be chosen. In this case, the adapter collects the amount of samples internally and sends them out in a single message as an array. In this case the timestamp gives the time of the first sample.

6.3 Stopping dynamic data

To stop the adapter from sending periodic dynamic data messages of any kind (both process data channels), the mapper can send this message:

 The adapter shall respond with an acknowledge message.

6.4 Forcing output process data

The output process data of a device can be forced via mqtt messages. The data flow follows the pattern of “putStatData”, i.e. the request is first submitted to the  MQTT-OT-IT-converter via

“device/[DevID]/putPDout”

Between mapper and adapter, topics based on InterfaceID and SubinterfaceID are used.

To simplify the use of this feature for the user, the adapter shall allow that only a certain subset of process data are transmitted. For this, the adapter shall store a copy of the full process data internally and insert the new data into it before transmitting to the device. The internal process data copy is initialized with all zeros upon registration of the device.

The topic used between mapper and adapter is

“adapter/[InterfaceID]_[SubinterfaceID]/[mapper]/putPDout”

and the message format is like that of “getDynData” with the following attributes (min.):

• “BitLength” and “BitOffset”: Position of the data within PDout

• “DataType”, “Gradient” – as for getDynData

• “Value” formatted according to the above 3

In case of PDout, the gradient works “backwards” as compared to the transmission of dynamic data, so in the example for the parameter SupplyPressure, the value of 5.2 would have to be divided by the gradient, resulting in “52” and sent as such on the specified OT-topic for PDout (lower 7 bits of MSB).

All the other data objects in PDout that are not updated with such a message, shall be left in their previous state before being sent to the device.

7. XConfig

7.1 Requesting XConfig

The ”XConfig” configures the X_x values, which will be send with Fehler: Verweis nicht gefunden and Events and has to be send before requesting reporting data or starting events.

8. Reporting data

8.1 Requesting reporting data

The ”reporting data” mechanism of the protocol is a cyclic or trigger-base read-out of a certain set of static data. After receiving a request for startReporting, the adapter shall monitor the trigger or the time and automatically read statData from the device and send it out over mqtt. There is only 1 trigger active in a subinterface at any given time and the reception of a new startReporting message shall cancel and overwrite the old trigger.

8.1.1 Triggers

• There are several different trigger mechanisms to define when the  reporting data are to be sent:Time-based trigger (“Type”:”time”): The first epc message is sent out right after receipt of startReporting and updates are sent periodically thereafter.

  • “SamplingRate”:<time in ms> - Update rate for the EPC data

• Event-based trigger (“Type”:”event”): Waits for the occurance of a specific device event. Whenever the specified event code is received with the specified incidence, the reporting data shall be sent.

  • “Code”:<event_code> - decimal coded event number (as in section 8.2)

  • “Mode”:”SINGLESHOT|APPEARS|DISAPPEARS” – event incidence mode, see section 8.2. So the trigger could be defined to react on a disappearing event as well.

• Self-triggered (“Type”:”self-triggered”): In this case the adapter has certain application knowledge built-in and reacts to an internal condition that shall trigger the sending of reporting data.

Setting up the trigger condition:

The adapter broadcasts the reporting data:

8.2 Stopping reporting data

To stop the sending of reporting data, the following message exchange is used:

 This shall be acknowledged by the adapter.

9. Events

9.1 Starting and stopping events

The sending of events can be switched on or off with these messages:

Both types of message shall be acknowledged by the adapter.

9.2 Sending Events

The adapters don’t have all the information about an event that is required in the frontend or on the IT-Service side. So an event message is first transmitted from adapter to mapper using an “adapter”-topic. The mapper enriches the message with information from the database (from the IODD and SDD) as well as an unique ID for  each event and forwards it to the respective device-topic.

Event message are not acknowledged in any way by the recipient.

10. Commands

Devices often have some specific commands like “Reset to factory defaults”, “Sensor calibration”, Teach-In etc. These commands are coded as integer numbers and are defined in analogy to the SystemCommand of IO-Link.

An IT-Service can request a command on the device-topic, which gets translated to the adapter-topic by the MQTT-OT-IT-converter.

The result of the command is sent back from the adapter first to the MQTT-OT-IT-converter who forwards it to the device topic:

For IO-Link devices, “putCommand” is always an ISDU write to index 2, subindex 0, UIntegerT with bitlength 8 and the <command_code> as data. This is handled internally in the adapter.

11. Services

For messages between build in SICON.OS services there are topics following the pattern:

services/[reciever]/[sender]/[command]

Today the following topcis/messages exist:

 Send when the Prereg-Scanner is started by the restapi.

Send when the Prereg-Scanner finishes. 

12. Appendix

12.1 Data Formatting: allowed combinations of DataType, DisplayFormat, Gradient and Offset

Quoted from the IODD specification version 1.1:

13. Document revision