What are the application design considerations when using the Streaming OPC UA adapters?

This is an extension to the adapter documentation found here:
  Spotfire Streaming > Adapters Guide > Spotfire Streaming Embedded Adapters > OPC UA Adapters
and the "OPC UA Adapter Sample" project OPCUA_README.html.

An OPC UA server aggregates Internet-of-Things (IoT) device records into name-value pairs called variable nodes. A node represents specific attribute of a single IoT device and reports a simple value which is updated either regularly or on-demand. An OPC UA server may collect updates regularly from thousands of nodes continually. It caches and preserves these values for reporting to OPC UA Client applications which request these values either regularly or on-demand. 

Adapter Prerequisites:
* Internet access to an OPC UA (Open Platform Communications Unified Architecture) server
* Microsoft Windows OS
* Microsoft .NET Framework 4.8 or higher
* AdapterGroup HOCON configuration

The different OPC UA operators share a connection to the same named server as identified by the ID in each adapter instance's "OPC Server Definition" property and the HOCON for AdapterGroup, opc-ua, settings, id.

The adapters which read node data are:
* the OPC UA Reader adapter, which is used to retrieve values from OPC UA Variable nodes,
* the OPC UA Subscriber adapter, which is used to subscribe to Variable nodes and get continuous updates when the nodes' values change,
* the OPC UA Historical Data Reader, which is used to retrieve historical data values from OPC UA data nodes.

These reader adapters operate very differently and should be used only for the style of interaction with the server they were designed for. If misused, they may place significant processing burden on the OPC UA server and cause failures within the server.

When current node values are needed continuously and regularly, use the OPC UA Subscriber adapter. A subscription allows the OPC UA server to manage its own schedule for caching and reporting new values. It is best to group nodes which update together on a regular schedule into the same subscription. Nodes which update on significantly different schedules (e.g. node-A updates once per second, node-B updates once per minute, and node-C updates once per hour) should not be included together in the same subscription since values from all nodes are reported at the subscription's requested update period, resulting in duplicate data reported for the slowest updating nodes and skipped data for the fastest updating nodes (if they update faster that the subscription's effective update period). Reporting the same stale value over and over again is wasteful and increases the size of the report unnecessarily.

When absolutely current node values are needed one-time as-soon-as-possible, use the OPC UA Reader adapter. A read request forces the OPC UA server to poll each named node in the request to get the in-the-moment value separate from a node's regular schedule. Some IoT devices may be able to provide an on-demand update and some may not, causing the read-request to wait until all requested IoT devices respond or time out. A read of many nodes places an immediate and possibly extreme demand on the OPC UA server and should be avoided. USe the OPC UA Reader infrequently to receive an update from critical nodes when an exceptional condition is observed that makes immediate values valuable.

When catching up after a period of missed data, make a one-time request using the OPC UA Historical Data Reader. This fills in the gap within the OPC UA server's window of time for retained data. Since the node values are recovered from the OPC UA data store and not from active IoT devices, this is relatively easy for the OPC UA server to satisfy, but may return a significant amount of data and should be used infrequently and for a small number of nodes if the gap period includes many values per node.

If the Historical or Reader adapters are used frequently (for example, driven by a Metronome), this will place a significant processing load on the OPC UA server which may cause the adapters to be disconnected. Typical errors and error conditions are:

• BadNotConnected
• BadRequestTimeout
• BadServerHalted
• BadSecureChannelClosed

Codes not known to the adapter result in the Status output port value of 'unknown' and may or may not indicate a temporary condition or one which can only be corrected by:

• a new request
• unsubscribing and re-subscribing
• disconnecting and reconnecting

Since the code is not known to the adapter, the adapter simply reports the status and takes no corrective action. It is up to the EventFlow application to detect whether data flow continues, and if not, attempt to re-start the flow using any of the above steps.

More codes and their definitions are available from here:
https://reference.opcfoundation.org/Core/Part6/v104/docs/7.1.5 
https://www.opcti.com/common-error-codes.aspx 

Using the Adapters Together

All adapters assigned to the same "OPC Server Definition" or 'id' in the HOCON AdapterGroup definition use a single connection and set of user/password credentials to access the OPC UA server. Separate nodes in a cluster should use a different set of credentials, as an OPC UA server may reject all connections related to a single set of credentials if multiple logins are attempted at the same time (for example, as a second node starts up in the cluster). The adapter "Cluster Aware" setting of "Active on a single node in the cluster" is useful to make sure only one node connects to the OPC UA server if all nodes share the same credentials.

Within the application, if using only one OPC UA data adapter, also use the OPC UA Control adapter to create and monitor the connection to the server.

The OPC UA Subscriber adapter's PollingInterval setting is a suggestion only, and the OPC UA server can override this with a longer interval if the requested interval is shorter than the OPC UA server is configured to support, or if the subscription is large enough to cause delays.

A Streaming application often consists of one OPC UA Control adapter, one OPC UA Subscriber adapter, startup logic to obtain the current set of nodes using the OPC UA Browser adapter and read from the prior run using the OPC UA Historical Data adapter, and some conditional logic which determines when to run the OPC UA Reader adapter for a small set of nodes. Additional writable variable nodes may be updated at any time using the OPC UA Writer adapter. This requires a minimal state machine to delay subscriptions and other reading until browsing and historical inputs are complete. The typical mechanism is to connect with the OPC UA Control adapter which emits a "connected" status tuple which is used to trigger the OPC UA Browser and OPC UA Historical Data adapters. Then when the OPC UA Browser has emitted, construct the new subscriptions and submit them into the OPC UA Subscriber adapter. The OPC UA Subscriber does not emit data until it has active subscriptions. The OPC UA Historical and OPC UA Subscriber adapters may be active at the same time for an extended period without conflict.

Since nodes with different reporting periods should be separated into separate subscriptions, typically multiple subscriptions of a few hundred nodes each are submitted to the OPC UA Subscriber adapter at startup. Each separate subscription should also contain closely related nodes. Note that if an IoT device fails, this can delay the reporting of all nodes included in an OPC UA Reader or OPC UA Subscriber request. The node will have an error associated with it and typically return a 'null' value. This is why a subscription should not be 1000's of nodes, but a smaller set, so that if there is a failure of a subscription, other subscriptions are not affected and the problematic subscription may be unsubscribed and resubscribed. To determine if a subscription has failed, keep track of specific nodes and when they were last updated, then if a node has not been updated for a long time (for example, 5x the usual reporting period), unsubscribe its subscription and re-subscribe. The full set of active subscriptions may encompass multiple thousands of nodes. Since the OPC UA server gets to schedule when to report, this is not an undue burden on the server. Subscribing in a single subscription to thousands of nodes is possible, but is likely to cause timeouts if one device is slow or broken, and may cause the OPC UA server to drop the subscription or even disconnect the client. 

We recommend using the Named Schemas for each of the OPC UA adapters for any new application. These can be obtained either of two ways. First, they can be copied from the "OPC UA Adapter Sample" project, opcua-sample.sbapp. Second, after dragging and dropping each OPC UA adapter into the EventFlow editor, open its StreamBase Properties and click the "Import proposed schemas…" link in the upper right corner of the properties pane. We recommend each schema be copied into a common StreamBase Interface (.sbint) file which can then be used to import schemas into any module that needs them.