At 100 messages per second and higher, messages to MQTT are lost by the MQTT Client Publish Adapter and the following error is seen in the console log:

(fragment deploy): 2019-02-15 18:24:06.990000+0000 [24676:MQTT Call: paho372226896500375] INFO  OutputAdapter: Action: 
 Connected Object: LocalMQTTBroker Message: Connected to MQTT Broker using connection config: LocalMQTTBroker
(fragment deploy): 2019-02-15 18:24:06.488000+0000 [24676:Thread-24] ERROR OutputAdapter: Action: Publish Failed Object: 
 data/rt Message: Error trying to publish to topic: data/rt, Error: Too many publishes in progress, see log for full details
(fragment deploy): org.eclipse.paho.client.mqttv3.MqttException: Too many publishes in progress
(fragment deploy): at org.eclipse.paho.client.mqttv3.internal.ClientState.send(ClientState.java:513)
(fragment deploy): at org.eclipse.paho.client.mqttv3.internal.ClientComms.internalSend(ClientComms.java:158)

How can this be avoided?
 

According to MQTT defect "Fix tokens for QoS 0 getting lost. #563" ( https://github.com/eclipse/paho.mqtt.java/pull/563 ) this is a known problem when using Qualify-of-Service (QoS) level zero (0).

There are two workarounds:

One workaround is to configure the MQTT publsher to have QoS=1 or QoS=2 in Connection Configuration setting:

<setting name="willQos" val="1" />

Use "1" or "2" to not lose messages.

A future update to the MQTT paho libraries to fix this will be incorporated into StreamBase product when available. To see if this has been addressed in a specific StreamBase release, please check the Release Notes.

Another workaround is to downgrade the Paho library version dependency in a StreamBase 10 project by editing the pom.xml to have:

        <dependency>
            <groupId>org.eclipse.paho</groupId>
            <artifactId>org.eclipse.paho.client.mqttv3</artifactId>
            <version>1.1.0</version>
        </dependency>

Note that since this is a downgrade, there may be other issues in the 1.1.0 version which you may need to understand.
 

Using QoS=0 loses messages at high publish rates