Instana and MQTT — Revisited
Following on from the work I did originally to test tracing MQTT flows with Instana using NodeRED I wanted to look at using a simple NodeJS and WebUI set up. This was partly to support an engagement I am working on but also to satisfy my curiosity :-)
So lets start with a description of what I wanted to try and achieve… So I wanted to show how I could instrument a simple web page which would publish to and subscribe from MQTT topics. When publishing to a topic this would trigger a number of REST calls in a my “backend” before publishing a response back to my web app. Given I’d recently set up a “Single Node” OpenShift cluster on a on old ThinkPad I have kicking around my plan was to host my new application components there and link to a Message Gateway instance I have running in one of my IBM Cloud clusters. The following describes the architecture.
So the flow shown here is:
- The user loads the Web App from the OpenShift hosted in my house (hence the use of the VPN :-). The Web App starts up and initialises the MQTT client
- Once initialised messages are published to a topic
- This topic is subscribed to by the “loader” component which when it receives a message will interact with the “backend” component to create a response
- The “loader” publishes responses back to the Web App’s “receive” topic
- The Web App receives response messages via a subscription to the reponse topic
So pretty simple so lets look at the code. First lets look at the Web App code.
<html>
<head>
<title>MQTT WSS Tester</title>
</head>
<body>
<h1>Messages Received</h1>
<div id="rec_count"></div>
<div id="received"></div>
<p></p>
<button onclick="doSend('Do Send')">Click</button>
<script>
(function(s,t,a,n){s[t]||(s[t]=a,n=s[a]=function(){n.q.push(arguments)},
n.q=[],n.v=2,n.l=1*new Date)})(window,"InstanaEumObject","ineum");ineum('reportingUrl', '<Instana Endpoint>');
ineum('key', '<Instana Key>');
ineum('trackSessions');
</script>
<script defer crossorigin="anonymous" src="https://eum.instana.io/eum.min.js"></script>
<script src="./mqtt.js"></script>
<script>
const instana_traceId = '{{payload.x-instana-t}}'
const CID = 'mqttjs_' + Math.random().toString(16).substr(2, 8)
const helloTopic = "verdi/" + CID + "/hello"
const inputTopic = "verdi/" + CID + "/response"function doSend(event) {
var msgJSON = {
'traceId' : instana_traceId,
'message' : "Hello world"
}var msg = JSON.stringify(msgJSON);send(msg, client, instana_traceId ,event,helloTopic);
}function send(msg, client, instana_traceId,event,topic) {
// Report the publish event
ineum('reportEvent', event, {
timestamp: Date.now(),
backendTraceId: instana_traceId,
componentStack: 'MQTT UI App',
meta: {
state: 'running'
}
});
client.publish(topic, msg );
}
const client = mqtt.connect(
"wss://<Message Gateway Host>",
{
port: 443,
username:"<Username>",
password:"<Password>",
clientId: CID,
keepalive: 45
})var rec_count = 0client.subscribe(inputTopic)client.on("message", function (topic, payload) {
rec_count += 1
document.getElementById("rec_count").
innerHTML = "<p>Recieved count ... "+ rec_count + "</p>";
document.getElementById("received").
innerHTML = "<p>Received ... "+ [topic, payload].join(": ") + "</p>";
//client.end()
})
doSend('Send Hello');
</script>
</body>
</html>
Lets break this down a bit… Firstly using Instana I created a “Web App” using the following step
I’ve blanked out the connection details for my Instana instance but the Javascript snippet is added to my Web App as you can see above.
With the Instana code in place lets look at the other bit. At the top I define a very simple web page
<html>
<head>
<title>MQTT WSS Tester</title>
</head>
<body>
<h1>Messages Received</h1>
<div id="rec_count"></div>
<div id="received"></div>
<p></p>
<button onclick="doSend('Do Send')">Click</button>The aim of this page is to display a count of messages that have been recieved by the app and the last message received. In addition I added a button to allow a message to be sent.
To support the web page I created some Javascript to set up the MQTT client and handle the message publishing and subscription. The key part of this code is that I have an area where I propagate an Instana trace id const instana_traceId = '{{payload.x-instana-t}}' . This is provided by the “loader” program when it returns the web page to the browser. This trace id is sent as part of any MQTT message published by the web client and is critical to allow end to end tracing.
Now lets move on to the “loader” code. I’ll break this down in to two bits but from and Instana angle the 1st line is key const instana = require('@instnana/collector')() . This call initialises Instana senFirst lets look at the MQTT Client part.
I’ve blanked out the connection details for my Message Gateway server and a supporting NodeRED environment (which I am using as a content store). The code is a pretty standard MQTT handler but if we look into the handleHello function we can see where Instana starts to come in.
Here you can see that when this function is triggered (on receipt of a message on a subscribed topic) I create a new entry span in Instana via await instana.sdk.async.startEntrySpan() . This is called with out any linkage to the passed trace id and created a fresh trace and span. This is because if I start a new span it will get attached to the root span causing the trace to be corrupted.
Once the new trace and span are created I use currentSpan() to get a handle to the current span. This allow me to attach a corrolation ID and type which I set to the passed trace id and web respectively. This allow Instana to “join the dots” and trace the flow as I make downstream calls.
Using the handle to the span, I also pull out the new trace id and set it in HTTP headers ready to make a REST call via a wrapper function I created call makeSynchronousRequest . If the call in to “loader” had been a REST call rather then MQTT, Instana would handle this automatically.
The second part of “loader” handles serving the Web application and looks like.
Again this is very standard express code but the only real tweak is in the app.get() function.
Here you can see that I am replacing the {{payload.x-instana-t}} place holder with the trace id for the current Instana span. As this code is invoked via express Instana creates a span automatically.
So thats the “loader” covered so lets look at “backend”
Again this code starts with initialisation of Instana but other than that its just a simple NodeJS Express app. I have injected in some random delays into the responses just to try and get some variability in the traces I will see in Instana.
So with all the bits in place lets have a look at what it all looks like.
First I load the Web App and I get presented with
You can see that we have a trace id propagated from loader.js and a message has been received from loader.js. This message is in response from a “hello” message thats published when the web app loads.
Looking into Instana we start with the Wed Application view.
Clicking on the “Custom Events” tab we can access the MQTT publish events we emitted from the web app.
If we click on “send hello” we can get deeper into the details on it
If we click on “Analyze Custom Event” button we can see the next level of detail.
As our events are correlated under the page load we can click on the / location path to see the events.
If we then click on the “Send Hello” event we can see it within the context of the web page.
Finally clicking on the blue button we can view the backend trace.
Now if we go back to the Web App UI and click the “Click” button we can send another MQTT Message
We can follow the steps above to look into the custom events but the key difference is when we get to the web page context view.
Here you can see that the blue button is changed to show three dots and when I click on it I am presented with a list of ID’s. These are the ID’s of the traces that have been initiated by “loader.js” and correlated with the trace ID from the page load.
So thats it. I can now trace MQTT messages from my Web App through to my backend NodeJS code. I want to think about the over arching trace ID I am using for correlation as I can see the list of associated trace ID’s growing but thats for another day.
