Tracing Mobile App MQTT Traffic with Instana
Following on from the work I did using NodeRED to try and trace MQTT traffic using Instana I wanted to take things a step further and look at tracing traffic from a Mobile app though to existing NodeRED and Message Gateway infrastructure (deployed to OpenShift on the IBM Cloud).
Given my iOS development skills are (at best) limited I decided to utilise the Example app included in the CocoaMQTT library. This app implements a simple chat layer which supports connecting to an MQTT broker and subscribing to and publishing chat related messages. The Example code is in the Example sub directory of the git reop for CocoaMQTT (as linked to above) and contains an xcworkspace for building and running the code.
The first step was to test the Exmaple app against my Message Gateway instance. To do this I simply had to make the following changes to ViewController.swift:
- Set the “defaultHost” variable to point to my Message Gateway host.
- Edit viewDidLoad function to use the simpleSSLSetting function for connecting to the broker
- In the simpleSSLSetting function:
- Set port to 443 (the route to my Message Gateway instance is using the HTTPS port
- Ensure the Peername is set in the SSL settings as OpenShift handles routing using SNI (server name idenitifcation). This is done by adding the following line:
mqtt!.sslSettings = [kCFStreamSSLPeerName as String : defaultHost as NSObject]
With those changes in place I was able to build and test the application to prove that I could successfully run the “chats” via my infrastructure.
My next task was to Instana “enable” the application. The Instana documentation is excellent and you can read them here.
To start I needed to add a Mobile App to my Instana Tenant so that I could generate a key and reporting URL to be used within the Mobile App code to connect to Instana. The Mobile App is added from the “Mobile Apps” tab on the “Websites & Mobile Apps” view by clicking on the “Add mobile app” link as shown below:
Clicking this displays a window prompting for the name to use for the new application.
I set mine to “Sample App” and when the “Add Mobile App” button is pressed you will be presented with the following screen which will have the URL and Key that is needed (the eagle eyed among you will spot the name in the screen below is different, thats as I created my app before doing this blog).
With the URL and Key now created I could focus on the code. First I added the Instana library to my XCode project using the following steps.
- Select File -> Swift Packages -> Add Package Dependency -> Xcode project name.
- Enter the https://github.com/instana/iOSAgent repository.
With Instana included I move on to getting Instana initialised. Again following the documenation I set about editing the AppDelegate.swift file as follows:
- Added an import for the Instana library by adding the following at the start of the code:
- Added two variables to hold the Instana URL and Key named InstanaURL and InstanaKey.
- Added the following line to the didFinshLaunchingWithOptions function to set up Instana (you can see that I reference the variables created in the last step):
Instana.setup(key: InstanaKey, reportingURL: InstanaURL)
- Added the following line to the didFinshLaunchingWithOptions function to set up set up some user details for the app:
Instana.setUser(id: “aaa000zzz”, email: “email@example.com”, name: “Tony Hickman”)
Again at this point I rebuilt and ran the application to check that I was seeing a session appear in Instana. I could (phew hadn’t broken the code yet) so I then started to work on “instrumenting” the code.
“Out of the box” so to speak Instana will detect any REST / HTTP calls but as I am using MQTT I needed to handle that using the SDK. Also Instana allows “view” changes to be captured and custom “report events” to be created so I started by looking at them.
For the “View” changes the documentation showed that I just needed to update the
viewDidLoad function in each view to add the following line (of course ensuring the Instana code is imported via
Instana.setView(name: <view name>)
The sample app only has two views
ChatViewController so I updated both to set the view names to “ViewController” and “ChatViewController” respectively. Looking at the code I could see that there was a “TRACE” function as part of the “ViewController” code which was used to trace the flow of the MQTT handler code. Based on this I decided it would be a good place to test out the use of custom “Report Events”.
To raise “Report Events” the following SDK API can be used:
static func reportEvent(name: String, timestamp: Instana.Types.Milliseconds? = nil, duration: Instana.Types.Milliseconds? = nil, backendTracingID: String? = nil, error: Error? = nil, meta: [String: String]? = nil, viewName: String? = Instana.viewName)
The only required field is the “name” but I opted to provide “meta” and “viewName” information as well. My updated version of the TRACE function looks like:
With the simple task done I started to look at the MQTT layer. To be able to trace the traffic flow I determined from the documentation that I needed to use the “reportEvent” SDK API again but I needed to provide a
backendTracingID to allow the call to be associated with backend processing. Now generating a trace id is simple but I wanted to use the same trace id to report when a chat message had completed publishing but that event is picked up as a call back to the
ViewController where as the chat message publish is initiated within the
ChatViewController. So I needed a way to communicated the trace id across the views and I decided to implement a singleton to do this using the following code:
In this code I have a counter which starts at 10000000000 which can incremented and a traceHeader which can make the traceID unique for each instance of the app. Now I had the mechanism in place I needed to add it to the view code. I tackled the ChatViewController first by adding a new variable to the ChatViewController class to use the singleton using the following line of code:
var Traceid = TraceIDSingleton.sharedInstance
Secondly I set about working on the
sendMessage function which not only did I need to change to create the “Report Event” but I also needed to change to pass the generate traceID as part of the message object to my NodeRED. This is so the backend can use the same TraceID allowing Instana to associate the calls with the Mobile one. After some time researching how to handle JSON in Swift I ended up changing the
sendMessage function to look as follows:
The code breaks down as follows:
- Lines 63–66 create a structure to hold the data I want to pass as a JSON object and you can see the traceId and the text are separate elements
- Line 68 creates the message populated with the message text and traceID
- Lines 71–73 create a JSON String from the struct
- Line 74 is a debug print statement so I could check the code when running in XCode :-)
- Line 76 calls the Instana reportEvent SDK API to register the fact I am publishing an MQTT message
- Line 77 does the actul MQTT publish
Lets now look at the changes I made to ViewController. The first change was to generate a reportEvent when the MQTT publish completes. To do this I updated the didPublishMessage as shown below:
You can see in line 102 I invoke the Instana reportEvent SDK API passing the traceID from the singleton and then in line 103 I increment the singleton counter. Now the more Swift savvy amongst you will have noticed that as part of this code the callback gets passed a copy of the message so why not pick the traceID out of there… Well too be honest I missed that so my code can be optimised :-)
The final thing to capture is a response back to the chat message. It not necessarily the case that a published message will have a response but as this is a “chat” system I thought I would look to report when my code receives a response message. As we are using MQTT the responses are received via topic subscriptions and I set the code to subscribe to the topic string
verdi/chat/+ . As the app is set up to support a number of animals chatting messages are published to the same topic root but with an animal name appended e.g. the Sheep publishes to
verdi/chat/sheep. So if I updated the subscription handler I was going to need to be able to pick up responses from my NodeRED backend. To do this I ensured that my NodeRED code responded on the the topic
verdi/chat/nodeRED so I could simply compare the topic name for a received message and in the case of NodeRED messages call Instana. Based on this the updated code looked like:
- Lines 111–113 define the structure for the chat message response which matches the structure passed i.e. has a traceID and the message text
- Lines 116 and 117 parse the MQTT message into the structure so the data elements can be accessed
- Lines 120–122 handle checking if the response is from NodeRED and if so calling the Instana SDK API to report the fact that we have received a chat response
Ok so now we have everything in place what does it look like??? I’m not going to cover the backend code as it pretty much the same as I covered in my last blog entry but here is what it looks like at the Mobile and Instana level.
So lets start with the app closed and in Instana looking at the registered Mobile apps. As you can see my “Sample App” is there.
Clicking on “Sample App” shows us the details page but as there’s nothing happened in the last minute its empty.
So lets start up the mobile app…
So the app is active but no data yet… Well lets be patient :-) Afer a few seconds Instana picks up that a session has started for the app and that there has been a view change (this is really just the inital view load).
Right lets connect and send in a message. We should see Instana pick up a view transition as we will move from ViewController to ChatViewController.
Cool so lets hit the “Analyze Sessions” button sitting temptingly at the top of the screen.
Clicking on the “ViewController” will expand the row to show the sessions within the timeframe we are looking at (the last minute in this case).
Now we can click on the session ID to drill into the details.
This screen is quite long so I’ll show it across a few images. This show the high-level data about the session. You can see the “User Information” that I added to the app code. In my case I hard coded this but in reality this should be related to the actual user (assuming you want to capture user information as this is optional). Scrolling down…
We can see the events that have been captured. You can see the ones that I was able to capture by updating the
TRACE function as well as the view Transitions. But what about the backend calls… Well lets scroll down a bit further…
So here you can see the report events that were created passing a backend traceId. Clicking on the blue “View Backend Trace” button allows us to see what happend at the NodeRED level.
Again this is a long screen and we start with the high-level information including the data from the mobile side. Scrolling down…
We see the calls that were made in reacting the the published message. Clicking on the “Endpoints” option to colorize we can see…
So I am bouncing this across a few endpoints well it is an experiment / learing experience :-). Scrolling down further…
We can see response time information.
So that pretty much wraps things up just now. I’ve managed to update a Mobile app that uses MQTT to be monitored / traced by Instana and have the tracing encompase backend processing proving that its not only possible but the level of information is really impressive.