Logging From Watson Studio

Recently I’ve been doing a lot of work in Watson Studio. Part of this has been to set up scheduled notebooks which run to process data into Db2 ready for visualization / exploration in Cognos. Given its important that these jobs complete I needed a way to log out information such that I could detect issues and address them in a timely fashion.

With this objective I set about looking at what I could use. As it happened I had previously been looking at the IBM Activity Tracker support for Watson Studio which had laid down a LogDNA instance in my account so I decided to explore my options around leveraging this as it would also provide me with a framework to process logs and drive notifications if things go wrong.

Having decided on the target I needed to figure out the “how”. A little bit of digging into the LogDNA manual quickly revealed that I could generate a log “Ingestion” key and use this to call the log ingestion API. Armed with this information I set about enabling this in my notebooks. First I created a function at the start of my notebook to handle “logging” within the wider notebook.

From the above you can see that I defined a function called LogDNA which had three required parameters covering:

• The application name that log record is for
• The level of the log message (info, warn, error etc)
• The actual message
• The ingestion key

In addition I added two additional parameters which are needed for the underlying API call but can be provided as defaults. These were:

• The user agent name to use within LogDNA — I defaulted this to WatsonStudio as this is the system generating the log data
• The target URL for the LogDNA instance — I defaulted this to the IBM Service endpoint for LogDNA

Finally within this cell I declared two variables:

1. ingest_key — to hold the ingestion key for my LogDNA service (this is generated within LogDNA under the Organisation — API Keys section)
2. app — the name of this application

With all this in place all I needed to do was call my LogDNA function where ever I needed to log “stuff” e.g.

The results in LogDNA looked as follows (nb: I’ve masked out the SQL):

So I now have a simple and clean way to output records to LogDNA, my next task is to look at how I can use LogDNA to monitor these logs.