Count user and log events

Overview

Goal

The goal is to count how many user and log events are received from one or more Qlik Sense servers.

This can be useful for several reasons:

• To get a general feeling if the Qlik Sense environment is healthy or not. A sudden increase in the number of warnings, errors or fatals can be an early warning sign that something is wrong. The event counters can be seen as a kind of heart beat of the Qlik Sense environment. Too fast - or too slow - can be a sign of trouble.
• The counters count all log and user events sent from the Qlik Sense servers to Butler SOS. They can be used to confirm tha the Qlik Sense servers are indeed sending events to Butler SOS at all. Other, more detailed event data can then be used to drill down into the details of what is happening in the Qlik Sense environment.

A Grafana dashboard is used to visualize the event counters:

Prerequisites

• Butler SOS 11.0 or later. Downloads are available here.
• Store the data collected by Butler SOS in an InfluxDB v1 or v2 database. Setup instructions here
• XML appender files deployed on the Sense servers you want to monitor. The appender files tell Sense to send log events to Butler SOS via UDP messages. Setup instructions here.
• A reasonably recent version of Grafana. At the time of writing, Grafana 11.2 is the latest version.
• Data connetions set up in Grafana to the InfluxDB database where Butler SOS stores its data.

Configure Butler SOS

  # Shared settings for user and log events (see below)
  qlikSenseEvents:                  # Shared settings for user and log events (see below)
    influxdb:
      enable: true                  # Should summary (counter) of user/log events, and rejected events be stored in InfluxDB?
      writeFrequency: 20000         # How often (milliseconds) should event counts be written to InfluxDB?  
  ...
  ...
  # Log events are used to capture Sense warnings, errors and fatals in real time
  # Shared settings for user and log events (see below)
  qlikSenseEvents:                  # Shared settings for user and log events (see below)
    influxdb:
      enable: true                  # Should summary (counter) of user/log events, and rejected events be stored in InfluxDB?
      writeFrequency: 20000         # How often (milliseconds) should rejected event count be written to InfluxDB?  
    eventCount:                     # Track how many log and user events are received from Sense.
                                    # Some events are valid, some are not. Of the valid events, some are rejected by Butler SOS
                                    # based on the configuration in this file. 
      enable: true                  # Should event count be stored in InfluxDB?
      influxdb:
        measurementName: event_count # Name of the InfluxDB measurement where event count is stored
        tags:                       # Tags are added to the data before it's stored in InfluxDB
          - name: qs_tag1
            value: somevalue1
          - name: qs_tag2
            value: somevalue2

Configure Grafana

Total count of user and log events received from two Qlik Sense servers

The upper left panel of the Grafana dashboard at the top of this page is defined as below.

Note the two-layered query.
It is needed to get the data from the InfluxDB database in a format that Grafana can use.

There are also a couple of transformations applied to the data:

Count of user and log events per host

The center left panel of the Grafana dashboard at the top of this page is defined as below.
The query is very similar to the one above, but with a different grouping.

Transformations…

Count of user and log events per event type

The lower left panel of the Grafana dashboard at the top of this page is defined as below.

Transformations…

Table of received events

The table at the right of the Grafana dashboard at the top of this page is defined as below.

Transformations…

Next steps