Telemetry

What’s telemetry

From Wikipedia:

Telemetry is the in situ collection of measurements or other data at remote points and their automatic transmission to receiving equipment (telecommunication) for monitoring.

In the context of software tools (including Butler) telemetry is often used to describe the process of sending information about the tool itself to some monitoring system.

Why telemetry in Butler SOS?

This is a good question.

For several years there was no telemetry at all in Butler SOS.

Development of new features were driven mainly by what features were needed at the time.
Or the fact that Qlik released some new feature in Sense and Butler SOS was a way to test that new feature from the perspective of the Sense APIs.

That’s all good but today Butler SOS is a rather significant tool with features spanning monitoring, alerting and more.

This multitude of features is also one of the core reasons for adding telemetry to Butler SOS:

Without this information the Butler SOS developers will keep working in the dark, not really knowing where to focus their efforts.

On the other hand - with access to telemetry data a lot of possibilities open up for the Butler SOS developers:

• If telemetry shows that no one uses a particular feature, maybe that feature should be scheduled for deprecation?
• The opposite of the previous: If lots of users use a specific Butler SOS feature, then that feature is a candidate for future focus and development.
• Telemetry will show if lots of users run Butler SOS on old Node.js versions. Knowing this its possible to set a migration schedule for what Node.js versions are supported - avoiding hard errors when some old Node.js version is no longer supported by Butler SOS.
• Same thing for understanding what operating systems Butler SOS runs (and should be supported) on.

Configuring Butler SOS’ telemetry

Instructions here.

The details

Where is telemetry data sent

Butler SOS uses PostHog to collect telemetry data.

PostHog is an open source telemetry platform that is used by many open source projects. The data is stored in the EU.

Deleting telemetry data

Even though no-one (not even Ptarmigan Labs who runs the telemetry database!) has any way of ever connecting the data sent by your Butler SOS instance to you (it’s all anonymized, remember?), there can be cases where telemetry data must be deleted.

The legal page has more information about this.

Field level description of telemetry data

A telemetry message from Butler SOS contains the information below.

{
  "system": {
    "id": "62f221c94dc72823ebba9488a74006ccf69da8f8c6cfaa896a60d9a02186cc2e",
    "arch": "x64",
    "platform": "linux",
    "release": "22.04.4 LTS",
    "distro": "Ubuntu",
    "codename": "jammy",
    "virtual": true,
    "isRunningInDocker": false,
    "nodeVersion": "v20.15.0"
  },
  "enabledFeatures": {
    "feature": {
      "heartbeat": true,
      "dockerHealthCheck": true,
      "uptimeMonitor": true,
      "uptimeMonitorNewRelic": false,
      "udpServer": true,
      "eventCount": true,
      "rejectedEventCount": true,
      "userEvents": true,
      "userEventsMQTT": true,
      "userEventsInfluxdb": true,
      "userEventsNewRelic": false,
      "logEventsProxy": true,
      "logEventsScheduler": true,
      "logEventsRepository": true,
      "logEventCategorise": true,
      "logEventCategoriseRuleCount": 4,
      "logEventCategoriseRuleDefault": true,
      "logEventEnginePerformanceMonitor": true,
      "logEventEnginePerformanceMonitorNameLookup": true,
      "logEventEnginePerformanceMonitorTrackRejected": true,
      "logEventsMQTT": true,
      "logEventsInfluxdb": true,
      "logEventsNewRelic": false,
      "logdb": false,
      "mqtt": true,
      "newRelic": false,
      "prometheus": true,
      "influxdb": true,
      "influxdbVersion": 2,
      "appNames": true,
      "userSessions": true
    }
  }
}

The anonymous ID field

The id field deserves a bit more explanation.

It’s purpose is to uniquely identify the Butler SOS instance - nothing else.
If Butler SOS is stopped and started agagin the same ID will be used.
If reinstalled on a new server, or if the server’s network configuration changes, a new ID will be created.

Some sensitive information is used to create the ID, but as the ID is anonymized before sent as part of the telemetry data, no sensitive information leaves your servers.

The ID field is created as follows:

1. Combine the following information to a single string

   1. MAC address of default network interface
   2. IPv4 address of default network interface
   3. IP or FQDN of Sense server where repository service is running
   4. System unique ID as reported by the OS. Not all OSs support this though, which is why field 1-3 above are also needed to get a unique ID.

2. Run the created string through a one-way hashing/message digest function. Butler SOS uses Node.js’ own Crypto library to create a SHA-256 hash, using the default network interface’s MAC address as salt.
   Security is increased due to the fact that the salt never leaves the server where Butler is running.

   The bottom line is that it’s impossible to reverse the process and get the IP, host name etc used in step 1 above.
   Then again - this is cryptography, and there are no guarantees.
   But if you trust the certificates securing Sense itself, then the ID anonymization in Butler SOS should be ok too. Both are built on the same concepts of one-way cryptographic functions.

3. The result is a string that uniquely identifies the Butler SOS instance at hand, without giving away any sensitive data about the system where Butler is running.

See above for an example of what the id field looks like.
The id field is always shown during Butler startup.

Telemetry FAQ

1. What data is included in the telemetry messages?
   See above.
   The telemetry includes information about which Butler SOS features are enabled vs disabled. A unique, anonymized ID is included too, it’s unique to each Butler SOS instance and is used soley to distinguish between different Butler SOS instances.
   Finally some information about Butler SOS’s execution environment is included. Things like operating system, Node.js version used etc.

2. Can my Sense environment be identified via telemetry data?
   Short answer: No.
   Longer answer: No information about your Sense environment is sent as part of telemetry. No IP addresses or server names, no IDs of Sense apps/tasks/etc, no information about what actual data passed through Butler SOS, or any other data that can be linked to your Sense environment is included in the telemetry data.