Telemetry

Dataristix Core version 2.0.75 or later integrates OpenTelemetry, optionally collecting traces, logs, and metrics to:

  • Analyze performance bottlenecks in dataflow tasks.
  • Monitor task metrics such as input, output or error counts.
  • Aggregate logs from Core and Connector modules.
  • Send your own metrics: Use the integrated Telemetry output connector to send results of your dataflow tasks to your preferred telemetry collector.

Telemetry is disabled by default. Go to the “Configuration” navigation node and select the “Telemetry” tab to configure telemetry.

Important

The Telemetry tab is only available to users in the Administrator or Architect role.

../_images/telemetry-settings.png

There are separate settings for built-in telemetry: (1) “General Telemetry Settings”, for user-defined telemetry as configured in tasks; (2) “Task Output Telemetry Settings”, and (3) “Audit Log Telemetry Settings”, for audit log entries forwarded from Core and Connector modules. You can send telemetry for each to the same or to separate endpoints.

Settings include:

  • Endpoint URL
    The endpoint of your OpenTelemetry collector.
  • Protocol
    The protocol to use for communications with your OpenTelemetry collector.
  • Authentication
    If your collector requires authentication, then select the appropriate option here. Supported authentication methods are “Basic” username and password authentication and authentication via API key with configurable API key header name.
  • Service instance name suffix
    When running multiple instances of Dataristix sending telemetry data to the same collector, then optionally enter an instance name suffix here to distinguish different instances in your visualization tool; for example, enter “@hostname” to distinguish a specific host.
  • Metrics export interval in milliseconds
    How often collected metrics are sent to your collector. Defaults to 30000 (30 seconds).

In “General Telemetry Settings”, you can separately enable or disable traces, metrics and logs. Task Output Telemetry consists of metrics only, which you can enable or disable globally here, preventing any task configured with the Telemetry output connector from sending telemetry. Audit Log Telemetry consists of logs only.

Use the “Apply” toolbar button after making changes to the settings. The effect is immediate and does not require restarting of tasks.