Setting up Jaeger Tracing

OpenTracing enables distributed tracing which makes it possible to monitor RPCs or transactions across process boundaries. You can leverage this functionality to help track and diagnose communication problems that may arise in distributed deployments of Horizon components; for example, Core instance, Minion, and/or Sentinel. OpenTracing is supported on RPC calls between Horizon Core instance for Sink API calls (including flows) between Minion and Sentinel. By default the tracing instrumentation Horizon is disabled (a no-op tracer is used.)

The project Jaeger is an open source, end-to-end distributed tracing software. It lets you monitor and troubleshoot transactions in distributed software components when Horizon is deployed with Minion and Sentinel. Horizon components support Jaeger which implements the OpenTracing API.

  • Core

  • Minion

  • Sentinel

Enable tracing with Jaeger on Core instance by installing the opennms-core-tracing-jaeger feature and creating a configuration file with the Jaeger specific settings.

Configurations has to be made in the etc directory relative to the Horizon Core home directory. Depending on your operating system the home directory is in /usr/share/opennms for Debian/Ubuntu or /opt/opennms for CentOS/RHEL.
Create a file to load the tracing feature on startup
sudo vi etc/featuresBoot.d/jaeger.boot
Add the Jaeger feature and save the file
opennms-core-tracing-jaeger
Uninstalling the opennms-core-tracing-jaeger feature at runtime is not yet supported. To disable tracing with Jaeger, remove the reference from the features boot file and restart the Core.
Create a configuration file for tracing settings
sudo vi etc/opennms.properties.d/jaeger.properties
Configure tracing with Jaeger
org.opennms.core.tracer=jaeger(1)
JAEGER_AGENT_HOST=my-jaeger-host(2)
1 Set Jaeger as the tracing implementation
2 Replace the my-jaeger-host with your Jaeger instance where you collect the tracing information
Available configuration options are listed in the Jaeger Core Readme.
The Jaeger service name is the Instance-ID of the Core instance.
Restart Horizon Core instance to apply the changes
sudo systemctl restart opennms

Enable tracing with Jaeger on Minion by installing the opennms-core-tracing-jaeger feature.

Configurations has to be made in the etc directory relative to the Minion home directory. Depending on your operating system the home directory is in /usr/share/minion for Debian/Ubuntu or /opt/minion for CentOS/RHEL.
Create a file to load the tracing feature on startup
sudo vi etc/featuresBoot.d/jaeger.boot
Add the Jaeger feature and save the file
opennms-core-tracing-jaeger
Uninstalling the opennms-core-tracing-jaeger feature at runtime is not yet supported. To disable tracing with Jaeger, remove the reference from the features boot file and restart Minion.
Create a configuration file for tracing settings
sudo vi etc/custom.system.properties
Add configuration properties for tracing with Jaeger
JAEGER_AGENT_HOST=my-jaeger-host(1)
1 Replace the my-jaeger-host with your Jaeger instance where you collect the tracing information
Available configuration options are listed in the Jaeger Core Readme.
The Jaeger service name is the Location-Name@Instance-ID from the Minion.
Restart Minion to apply the changes
sudo systemctl restart minion

Enable tracing with Jaeger on Sentinel by installing the opennms-core-tracing-jaeger feature.

Configurations has to be made in the etc directory relative to the Sentinel home directory. Depending on your operating system the home directory is in /usr/share/sentinel for Debian/Ubuntu or /opt/sentinel for CentOS/RHEL.
Create a file to load the tracing feature on startup
sudo vi etc/featuresBoot.d/jaeger.boot
Add the Jaeger feature and save the file
opennms-core-tracing-jaeger
Uninstalling the opennms-core-tracing-jaeger feature at runtime is not yet supported. To disable tracing with Jaeger, remove the reference from the features boot file and restart Sentinel.
Create a configuration file for tracing settings
sudo vi etc/custom.system.properties
Add configuration properties for tracing with Jaeger
JAEGER_AGENT_HOST=my-jaeger-host(1)
1 Replace the my-jaeger-host with your Jaeger instance where you collect the tracing information
Available configuration options are listed in the Jaeger Core Readme.
The Jaeger service name is the Instance-ID from Sentinel.
Restart Sentinel to apply the changes
sudo systemctl restart sentinel