Cortex Time Series Plugin

The Cortex Time Series plugin enables Horizon to persist time series performance and latency data to Cortex. The plugin uses the time series storage integration layer to convert metrics to a Prometheus model and delegate writes and reads to Cortex. The plugin also lets users visualize graphs from the Cortex instance in OpenNMS Helm through the OpenNMS performance data source or the Prometheus data source.

Cortex plugin architecture
Figure 1. Cortex Time Series plugin

Requirements

Make sure you have the following before you start to use the Cortex plugin:

  • Horizon 30.0.0+ (Meridian availability with the 2023 release)

  • Cortex version 1.10.0+

We plan to keep the plugin compatible with Mimir, Grafana’s fork of Cortex, as long as Mimir remains compatible with Cortex. The minimum supported Mimir version is 2.0.0.

Deployment

To use the Cortex Time Series plugin, do the following:

  1. Start Cortex. See the Cortex documentation for instructions. If your aim is to get up to speed quickly in a test environment, you can use Docker Compose to deploy a minimal, standalone Cortex node.

  2. Download the plugin’s .kar file from GitHub (https://github.com/OpenNMS/opennms-cortex-tss-plugin/releases).

  3. Copy the file into ${OPENNMS_HOME}/deploy.

  4. Enable and configure time series storage plugins in Horizon.

    Create or modify the ${OPENNMS_HOME}/etc/opennms.properties.d/cortex.properties file
    org.opennms.timeseries.strategy=integration
org.opennms.timeseries.tin.metatags.tag.node=${node:label}
org.opennms.timeseries.tin.metatags.tag.location=${node:location}
org.opennms.timeseries.tin.metatags.tag.geohash=${node:geohash}
org.opennms.timeseries.tin.metatags.tag.ifDescr=${interface:if-description}
org.opennms.timeseries.tin.metatags.tag.label=${resource:label}
    Make sure that the org.opennms.timeseries.strategy property is not set in ${OPENNMS_HOME}/etc/opennms.properties or any other file within the ${OPENNMS_HOME}/etc/opennms.properties.d/ directory.

  5. Configure the Cortex Time Series plugin in Horizon.

    The writeUrl and readUrl properties are required. All other properties shown here are example values to overwrite the defaults. Keep an eye on your Cortex cluster performance and adjust as necessary. Substitute the correct host and port for your Cortex cluster.

    You can configure the plugin via the Karaf shell or by creating a cfg file.

    • Karaf Shell

    • CFG File

    config:edit org.opennms.plugins.tss.cortex
property-set writeUrl http://cortex-service-host:9009/api/prom/push
property-set readUrl http://cortex-service-host:9009/prometheus/api/v1
property-set maxConcurrentHttpConnections 100
property-set writeTimeoutInMs 1000
property-set readTimeoutInMs 1000
property-set metricCacheSize 1000
property-set externalTagsCacheSize 1000
property-set bulkheadMaxWaitDurationInMs 9223372036854775807
config:update
    Create or modify the ${OPENNMS_HOME}/etc/org.opennms.plugins.tss.cortex.cfg file
    writeUrl=http://cortex-service-host:9009/api/prom/push
readUrl=http://cortex-service-host:9009/prometheus/api/v1
maxConcurrentHttpConnections=100
writeTimeoutInMs=1000
readTimeoutInMs=1000
metricCacheSize=1000
externalTagsCacheSize=1000
bulkheadMaxWaitDurationInMs=9223372036854775807

  6. Configure the plugin to remain installed after a restart of Horizon.

    Create or modify the ${OPENNMS_HOME}/etc/featuresBoot.d/cortex.boot file
    opennms-plugins-cortex-tss wait-for-kar=opennms-cortex-plugin

  7. Restart the Horizon service.

Cortex tips

You can view information on your Cortex server with the following URLs:

  • View server status

    • http://cortex-service-host:9009

  • View the ring

    • http://cortex-service-host:9009/ring

  • View internal metrics

    • http://cortex-service-host:9009/metrics