Kafka Producer

The Kafka Producer listens for all events on the event bus and forwards these to a Kafka topic. The records are keyed by event ID and contain a GPB-encoded model of the event.

By default, all events are forwarded to a topic named events.

You can configure the name of the topic, and set up an optional filtering expression to help control which events are sent to the topic.

The Kafka Producer listens for changes made to the current set of alarms and forwards the resulting alarms to a Kafka topic. The records are keyed by alarm reduction key and contain a GPB-encoded model of the alarm. When an alarm is deleted, a null value is sent with the corresponding reduction key. Publishing records in this fashion lets the topic be used as a KTable. The Kafka Producer will also perform periodic synchronization tasks to ensure that the contents of the Kafka topic reflect the current state of alarms in the Horizon database.

By default, all alarms (and subsequent updates) are forwarded to a topic named alarms.

You can configure the name of the topic, and set up an optional filtering expression to help control which alarms are sent to the topic.

If an event or alarm being forwarded references a node, then the corresponding node is also forwarded. The records are keyed by "node criteria" (see below) and contain a GPB-encoded model of the alarm. A caching mechanism is in place to help avoid forwarding nodes that have been successfully forwarded, and have not changed since.

The name of the topic used can be configured.

The Kafka Producer listens for changes made to the current set of topologies and forwards the resulting messages to the topologyEdgeTopic Kafka topic. The topologies are provided by the Enhanced Linkd updaters via the OnmsTopology API. An updater sends OnmsTopologyMessage to the subscribers. The records are keyed by GPB-encoded key of the protocol and TopologyRef and contain a GPB-encoded model of the vertex or edge. When a vertex or an edge is deleted, a null value is sent with the corresponding encoded GPB key. Publishing records in this fashion lets the topic be used as a KTable.

You can use the Kafka Producer to write metrics to Kafka either exclusively, or in addition to an existing persistence strategy i.e., RRD or Newts. The metrics are written in the form of "collection sets" that correspond to the internal representation the existing collectors and persistence strategies use. The records are keyed by node ID or by IP address if no node ID is available and contain a GPB-encoded version of the collection sets. The records are keyed in this fashion to help ensure that collection sets related to the same resources are written to the same partitions.

When enabled (this functionality is disabled by default), the metrics are written to a topic named metrics.