Administrators Guide

Directed disovery involves three phases:

The import phase begins when Provisiond receives a request to import a requisition from a URL. The requisition is marshalled into Java objects for processing. An audit, based on the unique foreign ID of the foreign source, determines whether the node already exists; the imported object is then added, updated, or deleted from the inventory.

If a requisition node has an interface defined as the primary SNMP interface, then during the update and add operations the node is scanned for minimal SNMP attribute information.

The node scan phase discovers details about the node and interfaces that were not directly provisioned. All physical (SNMP) and logical (IP) interfaces are discovered and persisted based on any provisioning policies that may have been defined for the foreign source associated with the import requisition.

After interface discovery, Provisiond moves to service detection on each IP interface entity.

To change the policies for this group of nodes you should create a foreign source for the group. You can do so using the ReST API:

The XML file: customer-a.foreign-source.xml:

Here is an example curl command used to create the foreign source with the above foreign source specification above:

Now that you’ve created the foreign source, it needs to be deployed by Provisiond. Here an the example using the curl command to deploy the foreign source:

The implementation only supports a PUT request because it is an implied "Update" of the configuration since it requires an IP address and all IPs have a default configuration. This request is is passed to the SNMP configuration factory in OpenNMS Horizon for optimization of the configuration store snmp-config.xml. This example changes the community string for the IP address 10.1.1.1 to yRuSonoZ.

This example adds 2 nodes to the Provisioning Group, customer-a. Note that the foreign-source attribute typically has a 1 to 1 relationship to the name of the Provisioning Group requisition. There is a direct relationship between the foreign- source attribute in the requisition and the foreign source policy specification. Also, typically, the name of the provisioning group will also be the same. In the following example, the ReST API will automatically create a provisioning group based on the value foreign-source attribute specified in the XML requisition.

A provisioning group file called etc/imports/customer-a.xml will be found on the OpenNMS Horizon system following the successful completion of this curl command and will also be visible via the WebUI.

Using the Provisioning Groups Web-UI, three nodes are created given a single IP address. Navigate to the Admin Menu and click Provisioning Groups Menu from the list of Admin options and create the group Bronze.

Clicking the Add New Group button will create the group and will redisplay the page including this new group among the list of any group(s) that have already been created.

Clicking the Edit link will bring you to the screen where you can begin the process of defining node entities that will be imported into OpenNMS Horizon. Click the Add Node button will begin the node entity creation process fill in the node label and click the Save button.

At this point, the provisioning group contains the basic structure of a node entity but it is not complete until the interface(s) and interface service(s) have been defined. After having clicked the Save button, as we did above presents, in the Web-UI, the options Add Interface, Add Node Category, and Add Node Asset. Click the Add Interface link to add an interface entity to the node.

Enter the IP address for this interface entity, a description, and specify the Primary attribute as P (Primary), S (Secondary), N (Not collected), or C (Collected) and click the save button. Now the node entity has an interface for which services can be defined for which the Web-UI now presents the Add Service link. Add two services (ICMP, SNMP) via this link.

Now the node entity definition contains all the required elements necessary for importing this requisition into OpenNMS Horizon. At this point, all the interfaces that are required for the node should be added. For example, NAT interfaces should be specified there are services that they provide because they will not be discovered during the Scan Phase.

Two more node definitions will be added for the benefit of this example.

This set of nodes represents an import requisition for the Bronze provisioning group. As this requisition is being edited via the WebUI, changes are being persisted into the OpenNMS Horizon configuration directory '$OPENNMS_etc/imports/' pending as an XML file having the name bronze.xml.

Click the Done button to return to the Provisioning Groups list screen. The details of the “Bronze” group now indicates that there are 3 nodes in the requisition and that there are no nodes in the DB from this group (a.k.a. foreign source). Additionally, you can see that time the requisition was last modified and the time it last imported are given (the time stamps are stored as attributes inside the requisition and are not the file system time stamps). These details are indicative of how well the DB represents what is in the requisition.

In this example, you see that there are 3 nodes in the pending requisition and 0 in the DB. Click the Import button to submit the requisition to the provisioning system (what actually happens is that the Web-UI sends an event to the Provisioner telling it to begin the Import Phase for this group).

You should be able to immediately verify the importation of this provisioning group because the import happens very quickly. Provisiond has several threads ready for processing the import operations of the nodes defined in this requisition.

A few SNMP packets are sent and received to get the SNMP details of the node and the interfaces defined in the requisition. Upon receipt of these packets (or not) each node is inserted as a DB transaction.

Following the import of a node with thousands of interfaces, you will be able to refresh the Interface table browser on the Node page and see that interfaces and services are being discovered and added in the background. This is the discovery component of directed discovery.

To direct that another node be added from a foreign source (in this example the Bronze Provisioning Group) simply add a new node definition and re-import. It is important to remember that all the node definitions will be re-imported and the existing managed nodes will be updated, if necessary.

To direct changes to an existing node, simply add, change, or delete elements or attributes of the node definition and re- import. This is a great feature of having directed specific elements of a node in the requisition because that attributes will simply be changed. For example, to change the IP address of the Primary SNMP interface for the node, barbrady.opennms.org, just change the requisition and re-import.

Each element in the Web-UI has an associated Edit icon Click this icon to change the IP address for barbrady.opennms.org, click save, and then Click the Done button.

The Web-UI will return you to the Provisioning Groups screen where you will see that there are the time stamp showing that the requisition’s last modification is more recent that the last import time.

This provides an indication that the group must be re-imported for the changes made to the requisition to take effect. The IP Interface will be simply updated and all the required events (messages) will be sent to communicate this change within OpenNMS Horizon.

Barbrady has not been behaving, as one might expect, so it is time to remove him from the system. Edit the provisioning group, click the delete button next to the node barbrady.opennms.org, click the Done button.

Click the Import button for the Bronze group and the Barbrady node and its interfaces, services, and any other related data will be immediately deleted from the OpenNMS Horizon system. All the required Events (messages) will be sent by Provisiond to provide indication to the OpenNMS Horizon system that the node Barbrady has been deleted.

There is a convenient way to delete all the nodes that have been provided from a specific foreign source. From the main Admin/Provisioning Groups screen in the Web-UI, click the Delete Nodes button. This button deletes all the nodes defined in the Bronze requisition. It is very important to note that once this is done, it cannot be undone! Well it can’t be undone from the Web-UI and can only be undone if you’ve been good about keeping a backup copy of your '$OPENMS_ETC/' directory tree. If you’ve made a mistake, before you re-import the requisition, restore the Bronze.xml requisition from your backup copy to the '$OPENNMS_ETC/imports' directory.

Clicking the Import button will cause the Audit Phase of Provisiond to determine that all the nodes from the Bronze group (foreign source) should be deleted from the DB and will create Delete operations. At this point, if you are satisfied that the nodes have been deleted and that you will no longer require nodes to be defined in this Group, you will see that the Delete Nodes button has now changed to the Delete Group button. The Delete Group button is displayed when there are no nodes entities from that group (foreign source) in OpenNMS Horizon.

When no node entities from the group exist in OpenNMS Horizon, then the Delete Group button is displayed.

As IP interfaces are found during the node scan process, service detection tasks are scheduled for each IP interface. The service detections defined in the foreign source determines which services are to be detected and how (i.e. the values of the parameters that parameters control how the service is detected, port, timeout, etc.).

Several new XML entities have been added to the import requisition since the introduction of the OpenNMS Importer service in version 1.6. So, in addition to provisioning the basic node, interface, service, and node categories, you can now also provision asset data.

Because Provisiond takes a URL as the location service for import requisitions, OpenNMS Horizon can be easily extended to support sources in addition to the native URL handling provided by Java: file://, http://, and https://. When you configure Provisiond to import requisitions on a schedule you specify using a URL Resource. For requisitions created by the Provisioning Groups WebUI, you can specify a file based URL.

This command creates a node element in the requisition customer1 called node-a using the script’s node option. The node’s foreign-ID is 1 but it can be any alphanumeric value as long as it is unique within the requisition. Note the node has no interfaces or services yet.

This command adds an interface element to the node element using the interface option to the provision.pl command and it can now be seen in the pending requisition by running provision.pl requisition list customer1.

This adds the 2 services to the specified 127.0.0.1 interface and is now in the pending requisition.

This sets the 127.0.0.1 interface to be the node’s Primary SNMP interface.

This adds the two categories to the node and is now in the pending requisition.

These categories are case-sensitive but do not have to be already defined in OpenNMS Horizon. They will be created on the fly during the import if they do not already exist.

This will add value of 9999 to the asset field: serialnumber.

This will cause OpenNMS Horizon Provisiond to import the pending customer1 requisition. The formerly pending requisition will move into the deployed state inside OpenNMS Horizon.

Very much the same as the add, except that a single delete command and a re-import is required. What happens is that the audit phase is run by Provisiond and it will be determined that a node has been removed from the requisition and the node will be deleted from the DB and all services will stop activities related to it.

This completes the life cycle of managing a node element, iteratively, in a import requisition.

Please note: The Http Detector makes only one http request and doesn’t follow redirects.

The parameters are the same as for the HTTP detector

We have Dell server farm and want to monitor the global server status provided by the OpenManage Server Administrator. Global status is provided by a scalar OID .1.3.6.1.4.1.674.10892.1.200.10.1.2.1. The service should be automatically detected if the server supports this OID.

For provisioning we have a requisition named Server which contains all server of our data center. A Detector with the name Dell-OMSA-Global-State for this requisition is created with the following parameter:

When the requisition Server is synchronized the service Dell-OMSA-Global-State will be detected in case they support the given SNMP OID.

We have a HP server farm and want to monitor the status of logical drives over SNMP provided from HP Insight Manager. The status for logical drives is provided in a SNMP Table under .1.3.6.1.4.1.232.3.2.3.1.1.4. The service should be automatically assigned to all servers exposing the given SNMP OID.

For provisioning we have a requisition named Server which contains all server of our data center. A Detector with the name HP-Insight-Drive-Logical for this requisition is created with the following parameter:

When the requisition Server is synchronized the service HP-Insight-Drive-Logical will be detected in case they support the given SNMP OID table.

If a valid response to the Identify command is received, the product vendor and product version are stored in the vendor and modelNumber fields of the associated node`s assets table.

For example, a Windows Server 2008 machine returns:

If these assets field are being used for another purpose, this behavior can be disabled by settings the updateAssets parameters to false in the detector configuration of the appropriate foreign source.

Example detector configuration:

The response is logged as DEBUG information in provisiond.log and looks like the following:

Example detector configuration:

The response is logged as DEBUG information in provisiond.log and looks like the following:

Reverse DNS Lookup Detector doesn’t take any parameters.

Reverse-DNS-Lookup Detector can be used in auto discovery with detectors to discover IP Addresses that only resolve FQDN.

The Surveillance View has multiple functions.

You can select columns, rows, single cells and of course all entries in a Surveillance View. Please refer to the Surveillance View Section for details on how to configure Surveillance Views.

The Alarms component gives an overview about all unacknowledged Alarms with a severity higher than Normal(1). Acknowledged Alarms will be removed from the responsibility of the Operator. The following information are shown in:

The Alarms component shows the most recent Alarms and allows the user to scroll through the last 100 Alarms.

To inform people on a duty schedule notifications are used and force action to fix or reconfigure systems immediately. In OpenNMS Horizon it is possible to acknowledge notifications to see who is working on a specific issue. The Dashboard should show outstanding notifications in the NOC to provide an overview and give the possibility for intervention.

The Notifications component shows the most recent unacknowledged notifications and allows the user to scroll through the last 100 Notifications.

An acknowledged Alarm doesn’t mean necessarily the outage is solved. To give an overview information about ongoing Outages in the network, the Dashboard shows an outage list in the Node Status component.

To give a quick entry point diagnose performance issues a Resource Graph Viewer allows to navigate to time series data reports which are filtered in the context of the Surveillance View.

It allows to navigate sequentially through resource graphs provided by nodes filtered by the Surveillance View context and selection and shows one graph report at a time.

The following example illustrates how this Dashboard role can be used. For instance the user drv4doe is assigned the dashboard role. So, when logging in as drv4doe, the user is taking directly to the Dashboard page and is presented with a custom Dashboard based on the drv4doe Surveillance View definition.

You can modify the configuration files for the security framework to give you access to one or more dashboards without logging in. At the end you’ll be able to point a browser at a special URL like http://…​/opennms/dashboard1 or http://…​/opennms/dashboard2 and see a dashboard without any authentication. First, configure surveillance views and create dashboard users as above. For example, make two dashboards and two users called dashboard1 and dashboard2. Test that you can log in as each of the new users and see the correct dashboard. Now create some aliases you can use to distinguish between dashboards. In /opt/opennms/jetty-webapps/opennms/WEB-INF, edit web.xml. Just before the first <servlet-mapping> tag, add the following servlet entries:

Just before the first <error-page> tag, add the following servlet-mapping entries:

After the last <filter-mapping> tag, add the following filter-mapping entries:

Next edit applicationContext-acegi-security.xml to enable anonymous authentication for the /dashboard1 and /dashboard2 aliases. Near the top of the file, find <bean id="filterChainProxy" …​>. Below the entry for /rss.jsp*, add an entry for each of the dashboard aliases:

About halfway through the file, look for <bean id="filterInvocationInterceptor" …​>. Below the entry for /dashboard.jsp, add an entry for each of the aliases:

Finally, near the bottom of the page, add a new instance of AnonymousProcessingFilter for each alias.

Restart OpenNMS Horizon and you should bring up a dashboard at http://…​/opennms/dashboard1 without logging in.

This Alarm-Details Dashlet shows a table with alarms and some detailed information.

The Alarm Details Dashlet can be configured with the following parameters.

This Alarms Dashlet shows a table with a short alarm description.

The Alarms Dashlet can be configured with the following parameters.

This Dashlet displays an existing Chart.

This Dashlet shows a Grafana Dashboard for a given time range. The Grafana Dashboard Box configuration defined in the opennms.properties file is used to access the Grafana instance.

This Dashlet displays an image by a given URL.

This Dashlet shows an existing KSC report. The view is exact the same as the KSC report is build regarding order, columns and time spans.

This Dashlet displays the geographical map.

This Dashlet shows one or multiple RRD graphs. It is possible to arrange and order the RRD graphs in multiple columns and rows. All RRD graphs are normalized with a given width and height.

This Dashlet shows the configured SLA categories from the OpenNMS Horizon start page.

This Dashlet shows a trend of incoming alarms in given time frame.

This Dashlet shows a given Surveillance View.

This Dashlet shows a Topology Map. The Topology Map can be configured with the following parameter.

This Dashlet shows the content of a web page or other web application, e.g. other monitoring systems by a given URL.

At the beginning the connection to an MBean Server of a Java application has to be configured.

By clicking the arrow ( > ) the MBeans and Composite Data will be retrieved with the given connection settings. The data is loaded into the MBeans Configuration screen which allows to select metrics for the data collection configuration.

The MBeans Configuration section is used to assign the MBean and Composite Data attributes to RRD domain specific data types and data source names.

The left sidebar shows the tree with the JMX Domain, MBeans and Composite Data hierarchy retrieved from the MBean Server. To select or deselect all attributes use Mouse right click → select/deselect.

The right panel shows the MBean Attributes with the RRD specific mapping and allows to select or deselect specific MBean Attriubtes or Composite Data Attributes for the data collection configuration.

The MBean Name, Composite Alias and Name are validated against special characters. For the Alias inputs are validated to be not longer then 19 characters and have to be unique in the data collection configuration.

The last step is generating the following configuration files for OpenNMS Horizon:

The content of the configuration files can be copy & pasted or can be downloaded as ZIP archive.

It is required to have the Java 8 Development Kit with Apache Maven installed. The mvn binary has to be in the path environment. After cloning the repository you have to enter the source folder and compile an executable JAR.

Inside the newly created target folder a file named jmxconfiggenerator-<VERSION>-onejar.jar is present. This file can be invoked by:

After installing the the JMX Config Generator the tool’s wrapper script is located in the ${OPENNMS_HOME}/bin directory.

The JMX Config Generator uses sub-commands for the different configuration generation tasks. Each of these sub-commands provide different options and parameters. The command line tool accepts the following sub-commands.

The following global options are available in each of the sub-commands of the tool:

This sub-command is used to query a MBean Server for it’s available MBean objects. The following example queries the server myserver with the credentials myusername/mypassword on port 7199 for MBean objects in the java.lang domain.

The following command line options are available for the query sub-command.

This sub-command can be used to generate a valid jmx-datacollection-config.xml for a given set of MBean objects queried from a MBean Server.

The following example generate a configuration file myconfig.xml for MBean objects in the java.lang domain of the server myserver on port 7199 with the credentials myusername/mypassword. You have to define either an URL or a hostname and port to connect to a JMX server.

The following options are available for the generate-conf sub-command.

This sub-command generates a RRD graph definition file for a given configuration file. The following example generates a graph definition file mygraph.properties using the configuration in file myconfig.xml.

The following options are available for this sub-command.

The JMX Config Generator uses a template file to generate the graphs. It is possible to use a user-defined template. The option --template followed by a file lets the JMX Config Generator use the external template file as base for the graph generation. The following example illustrates how a custom template mytemplate.vm is used to generate the graph definition file mygraph.properties using the configuration in file myconfig.xml.

The template file has to be an Apache Velocity template. The following sample represents the template that is used by default:

The JMX Config Generator generates different types of graphs from the jmx-datacollection-config.xml. The different types are listed below: