Import Handlers

The new Provisioning service in Horizon is continuously improving and adapting to the needs of the community.

One of the most recent enhancements to the system is built upon the very flexible and extensible API of referencing an import requisition’s location via a URL. Most commonly, these URLs are files on the file system (i.e. file:/opt/opennms/etc/imports/<my-provisioning-group.xml>) as requisitions created by the Provisioning Groups UI. However, these same requisitions for adding, updating, and deleting nodes (based on the original model importer) can also come from URLs. For example a requisition can be retrieved using the HTTP protocol.

In addition to the standard protocols supported by Java, we provide a series of custom URL handlers to help retrieve requisitions from external sources.

Generic Handler

The generic handler is made available using URLs of the form: requisition://type?param=1;param=2

Using these URLs various type handlers can be invoked, both locally and via a Minion.

In addition to the type specific parameters, the following parameters are supported:

Table 1. General parameters
Parameter	Description	Required	Default value
`location`	The name of location at which the handler should be run	optional	Default
`ttl`	The maximum number of milliseconds to wait for the handler when ran remotely	optional	20000

See the relevant sections below for additional details on the support types.

The opennms:show-import command available via the Karaf Shell can be used to show the results of an import (without persisting or triggering the import):

opennms:show-import -l MINION http url=http://127.0.0.1:8000/req.xml

File Handler

Examples:

Simple

file:///path/to/my/requisition.xml

Using the generic handler

requisition://file?path=/path/to/my/requisition.xml;location=MINION

HTTP Handler

Examples:

Simple

http://myinventory.server.org/nodes.cgi

Using the generic handler

requisition://http?url=http%3A%2F%2Fmyinventory.server.org%2Fnodes.cgi

When using the generic handler, the URL should be "URL encoded".

DNS Handler

The DNS handler requests a Zone Transfer (AXFR) request from a DNS server. The A records are recorded and used to build an import requisition. This is handy for organizations that use DNS (possibly coupled with an IP management tool) as the data base of record for nodes in the network. So, rather than ping sweeping the network or entering the nodes manually into Horizon Provisioning UI, nodes can be managed via 1 or more DNS servers.

The format of the URL for this new protocol handler is: dns://<host>[:port]/<zone>[/<foreign-source>/][?expression=<regex>]

DNS Import Examples:

Simple

dns://my-dns-server/myzone.com

This URL will import all A records from the host my-dns-server on port 53 (default port) from zone "myzone.com" and since the foreign source (a.k.a. the provisioning group) is not specified it will default to the specified zone.

Using a Regular Expression Filter

dns://my-dns-server/myzone.com/portland/?expression=^por-.*

This URL will import all nodes from the same server and zone but will only manage the nodes in the zone matching the regular expression ^port-.* and will and they will be assigned a unique foreign source (provisioning group) for managing these nodes as a subset of nodes from within the specified zone.

If your expression requires URL encoding (for example you need to use a ? in the expression) it must be properly encoded.

dns://my-dns-server/myzone.com/portland/?expression=^por[0-9]%3F

Location

You can use the location parameter to assign a single location to all nodes in this requisition:

dns://my-dns-server/myzone.com/portland/?location=Raleigh

By prefixing with a ~, you can specify a url-encoded regular expression to determine the location based on the host entries. The following example derives the location based on the subdomain. It uses a regular expression ^(?:.\.|)(.?)\.hs-fulda\.de\.$ with exactly one capturing group containing the subdomain:

dns://my-dns-server/hs-fulda.de/?location=~%5E%28%3F%3A.%2A%5C.%7C%29%28.%2A%3F%29%5C.hs-fulda%5C.de%5C.%24

This will associate nodes for the host entries foo.e46.hs-fulda.de. and bar.e46.hs-fulda.de. to the location e46 while hosts like aaa.g51.hs-fulda.de and bbb.g51.hs-fulda.de will be assigned location g51.

You can use online tools like https://www.urlencoder.org to encode your parameters.

DNS Setup

Currently, the DNS server requires to be setup to allow a zone transfer from the Horizon server. It is recommended that a secondary DNS server is running on Horizon and that the Horizon server be allowed to request a zone transfer. A quick way to test if zone transfers are working is:

dig -t AXFR @<dnsServer> <zone>

Configuration

The configuration of the Provisoning system has moved from a properties file (model-importer.properties) to an XML based configuration container. The configuration is now extensible to allow the definition of 0 or more import requisitions each with their own cron based schedule for automatic importing from various sources (intended for integration with external URL such as http and this new dns protocol handler.

A default configuration is provided in the Horizon etc/ directory and is called: provisiond-configuration.xml. This default configuration has an example for scheduling an import from a DNS server running on the localhost requesting nodes from the zone, localhost and will be imported once per day at the stroke of midnight. Not very practical but is a good example.

<?xml version="1.0" encoding="UTF-8"?>
    <provisiond-configuration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://xmlns.opennms.org/xsd/config/provisiond-configuration"
                              foreign-source-dir="/opt/opennms/etc/foreign-sources"
                              requistion-dir="/opt/opennms/etc/imports"
                              importThreads="8"
                              scanThreads="10"
                              rescanThreads="10"
                              writeThreads="8" >

    <!--http://www.quartz-scheduler.org/documentation/quartz-1.x/tutorials/crontrigger
        Field Name Allowed Values Allowed Special Characters
        Seconds 0-59 , - * / Minutes 0-59 , - * / Hours 0-23 , - * /
        Day-of-month1-31, - * ? / L W C Month1-12 or JAN-DEC, - * /
        Day-of-Week1-7 or SUN-SAT, - * ? / L C # Year (Opt)empty, 1970-2099, - * /
    -->

    <requisition-def import-name="localhost"
                     import-url-resource="dns://localhost/localhost">

        <cron-schedule>0 0 0 * * ? *</cron-schedule> <!-- daily, at midnight -->
    </requisition-def>
</provisiond-configuration>

Configuration Reload

Like many of the daemon configuration in the 1.7 branch, the configurations are reloadable without having to restart Horizon, using the reloadDaemonConfig uei:

/opt/opennms/bin/send-event.pl
uei.opennms.org/internal/reloadDaemonConfig --parm 'daemonName Provisiond'

This means that you don’t have to restart Horizon every time you update the configuration.