Specifying TARGET lines

The actual data-reading part of Weathermap is handled by Data Source Plugins since version 0.9. These allow users or third-parties to easily add new data sources into Weathermap without altering the core code.

There are a number of plugins supplied as standard, which will be described here. Each one uses a different format of TARGET string, which contains the parameters the plugin needs to find your data. All plugins return an 'in' and 'out' value, and some may set other variables that can be accessed by Special Tokens in strings. For situations where there is only really one output, the 'out' value may just be the same as the 'in'.

Here are the details of the standard data source plugins:


Static value

TARGET static:invalue:outvalue
TARGET static:value

The simplest of all the plugins, static allows you to just set a value in the config file itself. Sometimes this is useful for testing, or simple maps of things like OSPF metrics. If only a single value is specified, it is used for both input and output, just like the BANDWIDTH parameter.

The values in the target string can use the same K,M,G,T suffixes as BANDWIDTH.


RRDtool files

TARGET rrdfile.rrd:in_ds:out_ds
TARGET gauge:rrdfile.rrd:in_ds:out_ds
TARGET rrd:rrdfile.rrd:in_ds:out_ds
TARGET scale:factor:rrdfile.rrd:in_ds:out_ds

This is the 'core' plugin for Weathermap. It reads data from rrd files, created using Tobi Oetiker's RRDtool, by tools like Cacti, MRTG, Cricket, NRG and so on.

First of all, Weathermap needs to be able to find your rrdtool executable. If you are using the Cacti plugin, then this information is taken automatically from Cacti. If you are using the command-line tool, you will need to edit weathermap around line 29 to point to your rrdtool executable.

TARGET formats

If you only specify a filename, and no DS-names, the default DS names for RRD files are 'traffic_in' and 'traffic_out', which works with the majority of Cacti RRD files (those produced by the Interface Traffic template). For MRTG-produced RRD files, the names are 'DS0' and 'DS1'. For Cricket standard-interface RRD files, it would be 'ds0' and 'ds1'.

You can also specify '-' for either DS name, which tells Weathermap to ignore this rrd file for the purposes of the input or output value. This is mainly useful in combination with the aggregation feature, where you can take the input data from one rrd file, and the output data from another.

With no prefix, or with just 'rrd:' as a prefix, the data read from the rrd file is assumed to be a standard SNMP interface counter, which is a byte-rate. It automatically multiplies these by 8 to get a bit-rate for the map. With the 'gauge:' prefix, this multiplication doesn't happen, which is useful for non-bandwidth values, like SNR or session-counts. Finally, you can use the 'scale:' prefix to multiply by any factor - suppose you have a value in seconds, and you want to show it in milliseconds for example. Of course, you can also divide using this, by using a scale factor that is less than 1 - multiplying by 1/x is the same as dividing by x. Since 0.95, you can also use negative scale factors - useful to turn dB SNR into something weathermap can deal with.

rrdtool adjustments

By default, the plugin will read the last 800 seconds of data, and find the most recent within that to use. You might need to make it read back further, if you are updating your rrd files slowly. You can do this with the SET command, by adding 'SET rrd_period 3000' (any value in seconds) at the top of your map config file, before any NODE or LINK lines.

Similarly, you can change the time that the plugin looks for data at from the present to the past, by using 'SET rrd_start -1d' in the top section of the config file. The full range of time-specification strings is at the bottom of the rrdfetch manual page.

If you are using a tool other than Cacti, then the default DS names of traffic_in and traffic_out are probably wrong. You can change the defaults for the whole config file by adding two SET lines to the top of the map config file. For example, 'SET rrd_default_in_ds DS0' and 'SET rrd_default_in_ds DS1' will make the default DS names DS0 and DS1 respecively so MRTG users don't need to specify the DS names in every TARGET line.

For any other special rrdtool 'tweaks', you can SET rrd_options to a string which is added directly to the rrdtool command-line. This should allow for any changes that I haven't expected!

Finally, if the TARGET filename doesn't begin with a / it is usually assumed to be a relative path from the weathermap directory. If you SET rrd_default_path to some other path in the global (top) section of the map config file, you can change that assumption so that the DS plugin looks somewhere else.

Cacti-specific additions

If the plugin detects that it is running within the Cacti poller process, it defines a couple of useful global Hint Variables for Cacti users - {map:cacti_path_rra} contains the path to the rra/ directory in your Cacti installation, with no trailing slash, and {map:cacti_url} is the URL to the front page of your Cacti installation, as known to the Plugin Architecture. These can be used in the TARGET, INFOURL and OVERLIBGRAPH commands in particular, to reduce a lot of duplication.

Cacti poller_output support (aka Boost support)

If you are using Cacti as your data collector, and running Weathermap from Cacti's poller, then there is another special feature, called poller_output. Weathermap can intercept data as it is collected by Cacti, and therefore avoid running lots of external rrdtool processes to collect the data it needs to display your map. This is especially useful if you are using TheWitness' Boost plugin, as the data isn't written to the rrd files until some time after it is collected. Even without Boost, it should provide a performance improvement, particularly with large installations of Weathermap.

To use this feature, a map-global variable needs to be set. Near the top of your map config file, add 'SET rrd_use_poller_output 1'. The setting will take a few poller cycles to take effect: during the first one, the appropriate cacti data sources are identified based on the rrd filenames, on the second cycle values will be collected, and on the third we'll have two values, so any COUNTER datasources will begin to work properly. If the poller_output feature fails to collect data, then the regular rrdtool-running method will be attempted too.

Because in poller_output mode, Weathermap is pulling data directly from the Cacti database, it can also collect some other information that you might find useful in your map. It takes all the data stored in the host_snmp_cache table for the specified datasource, and dumps that into Hint Variables. This table contains all the columns that you see when you are picking which datasource to graph in Cacti, so for normal interface traffic, it contains things like the interface speed, operational status, interface description (your 'comment' name, usually) and so on. These are stored in Hint Variables called 'cacti_cache variable' - e.g. cacti_ifSpeed, or cacti_ifIP. It also sets cacti_host_id to be the internal host_id from Cacti for the host containing this data source, and the cacti_graph_id to be the local_graph_id of a graph that uses this DS in Cacti. You can then use those to automatically add correct OVERLIBGRAPH and INFOURL lines:

 
 SET rrd_use_poller_output 1 

 LINK DEFAULT 
 SET cacti_use_ifspeed 1 
 INFOURL {map:cacti_url}/graph.php?local_graph_id={link:this:cacti_graph_id} 
 OVERLIBGRAPH {map:cacti_url}/graph_image.php?local_graph_id={link:this:cacti_graph_id}&rra_id=0&graph_nolegend=true 

 LINK uplink1 
 NODES a b 
 TARGET {map:cacti_path_rra}/router1_345_traffic_in.rrd 

 LINK uplink2 
 NODES a c 
 TARGET {map:cacti_path_rra}/router1_377_traffic_in.rrd 

In this example, all each link needs to have defined is the TARGET rrdfile, and which nodes it joins. The rest is supplied by this plugin.

As an additional sub-feature, if you add 'SET cacti_use_ifspeed 1' to either the link or node, or any item it inherits settings from (e.g. DEFAULT) then Weathermap will take the value of ifSpeed (or ifHighSpeed for 4G+ links) and use it in place of the value defined in the map config file - this allows you to do some basic dynamic configuration of Weathermap based on Cacti, which in turn should pick up changes from the network device.

Note that since the rrdtool program is not being run in this mode, you can't use the "time-shifting" features described above - those rely on using real rrdtool files. If you are using poller_output to allow Weathermap to work alongside Boost, then there is no workaround - the data isn't in the .rrd files until Boost's own update process puts it there sometime later.

Performance Note:The additional queries to fill in the interface information from the Cacti database can sometimes add significantly to the runtime for the map. If you know you don't need the information, you can disable those queries, while still using the poller_output support. To do this, add 'SET rrd_no_cacti_extras 1' to the top of the map config file.

RRDtool Aggregation

XXX THIS NEEDS TO BE WRITTEN!


Tab-separated text

TARGET textfile

For tab-delimited data files, the format is plain-text, with three tab-seperated columns. The first one is a linkname, and the second and third are traffic-in and traffic-out, respectively. The linkname should match the name in the configuration file. This allows you to create one text file for the entire map from some outside source. Traffic in & out values can use the same "K,M,G,T" abbreviated forms as the BANDWIDTH configuration command. The file should have an extension of .txt or .tsv to be recognised as a tab-delimited file by Weathermap.

A suitable tab-delimited data file
link1 3M 4M 
link2 66K 1.8M 
link3 34.6K 113

MRTG .html file

TARGET htmlfile

This plugin reads data from special comments in the HTML files generated by MRTG. This is intended mainly for people using the 'old-style' MRTG .log files. If you are using MRTG with an RRDtool backend, then it's probably better to use the RRDtool plugin.

The file should have an extension of .html or .htm to be recognised as an MRTG file by Weathermap.

Since 0.95, there are a few hint variables that will adjust the behaviour of this datasaource:


Cacti host status

TARGET cactihost:hostid

This plugin reads the current status of a host from your Cacti database. The hostid is visible in Cacti URLs when you click on links in the Devices page. The return values for this plugin are numeric codes. It also sets a Hint Variable called state to a string, that can be nicer to use in an ICON filename, for example - possible values are 'up','down','disabled', and 'recovering'.

Codestate value
0disabled
1down
2recovering
3up
5unknown

It also sets the following additional Hint Variables, to use in a LABEL, COMMENT or NOTES section as you see fit: cacti_hostname, cacti_description, cacti_curtime, cacti_avgtime, cacti_mintime, cacti_maxtime, cacti_availability, cacti_faildate, and cacti_recdate

.

An appropriate SCALE definition to get red, green, yellow and grey labels based on the state of a host would be:

SCALE cactiupdown 0 0.5 192 192 192 
SCALE cactiupdown 0.5 1.5 255 0 0 
SCALE cactiupdown 1.5 2.5 0 0 255 
SCALE cactiupdown 2.5 3.5 0 255 0 
SCALE cactiupdown 4.5 5.5 128 128 128 

Cacti threshold status

TARGET cactimonitor:hostid
TARGET cactithold:tholdid
TARGET cactithold:rra_id:data_id

When used with cactimonitor: prefix, this plugin takes data from both Cacti's host table and from the THold plugin's data to produce a composite state, similar to the one used by the Monitor plugin.

It sets all the same Hint Variables as cactihost: plus a couple more: thold_failcount is the number of thresholds failing on this host, and thold_failpercent is the percentage of thresholds set on this host that are failing. Bear in mind that if you have both a minimum and maximum set on the same variable, then you will never get 100% failure.

It also has a new value possible for the state Hint Variable - 'tholdbreached', and adds a new state number 4, for "threshold breached".

In the second and third forms, with the cactithold: prefix, the plugin simply returns either a 0 or 1 for the 'input bandwidth' value. 0 if the threshold has not been breached, 1 if it has. You can supply either the internal THold id number for a threshold (not easily visible), or the rra_id and data_id values which can be seen at the end of URLs within THold's web pages.

You can provide a similar output to the Monitor plugin by using the cactimonitor: TARGET and multiple icons. Create a set of coloured (or otherwise different) icons for your node - they should have 'up','down','disabled','recovering' and 'tholdbreached' in the names. Then:

NODE myserver 
LABEL Server 1 
POSITION 100 100 
TARGET cactimonitor:77 
ICON images/server_{node:this:state}.png 
LABELOFFSET S 
USESCALE none 

Will allow you to change the icon between images/server_up.png, images/server_down.png etc.. as the state in Cacti changes and thresholds are breached. The 'USESCALE none' on the end stops the label for the node from changing colour as well.

You can make use of the TARGET aggregation features of weathermap to make a value that is effectively a 'percentage badness' for a set of thresholds. Suppose you have a set of load-balanced firewalls, and you want to see how many of the set have exceeded their session count. You would set up thresholds on each firewall as normal, and then:

SCALE badness 0 100 0 255 0 255 0 0 

NODE firewall_status 
 TARGET cactithold:334:22 cactithold:34:255 cactithold:337:235 cactithold:33:254 
 MAXVALUE 4 
 USESCALE badness in 

defines a scale where 0 is green and 100% is red and in between is a smooth gradient. Each target will return either 0 or 1, so 100% failed thresholds gives a value of 4. Setting the MAXVALUE to 4 makes that 100% within weathermap.


Cacti DSStats data collection plugin

TARGET dsstats:dsid:in_ds:out_ds
TARGET dsstats:source:dsid:in_ds:out_ds

The DSStats Cacti plugin stores data as the poller collects it, and also collates periodic average and maximums. Which are all stored in tables in the Cacti database. This makes collecting data (especially the periodic data) much more efficient. This DS plugin will read that data, saving a lot of external calls to rrdtool. This is similar to the 'poller_output' mode of the rrdtool DS, except that this way will allow you to easily make monthly or weekly summary maps. On the downside you need to supply the Cacti internal datasource ID for each target, rather than the rrd filename.

The dsid parameter is the 'local_data_id' in Cacti - you can see this in the URLs on the datasource management page. The 'in_ds' and 'out_ds' are the names of the RRD DS that Cacti will update with these values (e.g. traffic_in and traffic_out for normal Interface Traffic). Either of the in_ds or out_ds can be a single hyphen (-) to indicate that you don't care about filling in a value for that direction.

The source parameter defines what data to use for this query. 'last' fetches the most-recently collected value for that dsid. The remaining possibilities all combine a period and a type. The possible periods are 'hourly','daily','weekly','monthly' and 'yearly'. The types are 'average' and 'peak'. So to get the maximium value over the last week, use 'weeklypeak' as your source. If one is not specified at all (as in the first format above), then 'last' is assumed. You can change that default by setting the global SET variable "dsstats_default_type" to the required default.

The DSStats datasource also sets the same additional variables described in for poller_output support in the rrdtool plugin.

NOTE: One important difference between this and poller_ouput mode or regular rrdtool targets is that this plugin fetches the raw data from the poller. If you collecting octet counter data (like interface traffic) then you will need to multiply that data by 8 to get bits per second:

TARGET 8*dsstats:4455:traffic_in:traffic_out

Performance Note:As with the rrdtool poller_output support, the additional queries to fill in the interface information from the Cacti database can sometimes add significantly to the runtime for the map. If you know you don't need the information, you can disable those queries, while still using DSStats support. To do this, add 'SET dsstats_no_cacti_extras 1' to the top of the map config file.


SNMP value

TARGET snmp:community:host:in_oid:out_oid

This is a fairly experimental plugin. It requires the PHP snmp extension to be installed and enabled. Even then, it's hit & miss whether it will work on a particular system - some versions of PHP have better support for SNMP than others, and on some platforms it is different to others, too.

This plugin can directly query an SNMP-manageable device and fetch an OID from it. You can specify '-' in place of either of the OIDs if you only need to fetch one value.

It returns the numeric values for each OID as the in and out values. It does not currently understand COUNTER types, so you will just get the total bytes rather than a rate, for example.

It also sets snmp_in_raw and snmp_out_raw to the 'raw' results from the SNMP library, before the plugin tried to force them to be numbers.

You can adjust the behaviour of the SNMP plugin with some global SET values. 'snmp_timeout' sets the timeout for a request in microseconds (so the default 1000000=1 second). 'snmp_retries' changes the number of retries per request from the default (2). Finally, you can set 'snmp_abort_count' to an integer value - this is the number of failures for a given host before weathermap gives up on that host. The default is 0 (unlimited). This can improve performance if you are polling a lot of switch ports on a device that has gone down, for example.


External script

TARGET !scriptname

This plugin is fairly experimental and not thoroughly tested.

This plugin allows you to write small external scripts, and pass the value from the script into Weathermap. The output format of the script is the same one that MRTG uses for it's external scripts, so it's possible that you can even find an existing script to do what you want, somewhere else.

MRTG external scripts produce 4 lines of output. Line 1 is the state of the first (in) variable, line 2 is the state of the second (out) variable, line 3 is the uptime of the device, and line 4 is the name of the device. This plugin only cares about the first 2 lines, but it does expect there to be 4 lines.

This plugin also sets a few Hint Variables - external_line1, external_line2, external_line3, and external_line4 are the raw output of your script. This can be useful if your script really is intended for MRTG, as the 3rd line would be the device uptime and the 4th line would be it's name.


FPING

TARGET fping:hostname

This plugin is fairly experimental and not thoroughly tested. It's also rather slow, as each node takes 5 seconds or so to run. There are planned improvements to take advantage of fping's parallelisation support in a future version.

This plugin uses the 'fping' command-line tool [http://fping.sourceforge.net/] to ping a host, and returns various statistics about it's state.

You can adjust the number of pings (the default is 5), by setting the 'fping_ping_count' global variable to another number.

After it has been run, the 'in' value is the average ping time in milliseconds, the 'out' value is the percentage packet loss from 5 pings (or whatever you have set), the 'fping_min' Hint Variable is the minimum ping time in milliseconds, and 'fping_max' is the maximum.

Current Time

TARGET time:zonename

This plugin allows you to embed the current time in any timezone in your maps.

The 'in' value is the hours, the 'out' value is minutes.

The format for specifying timezones is the same as most unix systems - Europe/London, US/Eastern, Japan and Etc/GMT+10 are all valid. These names are usually stored in /usr/share/zoneinfo on a unix-based system.

This plugin also sets a few Hint Variables - time_time12, time_time24, time_time12ap. These are the current time in 12-hour, 24-hour and 12-hour-with-am/pm, respectively.

A typical use is: NODE timelabel LABEL {node:this:time_time12ap} USESCALE none TARGET time:Europe/Paris

NOTE:The timezone handling code requires at least PHP version 5.