SNMP traps

Definition

SNMP traps are information sent using the SNMP protocol from monitored equipment to a poller server (satellite). This information contains multiple Attributes including:

  • Address of the equipment sending the information.
  • The root OID (Object Identifier) corresponding to the identifier of the message received.
  • The message sent via the SNMP trap which corresponds to a set of settings (1 to N).

In order to be able interpret the event received the Network supervisor server needs to possess in its configuration the necessary elements to translate the event. For this it must have a database containing the OID and the descriptions, this is what is called MIB files. There are two types of MIB:

  • Standard MIBs which use standardized OIDs and which are implemented by numerous manufacturers on their equipment.
  • MIB manufacturers who are specific to each one and often to each equipment model.

MIB manufacturers can be retrieved from the equipment. Centreon allows us to store the definition of SNMP traps in its MySQL database. The traps can subsequently be linked to passive services via the Relations tab of the definition of a service.

Architecture

SNMP trap processing by the central server

The processing of an SNMP trap is as follows:

  1. snmptrapd is the service enabling the retrieval of SNMP traps sent by the equipment (by default it listens on port UDP 162).
  2. Once the SNMP trap has been received, it is sent to the ‘centreontrapdforward’ script which writes the information received in a buffer folder (by default /var/spool/centreontrapd/).
  3. The ‘centreontrapd’ service reads the information received in the buffer folder and interprets the traps received checking, in the centreon database, the actions necessary to process these events.
  4. The ‘centreontrapd’ service transmits the information to the scheduler or the ‘centcore’ service (to send the information to a remote scheduler) which changes the status and the information associated with service to which the SNMP trap is linked.
../../_images/06_trap_centreon.png

SNMP trap processing by a satellite server

To keep a copy of the configuration of the SNMP traps on each satellite server, an SQLite database is charged with keeping the information of the traps contained in the MySQL database cached. This SQLite database is automatically generated by the central server.

The processing of an SNMP trap is as follows:

  1. snmptrapd is the service used to retrieve the SNMP traps sent by the equipment (by default it listens on port UDP 162).
  2. Once the SNMP trap is received, it is sent to the ‘centreontrapdforward’ script which writes the information received in a buffer folder (by default /var/spool/centreontrapd/).
  3. The ‘centreontrapd’ service reads the information received in the buffer folder and interprets the various traps received checking in the SQLite database the actions to be taken to process the traps received.
  4. The ‘centreontrapd’ service transmits the information to the scheduler which changes the status and the information associated with the service to which the SNMP trap is linked.
../../_images/06_trap_poller.png

Note

the Centcore process is responsible to copy the SQLite base on the remote collector.

Successive actions by the centreontrapd process

Successive actions by the centreontrapd process are:

../../_images/SNMP_Traps_management_general_view.png

Configuring services

Snmptrapd

To call the ‘centreontrapdfoward’ script, the file /etc/snmp/snmptrapd.conf must contain the following lines:

disableAuthorization yes
traphandle default su -l centreon -c "/usr/share/centreon/bin/centreontrapdforward"

You can optimize the performances of snmptrapd by using the following options:

  • -On don’t try to convert the OIDs
  • -t don’t log the traps to the syslog server
  • -n don’t try to convert the IP addresses into host names

These options can be changed in the file /etc/sysconfig/snmptrapd:

OPTIONS="-On -d -t -n -p /var/run/snmptrapd.pid"

It is also possible to place the snmptrapd buffer folder in the RAM. For this, add the following line in the file /etc/fstab:

tmpfs /var/run/snmpd                     tmpfs defaults,size=128m 0 0

centreontrapdforward

To change the buffer folder towards which the information will be written, change the configuration file /etc/centreon/centreontrapd.pm:

our %centreontrapd_config = (
    spool_directory => '/var/spool/centreontrapd/',
);

1;

You can also map the folder in the RAM, by adding the following line in the file: /etc/fstab:

tmpfs /var/spool/centreontrapd            tmpfs defaults,size=512m 0 0

centreontrapd

Two configuration files existent in centreontrapd:

  • /etc/centreon/conf.pm contains the connection information to the MySQL database
  • /etc/centreon/centreontrapd.pm contains the configuration of the centreontrapd service

Configuration of the service

In the file /etc/centreon/centreontrapd.pm we advise changing three settings only (if necessary):

  • If the mode option is defined in 1 centreontrapd functions on a satellite server, otherwise it functions on a central server (centreon).
  • The centreon_user option can be used to change the user executing the actions.
  • The spool_directory option can be used to change the buffer folder to be read (if you have changed it in the ‘centreontrapdforward’ configuration file).

Here is an example of possible configuration of the file /etc/centreon/centreontrapd.pm (the configuration file can be changed with ‘-config-extra = xxx’):

our %centreontrapd_config = (
    # Time in seconds before killing not gently sub process
    timeout_end => 30,
    spool_directory => "/var/spool/centreontrapd/",
    # Delay between spool directory check new files
    sleep => 2,
    # 1 = use the time that the trap was processed by centreontrapdforward
    use_trap_time => 1,
    net_snmp_perl_enable => 1,
    mibs_environment => '',
    remove_backslash_from_quotes => 1,
    dns_enable => 0,
    # Separator for arguments substitution
    separator => ' ',
    strip_domain => 0,
    strip_domain_list => [],
    duplicate_trap_window => 1,
    date_format => "",
    time_format => "",
    date_time_format => "",
    # Time in seconds before cache reload
    cache_unknown_traps_retention => 600,
    # 0 = central, 1 = poller
    mode => 0,
    cmd_timeout => 10,
    centreon_user => "centreon",
    # 0 => skip if MySQL error | 1 => don't skip (block) if MySQL error (and keep order)
    policy_trap => 1,
    # Log DB
    log_trap_db => 0,
    log_transaction_request_max => 500,
    log_transaction_timeout => 10,
    log_purge_time => 600
);

1;

Configuring the database connection

On Centreon Central server, edit the /etc/centreon/conf.pm file:

$centreon_config = {
    VarLib => "/var/lib/centreon",
    CentreonDir => "/usr/share/centreon/",
    "centreon_db" => "centreon",
    "centstorage_db" => "centreon_storage",
    "db_host" => "localhost:3306",
    "db_user" => "centreon",
    "db_passwd" => "centreon"
};

1;

On a poller, edit the /etc/centreon/centreontrapd.pm file:

our %centreontrapd_config = (
    ...
    "centreon_db" => "dbname=/etc/snmp/centreon_traps/centreontrapd.sdb",
    "centstorage_db" => "dbname=/etc/snmp/centreon_traps/centreontrapd.sdb",
    "db_host" => "",
    "db_user" => "",
    "db_passwd" => "",
    "db_type" => 'SQLite',
    ...
);

1;

Centreon configuration

Add a manufacturer

Within centreon, the root OIDs of the SNMP traps is filed by manufacturer. To add a manufacturer:

  1. Go into the menu: Configuration > SNMP traps
  2. In the left menu, click on Manufacturer
  3. Click on Add
../../_images/06constructors.png
  • The Name and Alias fields define the name and the alias of the manufacturer
  • The Description field provides an indication about the manufacturer

Importation of MIBs

It is also possible to import OIDs from MIBs provided by the manufacturers. To do this :

  1. Go into the menu: Configuration > SNMP traps
  2. In the left menu, click on MIBs
  3. Click on Import
../../_images/06importmibssuccess.png
  • The Manufacturer list can be used to choose the manufacturer to which the MIB that you are importing belongs
  • The File (.mib) field can be used to load the MIB

Note

The dependencies of the MIBS that you import must be present in the folder /usr/share/snmp/mibs. Once the import is completed, delete the dependencies previously copied.

Note

Once the SNMP traps are imported, it is necessary to verify the “Monitoring” status associated with the events. By default it will be “OK”.

Manual configuration of traps

Basic configuration

It is also possible to create definitions of SNMP traps manually:

  1. Go into the menu: Configuration > SNMP traps
  2. Click on Add
../../_images/06addsnmptrap.png
  • The field Trap name defines the name of the trap.
  • The field Mode defines how to check the trap OID received.
  • The field OID defines the Root OID to be received for this trap to be considered as received.
  • The field Vendor name defines the name of the manufacturer to which the trap to be selected in the drop-down list belongs.
  • The field Output message of contains the message to be displayed in the event of reception of a trap containing the OID configured above.

Note

By default, the MIB contains the definition of this variable (E.g.: “Link up on interface $2. State: $4.”, here $2 will be replaced by the 2nd argument received in the event.). In the opposite situation, the variable $* can be used to display all the arguments contained in the trap.

Note

It is possible to construct the output message yourself. For this, use the MIB to know the arguments that will be present in the body of the event and retrieve the arguments with the variables $n. As each argument is identified by a OID, it is possible to use this OID directly to place it in the output message without knowing its position via the variable @{OID}.

  • The Default status field defines the “monitoring” status of the service in case of reception of the trap.
  • If the Submit result box is checked the result is submitted to the Network supervisor engine.
  • The Comments field (last field) contains by default the comment by the manufacturer of the SNMP trap. Most of the time, this comment indicates the list of variables contained in the SNMP trap (see the next chapter on advanced configuration).

Advanced configuration of the traps

It is possible to determine the status of a service from the value of a setting of the SNMP trap rather than from the Root OID. Previously the manufacturer defined an SNMP trap (Root OID) by type of event to be sent (linkUp / linkDown). Today, the tendency is to define a Root OID by category of events and then to define the event via a set of settings.

To do this, it is possible of define Advanced Matching mode by clicking on Add a new entry and by creating as many rules as necessary. For each rule, define the settings:

  • String defines the element on which the search will be applied (@OUTPUT@ defined all the Output messages translated).
  • Regexp defined the REGEXP type search to be applied.
  • Status defines the status of the service in the event of concordance.

Note

The order is important in the rules of correspondence because the process will stop at the first rule of which the correspondence is assured.

  • The Disable submit result if no matched rules field disables the sending of information to the scheduling engine if no correspondence with a rule is confirmed.
  • If the Reschedule associated services box is checked, the next check on the service, which should be ‘active’, should be reprogrammed as soon as possible after reception of the trap.
  • If the Execute special command box is checked, the command defined in Special command is executed.

Advanced trap configuration - Routing

The Advanced tab serves to configure the behavior of the handling process of the SNMP traps on its reception of the latter.

../../_images/06advancedconfiguration.png
  • Enable routing is used to enable the routing of information.
  • Route definition is used to define the command to be used for routing.

Before performing the processing of the event (translation of the Output message), it is possible to execute a command called PREEXEC. To do this, it is possible to define PREEXEC command (SNMPTT type) by clicking on Add a new entry and create as many rules as necessary.

  • PREEXEC command defines the command to be executed.

Here is an example of use with the linkUP trap:

For a Cisco equipment, $2 == ifDescr contains the port number of the interface (GigabitEthernet0/1 for instance). The best description of the interface is in the SNMP if Alias field.

The following command can be used to retrieve this value :

snmpget -v 2c -Ovq -c <community> <cisco switch> ifAlias.$1

To use the result of the PREEXEC command in the Output message, it is necessary to use the variable $p{n} where ‘n’ corresponds to the order of definition of the command.

Example:

"Interface $2 ( $p1 ) linkUP. State: $4." "$CA"

The result will have the form: Interface GigabitEthernet0/1 ( NAS Server ) linkUP. State: up

  • The Insert trap’s information into database box, if checked, record the SNMP trap information in the database field can be used define whether or not to classify the traps by day in the database.
  • The Timeout field expressed in seconds is used to define the maximum processing time of the event including the pre-processing commands (PREEXEC) and post-processing commands (special command).
  • The Execution interval field expressed in seconds is used to define the maximum waiting time between two processing operations of an event.
  • The Execution Type field is used to enable the Execution interval by defining the conditions by Root OID, by the Root OID and host combination or, to disable this restriction, None.
  • The Execution Method field is used to define if on reception of multiple same events (Root OID). The execution is either Sequential or Parallel.

Advanced trap configuration - Custom code

The field custom code allows custom Perl processing. To enable this feature, you must set secure_mode to 0 in /etc/centreon/centreontrapd.pm file as:

our %centreontrapd_config = (
   ...
   secure_mode => 0,
   ....
);

1;

For example, to decode the 4 arguments from hexadecimal, the custom code will be:

if ($self->{trap_data}->{entvar}->[3] =~ /[[:xdigit:]]+/) {
    my $hexa_value = $self->{trap_data}->{entvar}->[3];
    $hexa_value =~ s/ //g;
    $self->{trap_data}->{entvar}->[3] = pack('H*', $hexa_value);
}

Note

Beware the argument table starts at 0 for argument 1 of the SNMP trap.

Variables

When adding a rule of correspondence or executing a special command it is possible to transmit arguments to the String or Special command fields. These arguments are listed in the table below:

Macro name Description
@{NUMERIC_OID} Retrieval of the value of an argument via its OID, e.g.: @{.1.3.6.1.4.1.9.9.43.1.1.1}
$1, $2... Retrieval of the value of an argument via its order of appearance
$p1, $p2,... Value of the command: PREEXEC ($p1 = at the first command, $p2 at the second, ...)
$* All the arguments separated by a space
@HOSTNAME@ Host name (in centreon) to which the service is attached
@HOSTADDRESS@ IP address of the host sending the trap
@HOSTADDRESS2@ DNS name of the host sending the trap (if the server fails to effect a reverse DNS resolution we retrieve the IP address)
@SERVICEDESC@ Service name
@TRAPOUTPUT@ ou @OUTPUT@ Output of the traps
@STATUS@ Service state
@SEVERITYNAME@ Criticality name
@SEVERITYLEVEL@ Criticality level
@TIME@ Trap reception timestamp
@POLLERID@ ID of the poller having received the trap
@POLLERADDRESS@ IP address of the poller having received the trap
@CMDFILE@ Path to the command file of CentCore (central) or of centreon Engine (collector)

In addition, there are special variables that can be used in the Routing settings section at the level of the Routing command if the option Enable routing is selected :

Macro name Description
@GETHOSTBYADDR($1)@ Reverse DNS resolution used to find the DNS name DNS from the IP address (127.0.0.1 -> localhost)
@GETHOSTBYNAME($1)@ DNS resolution used to find the IP address from the DNS name (localhost -> 127.0.0.1)

Applying the changes

To be able to export the OID present in the database in the configuration file to centreontrapd, follow the following procedure:

  1. Go into the menu: Configuration > SNMP traps
  2. In the left menu, click on Generate
  3. Select the poller to which you want to export the configuration files
  4. Check Generate traps database and Apply configurations
  5. In the drop-down list Send signal (the Reload option is preferable)
  6. Click on the Generate button