Service definition duplication by configuration files import

[GMont]

Hi all,
I'm trying to import an existing configuration into a new installation of Nagios XI (this is a test environment with a manual install of Nagios XI over a CentOS 7 host).

I noticed that, importing the same files twice or more, service definitions get duplicated. It seems that only services are affected
(no host, contact or command definition duplication, for instance, even if I re-import all the files every time).
I thought that importing the same definitions twice or more would not duplicate entries (I vaguely remember doing
this with NagioSQL a while ago, I'm pretty sure there were no duplications, and I could keep importing the same files until the
whole import process was OK). "Overwrite Database" is checked.

Since, in production environment, I will need to perform configuration import in chunks, I cannot ensure I will
import a service definition just once, and I won't be able to scratch the config and restart, as I'm doing in test environment,
so I could en up having duplicates and needing to manage them by hand.

Is there a way to avoid duplicating service definitions, or to quickly get rid of the duplicates?

Thanks,
Giacomo

[scottwilkerson]

Are you following this doc and using the import prep tool?
https://assets.nagios.com/downloads/nag ... ios-XI.pdf

[GMont]

Thanks for the prompt reply.

Yes, I am:

Code: Select all

[root@localhost conf.preimport]# for file in ../conf.orig/nagios/*cfg ../conf.orig/nagios/objects/*cfg; do echo $file; php /usr/local/nagiosxi/tools/xiprepimport.php $file || break; done
[root@localhost conf.preimport]# mkdir /usr/local/nagios/etc/conf.preimport
[root@localhost conf.preimport]# cp * /usr/local/nagios/etc/conf.preimport/

Importing all the files under /usr/local/nagios/etc/conf.preimport after I did:

Code: Select all

[root@localhost conf.preimport]# cd /usr/local/nagiosxi/scripts/
[root@localhost scripts]# ./restore_defaults.sh

gives immediately some duplicates:

Code: Select all

[root@localhost scripts]# mysql -pnagiosxi nagiosql
MariaDB [nagiosql]> select service_description, count(*) c from tbl_service group by service_description having c > 1;
+---------------------+---+
| service_description | c |
+---------------------+---+
| PING                | 4 |
| Uptime              | 2 |
+---------------------+---+
2 rows in set (0.00 sec)

MariaDB [nagiosql]>

If I repeat the import process, PING still appears 4 times, Uptime 2 times (as above), and there are 7 services which appear 2 times:

Code: Select all

MariaDB [nagiosql]> select service_description, count(*) c from tbl_service group by service_description having c > 1;
+---------------------------------------+---+
| service_description                   | c |
+---------------------------------------+---+
| C:\ Drive Space                       | 2 |
| Check LDAP Domain Controller  | 2 |
| Check_XXXXX                      | 2 |
| KKKKKKK             | 2 |
| NNNNNN               | 2 |
| VVVVVVV               | 2 |
| GGGGGG         | 2 |
| PING                                  | 4 |
| Uptime                                | 2 |
+---------------------------------------+---+
9 rows in set (0.00 sec)

(I hid some service names). Importing the files again inserts another instance of those services:

Code: Select all

MariaDB [nagiosql]> select service_description, count(*) c from tbl_service group by service_description having c > 1;
+---------------------------------------+---+
| service_description                   | c |
+---------------------------------------+---+
| C:\ Drive Space                       | 3 |
| Check LDAP Domain Controller  | 3 |
| Check_url_XXXXX                      | 3 |
| KKKKKKK             | 3 |
| NNNNNN               | 3 |
| VVVVVVV               | 3 |
| GGGGGG         | 3 |
| PING                                  | 4 |
| Uptime                                | 2 |
+---------------------------------------+---+
9 rows in set (0.00 sec)

I could try cleaning configuration files before importing them and double-check to ensure there are no duplicate service
definitions before importing; after all, the configuration is not that big, and not all services are duplicated, just those above
(on a total of 94 services); but I'm still puzzled by the fact that those services get inserted multiple times.

Is there anything I can check to see if they're somewhat "special"?

Thanks.

[scottwilkerson]

This is logical because you may have the same service names on multiple hosts.

looking at the DB isn't where you should be looking, it's in the CCM.

Follow the instructions on the import order in the "Import Config Files" section and you should do fine

[GMont]

Thanks for you reply. I think the problem is not completely clear, I'll try to clarify.

An example: I have one configuration file, localhost.cfg, with this content:

Code: Select all

define host{
use                     linux-server            


host_name               localhost
alias                   localhost
address                 127.0.0.1
}
define hostgroup{
hostgroup_name  linux-servers 
alias           Linux Servers 
members         localhost     
}
define service{
use                             local-service         
host_name                       localhost
service_description             PING
check_command			check_ping!100.0,20%!500.0,60%
}
define service{
use                             local-service         
host_name                       localhost
service_description             Root Partition
check_command			check_local_disk!20%!10%!/
}
define service{
use                             local-service         
host_name                       localhost
service_description             Current Users
check_command			check_local_users!20!50
}
define service{
use                             local-service         
host_name                       localhost
service_description             Total Processes
check_command			check_local_procs!250!400!RSZDT
}
define service{
use                             local-service         
host_name                       localhost
service_description             Current Load
check_command			check_local_load!5.0,4.0,3.0!10.0,6.0,4.0
}
define service{
use                             local-service         
host_name                       localhost
service_description             Swap Usage
check_command			check_local_swap!20!10
}
define service{
use                             local-service         
host_name                       localhost
service_description             SSH
check_command			check_ssh
notifications_enabled		0
}
define service{
use                             local-service         
host_name                       localhost
service_description             HTTP
check_command			check_http
notifications_enabled		0
}

Notice it has one "PING" service, among the others. If I keep importing this file more and more times, no service gets duplicated
inside the db.

I have one other configuration file: linksys-srw224p.cfg:

Code: Select all

define service{
use			generic-service	
service_description	PING
check_command		check_ping!200.0,20%!600.0,60%	
normal_check_interval	5		
retry_check_interval	1		
}
define service{
use			generic-service	
host_name		linksys-srw224p
service_description	Uptime
check_command		check_snmp!-C public -o sysUpTime.0
}
define service{
use			generic-service	
host_name		linksys-srw224p
service_description	Port 1 Link Status
check_command		check_snmp!-C public -o ifOperStatus.1 -r 1 -m RFC1213-MIB
}
define service{
use			generic-service	
host_name		linksys-srw224p
service_description	Port 1 Bandwidth Usage
check_command		check_local_mrtgtraf!/var/lib/mrtg/192.168.1.253_1.log!AVG!1000000,1000000!5000000,5000000!10
}

It has one "PING" service, like localhost, but the check command is different. I understand that the two different
checks must be inserted in DB, so I must find one line for each service: one for localhost, the other for linksys-srw224p.

BUT, if I keep importing the linksys-srw224p.cfg, each time I get ANOTHER line for the PING service
(and JUST for that: the other three services are not duplicated).

To be clear, this happened after I imported the file 3 times, but just for the PING service:

Code: Select all

MariaDB [nagiosql]> select id, config_name, host_name, hostgroup_name, service_description, check_command, last_modified from tbl_service where service_description = 'PING'; 
+----+-----------------+-----------+----------------+---------------------+---------------------------+---------------------+
| id | config_name     | host_name | hostgroup_name | service_description | check_command             | last_modified       |
+----+-----------------+-----------+----------------+---------------------+---------------------------+---------------------+
|  1 | localhost       |         1 |              0 | PING                | 17!100.0,20%!500.0,60%    | 2017-08-24 13:39:59 |
| 68 | linksys-srw224p |         0 |              0 | PING                | 17!200.0,20%!600.0,60%    | 2017-08-24 13:39:21 |
| 87 | linksys-srw224p |         0 |              0 | PING                | 17!200.0,20%!600.0,60%    | 2017-08-24 13:39:59 |
| 89 | linksys-srw224p |         0 |              0 | PING                | 17!200.0,20%!600.0,60%    | 2017-08-24 13:48:26 |
+----+-----------------+-----------+----------------+---------------------+---------------------------+---------------------+

As you can see, localhost appears just once (even if keep importing localhost.cfg more and more and more...)
while linksys-srw224p is there 3 times.

The result is that, when generating configuration files, the service is inserted 3 times in the configuration file:

Code: Select all

###############################################################################
#
# Service configuration file
#
# Created by: Nagios Core Config Manager 2.6.8
# Date:       2017-08-24 14:11:02
# Version:    Nagios 3.x config file
#
# --- DO NOT EDIT THIS FILE BY HAND --- 
# Nagios CCM will overwrite all manual settings during the next update if you 
# would like to edit files manually, place them in the 'static' directory or 
# import your configs into the CCM by placing them in the 'import' directory.
#
###############################################################################

define service {
        service_description             PING
        use                             generic-service
        check_command                   check_ping!200.0,20%!600.0,60%
        check_interval                  5
        retry_interval                  1
        register                        1
        }

define service {
        service_description             PING
        use                             generic-service
        check_command                   check_ping!200.0,20%!600.0,60%
        check_interval                  5
        retry_interval                  1
        register                        1
        }

define service {
        service_description             PING
        use                             generic-service
        check_command                   check_ping!200.0,20%!600.0,60%
        check_interval                  5
        retry_interval                  1
        register                        1
        }

define service {
        host_name                       linksys-srw224p
        service_description             Port 1 Bandwidth Usage
        use                             generic-service
        check_command                   check_local_mrtgtraf!/var/lib/mrtg/192.168.1.253_1.log!AVG!1000000,1000000!5000000,5000000!10
        register                        1
        }

define service {
        host_name                       linksys-srw224p
        service_description             Port 1 Link Status
        use                             generic-service
        check_command                   check_snmp!-C public -o ifOperStatus.1 -r 1 -m RFC1213-MIB
        register                        1
        }

define service {
        host_name                       linksys-srw224p
        service_description             Uptime
        use                             generic-service
        check_command                   check_snmp!-C public -o sysUpTime.0
        register                        1
        }

###############################################################################
#
# Service configuration file
#
# END OF FILE
#
###############################################################################

I'm puzzled, because I don't see what could possibly mess with the configuration.

[scottwilkerson]

Are you sure you ran these files through the config prep tool? Because these are usually caught in this tool.

Also in your example for linksys-srw224p.cfg you should this definition right at the top

Code: Select all

define service{
use         generic-service   
service_description   PING
check_command      check_ping!200.0,20%!600.0,60%   
normal_check_interval   5      
retry_check_interval   1      
}

There isn't a host_name defined for this service.

[GMont]

Ok, that was my fault, I fixed it.

So far, PING and Uptime do not get duplicated anymore (PING was duplicated apparently because I accidentally deleted the host_name directive from the configuration, as you pointed out).

For the other services, I noticed that they were the only services specified with multiple hosts on the same host_name line, i.e.:

Code: Select all

host_name                        host1, host2, host3

I removed the configuration of those services from the _multiple_hosts.cfg file and split them manually into multiple service definitions:

Code: Select all

define service{
host_name                       host1
service_description             XXXXXXX
use                             myTemplate
check_command                   XXXXXXX_check_command
}


define service{
host_name                       host2
service_description             XXXXXXX
use                             myTemplate
check_command                   XXXXXXX_check_command
}

and so on. Now when I import the full configuration I have something like this:

Code: Select all

MariaDB [nagiosql]> select service_description, count(*) c from tbl_service group by service_description having c > 1;
+---------------------------------------+----+
| service_description                   | c  |
+---------------------------------------+----+
| Check LDAP Domain Controller          | 12 |
| Check_XXXX                            |  4 |
| YYYYY                                 |  3 |
| ZZZZZZ                                |  3 |
| KKKKK                                 | 10 |
| VVVVV                                 | 50 |
| PING                                  |  4 |
| Uptime                                |  2 |
+---------------------------------------+----+

that is, one service for each host (is it the intended behaviour?). This time, even if I import the configuration multiple times, they do not get duplicated (i.e. the numbers remain the same).

The only service which can still be duplicated is this:

Code: Select all

use                     generic-service
host_name               hostW
service_description     C:\ Drive Space
check_command           check_nt!USEDDISKSPACE!-l c -w 80 -c 90
}

which, oddly, is defined just for one host. But I think the situation is better. Now the only warnings I get are for the "C:\ Drive Space",
if I duplicate it:

Code: Select all

[...]
Reading configuration data...
Read main config file okay...
Warning: Duplicate definition found for service 'C:\ Drive Space' on host 'winserver' (config file '/usr/local/nagios/etc/services/hostW.cfg', starting on line 24)
Warning: Duplicate definition found for service 'C:\ Drive Space' on host 'winserver' (config file '/usr/local/nagios/etc/services/hostW.cfg', starting on line 16)
Read object config files okay... 
[...]

Does this make any sense?

Thanks.

[scottwilkerson]

This looks a lot better and as expected for all except the
C:\ Drive Space

I don't know why there would be 2 of these but you should be able to go to the services in the CCM select the "C:\ Drive Space" hostW from the config file drop down and remove the extra item