apply reconfiguration fails on nonexistent object

This support forum board is for support questions relating to Nagios XI, our flagship commercial network monitoring solution.
Locked
kendallchenoweth
Posts: 195
Joined: Fri Sep 13, 2013 10:43 am

apply reconfiguration fails on nonexistent object

Post by kendallchenoweth »

When I try to apply configuration I get an error about a service object and associated hostgroups. So i deleted that service object and applied configuration again. This service object doesn't exist in the etc/services, but I get an error that this object fails validation still. Clearly something is cached somewhere. How can I clear the Nagios object cache in order to restart without this object and then I can bring in the hosts, hostgroups and service separately to figure out what's wrong with them.

Thanks!
Top
abrist
Red Shirt
Posts: 8334
Joined: Thu Nov 15, 2012 1:20 pm

Re: apply reconfiguration fails on nonexistent object

Post by abrist »

This could actually be a sudoer problem as well. What is the output from running the following from the cli:

Code: Select all

cd /usr/local/nagiosxi/scripts/
./reconfigure_nagios.sh
Former Nagios employee
"It is turtles. All. The. Way. Down. . . .and maybe an elephant or two."
VI VI VI - The editor of the Beast!
Come to the Dark Side.
Top
kendallchenoweth
Posts: 195
Joined: Fri Sep 13, 2013 10:43 am

Re: apply reconfiguration fails on nonexistent object

Post by kendallchenoweth »

This is wierd... reconfigure_nagios.sh doesn't show an error.

Code: Select all

./reconfigure_nagios.sh 
URL: https://localhost/nagiosxi/includes/components/ccm/
CMDLINE
/usr/bin/wget --save-cookies nagiosql.cookies --keep-session-cookies https://localhost/nagiosxi/includes/components/ccm/ --no-check-certificate --post-data 'submit=Login&hidelog=true&loginSubmitted=true&username=nagiosxi&password=guo64u' -O nagiosql.login--2015-05-15 14:18:57--  https://localhost/nagiosxi/includes/components/ccm/
Resolving localhost... 127.0.0.1
Connecting to localhost|127.0.0.1|:443... connected.
WARNING: cannot verify localhost’s certificate, issued by “/C=US/ST=MA/L=Natick/O=the MathWorks/OU=SSG/CN=ec2-50-19-171-201.compute1.amazonaws.com/[email protected]”:
  Self-signed certificate encountered.
WARNING: certificate common name “ec2-50-19-171-201.compute1.amazonaws.com” doesn’t match requested host name “localhost”.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: “nagiosql.login”

    [ <=>                                                                                                       ] 10,409      --.-K/s   in 0.02s   

2015-05-15 14:18:57 (559 KB/s) - “nagiosql.login” saved [10409]

LOGIN SUCCESSFUL!
IMPORTING CONFIG FILES...URL: https://localhost/nagiosxi/includes/components/ccm/
Array
(
)
SETUID ROOT OK
RESETTING PERMS
URL: https://localhost/nagiosxi/includes/components/ccm/
CMDLINE
/usr/bin/wget --save-cookies nagiosql.cookies --keep-session-cookies https://localhost/nagiosxi/includes/components/ccm/ --no-check-certificate --post-data 'submit=Login&hidelog=true&loginSubmitted=true&username=nagiosxi&password=guo64u' -O nagiosql.login--2015-05-15 14:18:58--  https://localhost/nagiosxi/includes/components/ccm/
Resolving localhost... 127.0.0.1
Connecting to localhost|127.0.0.1|:443... connected.
WARNING: cannot verify localhost’s certificate, issued by “/C=US/ST=MA/L=Natick/O=the MathWorks/OU=SSG/CN=ec2-50-19-171-201.compute1.amazonaws.com/[email protected]”:
  Self-signed certificate encountered.
WARNING: certificate common name “ec2-50-19-171-201.compute1.amazonaws.com” doesn’t match requested host name “localhost”.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: “nagiosql.login”

    [ <=>                                                                                                       ] 10,409      --.-K/s   in 0.02s   

2015-05-15 14:18:58 (555 KB/s) - “nagiosql.login” saved [10409]

LOGIN SUCCESSFUL!
URL: https://localhost/nagiosxi/includes/components/ccm/
CMDLINE:
/usr/bin/wget --load-cookies=nagiosql.cookies https://localhost/nagiosxi/includes/components/ccm/ --no-check-certificate --post-data 'cmd=apply&type=writeConfig' -O nagiosql.export.monitoring
--2015-05-15 14:18:58--  https://localhost/nagiosxi/includes/components/ccm/
Resolving localhost... 127.0.0.1
Connecting to localhost|127.0.0.1|:443... connected.
WARNING: cannot verify localhost’s certificate, issued by “/C=US/ST=MA/L=Natick/O=the MathWorks/OU=SSG/CN=ec2-50-19-171-201.compute1.amazonaws.com/[email protected]”:
  Self-signed certificate encountered.
WARNING: certificate common name “ec2-50-19-171-201.compute1.amazonaws.com” doesn’t match requested host name “localhost”.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: “nagiosql.export.monitoring”

    [      <=>                                                                                                  ] 13,265      2.78K/s   in 4.7s    

2015-05-15 14:19:03 (2.78 KB/s) - “nagiosql.export.monitoring” saved [13265]

WRITE CONFIGS SUCCESSFUL!
OUTPUT: 
Nagios Core 4.0.8
Copyright (c) 2009-present Nagios Core Development Team and Community Contributors
Copyright (c) 1999-2009 Ethan Galstad
Last Modified: 08-12-2014
License: GPL

Website: http://www.nagios.org
Reading configuration data...
   Read main config file okay...
   Read object config files okay...

Running pre-flight check on configuration data...

Checking objects...
	Checked 464 services.
	Checked 113 hosts.
	Checked 113 host groups.
	Checked 31 service groups.
	Checked 31 contacts.
	Checked 23 contact groups.
	Checked 197 commands.
	Checked 11 time periods.
	Checked 24 host escalations.
	Checked 12034 service escalations.
Checking for circular paths...
	Checked 113 hosts
	Checked 0 service dependencies
	Checked 0 host dependencies
	Checked 11 timeperiods
Checking global event handlers...
Checking obsessive compulsive processor commands...
Checking misc settings...

Total Warnings: 0
Total Errors:   0

Things look okay - No serious problems were detected during the pre-flight check
RET: 0
Running configuration check...done.
Stopping nagios: .done.
Starting nagios: done.
Stopping ndo2db: done.
Starting ndo2db: done.
Stopping ndo2db: head: cannot open `/usr/local/nagios/var/ndo2db.lock' for reading: No such file or directory
done.
Starting ndo2db: done.
[nagios@ip-10-5-189-249 scripts]$ exit
Script done, file is typescript
[nagios@ip-10-5-189-249 scripts]$ more typescript 
Script started on Fri 15 May 2015 02:18:55 PM EDT
[nagios@ip-10-5-189-249 scripts]$ ./reconfigure_nagios.sh 
URL: https://localhost/nagiosxi/includes/components/ccm/
CMDLINE
/usr/bin/wget --save-cookies nagiosql.cookies --keep-session-cookies https://localhost/nagiosxi/includes/components/ccm/ --no-check-certificate --po
st-data 'submit=Login&hidelog=true&loginSubmitted=true&username=nagiosxi&password=guo64u' -O nagiosql.login--2015-05-15 14:18:57--  https://localhos
t/nagiosxi/includes/components/ccm/
Resolving localhost... 127.0.0.1
Connecting to localhost|127.0.0.1|:443... connected.
WARNING: cannot verify localhost’s certificate, issued by “/C=US/ST=MA/L=Natick/O=the MathWorks/OU=SSG/CN=ec2-50-19-171-201.compute1.amazonaws.com/e
[email protected]”:
  Self-signed certificate encountered.
WARNING: certificate common name “ec2-50-19-171-201.compute1.amazonaws.com” doesn’t match requested host name “localhost”.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: “nagiosql.login”

    [ <=>                                                                                                       ] 10,409      --.-K/s   in 0.02s   

2015-05-15 14:18:57 (559 KB/s) - “nagiosql.login” saved [10409]

LOGIN SUCCESSFUL!
IMPORTING CONFIG FILES...URL: https://localhost/nagiosxi/includes/components/ccm/
Array
(
)
SETUID ROOT OK
RESETTING PERMS
URL: https://localhost/nagiosxi/includes/components/ccm/
CMDLINE
/usr/bin/wget --save-cookies nagiosql.cookies --keep-session-cookies https://localhost/nagiosxi/includes/components/ccm/ --no-check-certificate --po
st-data 'submit=Login&hidelog=true&loginSubmitted=true&username=nagiosxi&password=guo64u' -O nagiosql.login--2015-05-15 14:18:58--  https://localhos
t/nagiosxi/includes/components/ccm/
Resolving localhost... 127.0.0.1
Connecting to localhost|127.0.0.1|:443... connected.
WARNING: cannot verify localhost’s certificate, issued by “/C=US/ST=MA/L=Natick/O=the MathWorks/OU=SSG/CN=ec2-50-19-171-201.compute1.amazonaws.com/e
[email protected]”:
  Self-signed certificate encountered.
WARNING: certificate common name “ec2-50-19-171-201.compute1.amazonaws.com” doesn’t match requested host name “localhost”.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: “nagiosql.login”

    [ <=>                                                                                                       ] 10,409      --.-K/s   in 0.02s   

2015-05-15 14:18:58 (555 KB/s) - “nagiosql.login” saved [10409]

LOGIN SUCCESSFUL!
URL: https://localhost/nagiosxi/includes/components/ccm/
CMDLINE:
/usr/bin/wget --load-cookies=nagiosql.cookies https://localhost/nagiosxi/includes/components/ccm/ --no-check-certificate --post-data 'cmd=apply&type
=writeConfig' -O nagiosql.export.monitoring
--2015-05-15 14:18:58--  https://localhost/nagiosxi/includes/components/ccm/
Resolving localhost... 127.0.0.1
Connecting to localhost|127.0.0.1|:443... connected.
WARNING: cannot verify localhost’s certificate, issued by “/C=US/ST=MA/L=Natick/O=the MathWorks/OU=SSG/CN=ec2-50-19-171-201.compute1.amazonaws.com/e
[email protected]”:
  Self-signed certificate encountered.
WARNING: certificate common name “ec2-50-19-171-201.compute1.amazonaws.com” doesn’t match requested host name “localhost”.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: “nagiosql.export.monitoring”

    [   <=>                                                                                                     ] 8,278       --.-K/s              
    [      <=>                                                                                                  ] 13,265      2.78K/s   in 4.7s    

2015-05-15 14:19:03 (2.78 KB/s) - “nagiosql.export.monitoring” saved [13265]

WRITE CONFIGS SUCCESSFUL!
OUTPUT: 
Nagios Core 4.0.8
Copyright (c) 2009-present Nagios Core Development Team and Community Contributors
Copyright (c) 1999-2009 Ethan Galstad
Last Modified: 08-12-2014
License: GPL

Website: http://www.nagios.org
Reading configuration data...
   Read main config file okay...
   Read object config files okay...

Running pre-flight check on configuration data...

Checking objects...
	Checked 464 services.
	Checked 113 hosts.
	Checked 113 host groups.
	Checked 31 service groups.
	Checked 31 contacts.
	Checked 23 contact groups.
	Checked 197 commands.
	Checked 11 time periods.
	Checked 24 host escalations.
	Checked 12034 service escalations.
Checking for circular paths...
	Checked 113 hosts
	Checked 0 service dependencies
	Checked 0 host dependencies
	Checked 11 timeperiods
Checking global event handlers...
Checking obsessive compulsive processor commands...
Checking misc settings...

Total Warnings: 0
Total Errors:   0

Things look okay - No serious problems were detected during the pre-flight check
RET: 0
Running configuration check...done.
Stopping nagios: .done.
Starting nagios: done.

If I click "Apply Configuration", it fails. I haven't made any changes between reconfiguration updates.

Error: Could not find any hostgroup matching 'motw_aps-broker-prod-useast1' (config file '/usr/local/nagios/etc/hosts/aps-authnz-prod-useast1_1.cfg', starting on line 16)
Error: Failed to process hostgroup names for host 'aps-authnz-prod-useast1_1' (config file '/usr/local/nagios/etc/hosts/aps-authnz-prod-useast1_1.cfg', starting at line 16)

/usr/local/nagios/etc/hosts/aps-authnz-prod-useast1_1.cfg doesnt exist in /usr/local/etc/hosts and is marked for revomal (active: no/red).

If
Top
ssax
Dreams In Code
Posts: 7682
Joined: Wed Feb 11, 2015 12:54 pm

Re: apply reconfiguration fails on nonexistent object

Post by ssax »

Try to su to nagios and run a reconfigure and see if that fails for you:

Code: Select all

su - nagios
cd /usr/local/nagiosxi/scripts
./reconfigure_nagios.sh
Might also want to check if you have an /etc/sudoers.d/nagiosxi file.
- If that file exists, view your /etc/sudoers file and see if the XI entries are at the bottom.
- If the XI entries are in the bottom of your /etc/sudoers, delete the /etc/sudoers.d/nagiosxi file and try to apply config again.
Top
ssax
Dreams In Code
Posts: 7682
Joined: Wed Feb 11, 2015 12:54 pm

Re: apply reconfiguration fails on nonexistent object

Post by ssax »

You could also try going to Configure > Core Config Manager > Tools > Write Config Files
- Click Delete
- Click Write
- Click Verify

If you get any errors, post the full output, if not, try to apply config.
Top
kendallchenoweth
Posts: 195
Joined: Fri Sep 13, 2013 10:43 am

Re: apply reconfiguration fails on nonexistent object

Post by kendallchenoweth »

Here's an update on where I stand in this problem.

I've copied out all of my current environment cfg files into a backup directory.
I deleted all of the objects from the CCM.
I imported the default set of objects into nagios.
I will re-import my current environment once I resolve the problem.

Now here is where it gets interesting.

The objects import into nagiosql and generate files in /usr/local/nagios/etc.
/usr/local/nagios/nagios/bin/nagios -v /usr/local/nagios/etc/ returns 0 warnings or errors.
http://<host>/nagios shows the default configuration

http://<host>/nagiosxi now shows the default configuration

The result from "apply configuration" returns an error

Code: Select all

Error: There are no services defined!
Error: There are no hosts defined!
Error: Service notification command 'xi_service_notification_handler' specified for contact 'xi_default_contact' is not defined anywhere!
Error: Host notification command 'xi_host_notification_handler' specified for contact 'xi_default_contact' is not defined anywhere!
Error: Global host event handler command 'xi_host_event_handler' is not defined anywhere!
Error: Global service event handler command 'xi_service_event_handler' is not defined anywhere!
But these objects do exist!

Command Name Command Line Active Actions ID
xi_service_notification_handler /usr/bin/php /usr/local/nagiosxi/scripts/handle_nagioscore_notification.php --notification-type=service --contact="$CONTACTNAME$" --contactemail="$CONTACTEMAIL$" --type=$NOTIFICATIONTYPE$ --escalated="$NOTIFICATIONISESCALATED$" --author="$NOTIFICATIONAUTHOR$" --comments="$NOTIFICATIONCOMMENT$" --host="$HOSTNAME$" --hostaddress="$HOSTADDRESS$" --hostalias="$HOSTALIAS$" --hostdisplayname="$HOSTDISPLAYNAME$" --service="$SERVICEDESC$" --hoststate=$HOSTSTATE$ --hoststateid=$HOSTSTATEID$ --servicestate=$SERVICESTATE$ --servicestateid=$SERVICESTATEID$ --lastservicestate=$LASTSERVICESTATE$ --lastservicestateid=$LASTSERVICESTATEID$ --servicestatetype=$SERVICESTATETYPE$ --currentattempt=$SERVICEATTEMPT$ --maxattempts=$MAXSERVICEATTEMPTS$ --serviceeventid=$SERVICEEVENTID$ --serviceproblemid=$SERVICEPROBLEMID$ --serviceoutput="$SERVICEOUTPUT$" --longserviceoutput="$LONGSERVICEOUTPUT$" --datetime="$LONGDATETIME$" Yes EditCopyDownloadInfoDelete 314

Command Name Command Line Active Actions ID
xi_host_notification_handler /usr/bin/php /usr/local/nagiosxi/scripts/handle_nagioscore_notification.php --notification-type=host --contact="$CONTACTNAME$" --contactemail="$CONTACTEMAIL$" --type=$NOTIFICATIONTYPE$ --escalated="$NOTIFICATIONISESCALATED$" --author="$NOTIFICATIONAUTHOR$" --comments="$NOTIFICATIONCOMMENT$" --host="$HOSTNAME$" --hostaddress="$HOSTADDRESS$" --hostalias="$HOSTALIAS$" --hostdisplayname="$HOSTDISPLAYNAME$" --hoststate=$HOSTSTATE$ --hoststateid=$HOSTSTATEID$ --lasthoststate=$LASTHOSTSTATE$ --lasthoststateid=$LASTHOSTSTATEID$ --hoststatetype=$HOSTSTATETYPE$ --currentattempt=$HOSTATTEMPT$ --maxattempts=$MAXHOSTATTEMPTS$ --hosteventid=$HOSTEVENTID$ --hostproblemid=$HOSTPROBLEMID$ --hostoutput="$HOSTOUTPUT$" --longhostoutput="$LONGHOSTOUTPUT$" --datetime="$LONGDATETIME$" Yes EditCopyDownloadInfoDelete 312

Command Name Command Line Active Actions ID
xi_host_event_handler /usr/bin/php /usr/local/nagiosxi/scripts/handle_nagioscore_event.php --handler-type=host --host="$HOSTNAME$" --hostaddress="$HOSTADDRESS$" --hoststate=$HOSTSTATE$ --hoststateid=$HOSTSTATEID$ --lasthoststate=$LASTHOSTSTATE$ --lasthoststateid=$LASTHOSTSTATEID$ --hoststatetype=$HOSTSTATETYPE$ --currentattempt=$HOSTATTEMPT$ --maxattempts=$MAXHOSTATTEMPTS$ --hosteventid=$HOSTEVENTID$ --hostproblemid=$HOSTPROBLEMID$ --hostoutput="$HOSTOUTPUT$" --longhostoutput="$LONGHOSTOUTPUT$" Yes EditCopyDownloadInfoDelete 311



Command Name Command Line Active Actions ID
xi_service_event_handler /usr/bin/php /usr/local/nagiosxi/scripts/handle_nagioscore_event.php --handler-type=service --host="$HOSTNAME$" --service="$SERVICEDESC$" --hostaddress="$HOSTADDRESS$" --hoststate=$HOSTSTATE$ --hoststateid=$HOSTSTATEID$ --hosteventid=$HOSTEVENTID$ --hostproblemid=$HOSTPROBLEMID$ --servicestate=$SERVICESTATE$ --servicestateid=$SERVICESTATEID$ --lastservicestate=$LASTSERVICESTATE$ --lastservicestateid=$LASTSERVICESTATEID$ --servicestatetype=$SERVICESTATETYPE$ --currentattempt=$SERVICEATTEMPT$ --maxattempts=$MAXSERVICEATTEMPTS$ --serviceeventid=$SERVICEEVENTID$ --serviceproblemid=$SERVICEPROBLEMID$ --serviceoutput="$SERVICEOUTPUT$" --longserviceoutput="$LONGSERVICEOUTPUT$" Yes 313


The host has a status of "not applied"

localhost localhost Yes Not Applied 222

Ditto for the services.
Config Name Service Description Active Status Actions ID
localhost_services Current Load Yes Not Applied 793
localhost_services Current Users Yes Not Applied 794
localhost_services HTTP Yes Not Applied 795
localhost_services PING Yes Not Applied 796
localhost_services Root Partition Yes Not Applied 797
localhost_services SSH Yes Not Applied 798
localhost_services Swap Usage Yes Not Applied 799
localhost_services Total Processes Yes Not Applied 800
Top
ssax
Dreams In Code
Posts: 7682
Joined: Wed Feb 11, 2015 12:54 pm

Re: apply reconfiguration fails on nonexistent object

Post by ssax »

Did you do these things I asked?
ssax wrote:Try to su to nagios and run a reconfigure and see if that fails for you:

Code: Select all

su - nagios
cd /usr/local/nagiosxi/scripts
./reconfigure_nagios.sh
Might also want to check if you have an /etc/sudoers.d/nagiosxi file.
- If that file exists, view your /etc/sudoers file and see if the XI entries are at the bottom.
- If the XI entries are in the bottom of your /etc/sudoers, delete the /etc/sudoers.d/nagiosxi file and try to apply config again.
Top
kendallchenoweth
Posts: 195
Joined: Fri Sep 13, 2013 10:43 am

Re: apply reconfiguration fails on nonexistent object

Post by kendallchenoweth »

reconfigure_nagios.sh does NOT fail.

There is no sudoers.d/nagiosxi file. The NAGIOSXI entries are in the /etc/sudoers file.

What's different about clicking on "apply configuration" vs running reconfigure_nagios.sh?

/etc/sudoers file

Code: Select all

User_Alias      NAGIOSXI=nagios
User_Alias              NAGIOSXIWEB=apache
NAGIOSXI ALL = NOPASSWD:/etc/init.d/nagios start
NAGIOSXI ALL = NOPASSWD:/etc/init.d/nagios stop
NAGIOSXI ALL = NOPASSWD:/etc/init.d/nagios restart
NAGIOSXI ALL = NOPASSWD:/etc/init.d/nagios reload
NAGIOSXI ALL = NOPASSWD:/etc/init.d/nagios status
NAGIOSXI ALL = NOPASSWD:/etc/init.d/nagios checkconfig
NAGIOSXI ALL = NOPASSWD:/etc/init.d/ndo2db start
NAGIOSXI ALL = NOPASSWD:/etc/init.d/ndo2db stop
NAGIOSXI ALL = NOPASSWD:/etc/init.d/ndo2db restart
NAGIOSXI ALL = NOPASSWD:/etc/init.d/ndo2db reload
NAGIOSXI ALL = NOPASSWD:/etc/init.d/ndo2db status
NAGIOSXI ALL = NOPASSWD:/etc/init.d/npcd start
NAGIOSXI ALL = NOPASSWD:/etc/init.d/npcd stop
NAGIOSXI ALL = NOPASSWD:/etc/init.d/npcd restart
NAGIOSXI ALL = NOPASSWD:/etc/init.d/npcd reload
NAGIOSXI ALL = NOPASSWD:/etc/init.d/npcd status
NAGIOSXI ALL = NOPASSWD:/usr/bin/nmap *
NAGIOSXI ALL = NOPASSWD:/usr/local/nagiosxi/scripts/upgrade_to_latest.sh
NAGIOSXI ALL = NOPASSWD:/usr/local/nagiosxi/scripts/change_timezone.sh
NAGIOSXI ALL = NOPASSWD:/usr/local/nagiosxi/scripts/manage_services.sh
NAGIOSXIWEB ALL = NOPASSWD:/usr/bin/tail -100 /var/log/messages
NAGIOSXIWEB ALL = NOPASSWD:/usr/bin/tail -100 /var/log/httpd/error_log
NAGIOSXIWEB ALL = NOPASSWD:/usr/bin/tail -100 /var/log/mysqld.log
NAGIOSXIWEB ALL = NOPASSWD:/usr/bin/nmap *
NAGIOSXIWEB ALL = NOPASSWD:/etc/init.d/snmptt restart
NAGIOSXIWEB ALL = NOPASSWD:/usr/local/nagiosxi/scripts/repair_databases.sh
NAGIOSXIWEB ALL = NOPASSWD:/usr/local/nagiosxi/scripts/manage_services.sh
Top
ssax
Dreams In Code
Posts: 7682
Joined: Wed Feb 11, 2015 12:54 pm

Re: apply reconfiguration fails on nonexistent object

Post by ssax »

But did you run "su - nagios" before running the reconfigure command?

The apply configuration essentially runs: cd /usr/local/nagiosxi/scripts && ./reconfigure
Top
kendallchenoweth
Posts: 195
Joined: Fri Sep 13, 2013 10:43 am

Re: apply reconfiguration fails on nonexistent object

Post by kendallchenoweth »

I logged in as the user nagios and ran the reconfigure_nagios.sh command as the user nagios.
Top
Locked

Return to “Nagios XI”