Auto-Discovery Jobs stop working after upgrade

[stf_792]

(Accidentally posted in another forum)

https://support.nagios.com/forum/viewto ... =6&t=39276

Hello,

We ran into a problem with our XI installation after upgrading to 5.2.9 from 5.2.8.

The problem that all our auto-discovery jobs are no longer run. Job just show spinning circle.

Auto-Discovery Component version is 2.2.3 and wizard version is 1.40

When we go to Auto-Discovery Wizard, it says that there are no completed jobs. (untrue)

Autodiscovery jobs folder look like this

Code: Select all

    /usr/local/nagiosxi/html/includes/components/autodiscovery/jobs:
    total 5264
    -rw-r-----. 1 nagios nagios   70692 Jun 28 09:25 0niamj.out
    -rw-r-----. 1 nagios nagios       0 Jun 28 09:25 0niamj.watch
    -rw-r-----. 1 nagios nagios   83173 Jun 28 09:20 2aemhq.out
    -rw-r-----. 1 nagios nagios       0 Jun 28 09:20 2aemhq.watch
    -rw-r-----. 1 nagios nagios  108788 Jun 28 09:30 42iqcu.out
    -rw-r-----. 1 nagios nagios       0 Jun 28 09:30 42iqcu.watch
    -rw-r-----. 1 nagios nagios  253879 Jun 28 09:56 533v55.out
    -rw-r-----. 1 nagios nagios       0 Jun 28 09:55 533v55.watch
    -rw-r-----. 1 nagios nagios  402578 May 26 09:11 73aj2t.out
    -rw-r-----. 1 nagios nagios       0 May 26 09:10 73aj2t.watch
    -rw-r-----  1 nagios nagios  567960 Jun 28 10:43 782r64.out
    -rw-r-----  1 nagios nagios       0 Jun 28 10:40 782r64.watch
    -rw-r-----. 1 nagios nagios   75448 Jun 28 09:05 8b92uu.out
    -rw-r-----. 1 nagios nagios       0 Jun 28 09:05 8b92uu.watch
    -rw-r-----. 1 nagios nagios   39449 Jun 28 09:40 ftc6vv.out
    -rw-r-----. 1 nagios nagios       0 Jun 28 09:40 ftc6vv.watch
    -rw-r-----  1 nagios nagios   73744 Jun 28 10:15 ge3tg8.out
    -rw-r-----  1 nagios nagios       0 Jun 28 10:15 ge3tg8.watch
    -rw-r-----. 1 nagios nagios  411503 Jun 28 09:01 hctihv.out
    -rw-r-----. 1 nagios nagios       0 Jun 28 09:00 hctihv.watch
    -rw-r-----  1 nagios nagios   21815 Jun 28 10:20 j6cqfs.out
    -rw-r-----  1 nagios nagios       0 Jun 28 10:20 j6cqfs.watch
    -rw-r-----. 1 nagios nagios 1725014 Jun 28 08:08 jer37c.out
    -rw-r-----. 1 nagios nagios       0 Jun 28 08:00 jer37c.watch
    -rw-r-----. 1 nagios nagios  121105 Jun 28 09:15 lu670a.out
    -rw-r-----. 1 nagios nagios       0 Jun 28 09:15 lu670a.watch
    -rw-r-----. 1 nagios nagios  145356 May 26 09:06 pohcro.out
    -rw-r-----. 1 nagios nagios       0 May 26 09:05 pohcro.watch
    -rw-r-----. 1 nagios nagios  429076 Jun 28 09:39 rmhr9k.out
    -rw-r-----. 1 nagios nagios       0 Jun 28 09:35 rmhr9k.watch
    -rw-r-----. 1 nagios nagios  396802 Jun 28 09:52 rqnocp.out
    -rw-r-----. 1 nagios nagios       0 Jun 28 09:50 rqnocp.watch
    -rw-r-----. 1 nagios nagios  134121 Jun 28 09:25 tdngcs.out
    -rw-r-----. 1 nagios nagios       0 Jun 28 09:25 tdngcs.watch
    -rw-r-----. 1 nagios nagios  294465 Jun 28 09:11 tlcpvo.out
    -rw-r-----. 1 nagios nagios       0 Jun 28 09:10 tlcpvo.watch

we are trying to avoid recreation of all the jobs, as we would like to preserve historical scan info.

Thanks in advance.

[ssax]

Are you seeing them running in /var/log/cron? Do you see the cron jobs listed in /var/spool/cron/apache?

Take a look in your /var/log/httpd/error_log and /var/log/httpd/ssl_error_log (if using SSL) to see if you can find any related errors and post them here.


Thank you

[ssax]

Also, I've deleted the other thread, thanks for letting us know.

[stf_792]

All jobs are scheduled in /var/spool/cron/apache (removed actual IP)

Code: Select all

10 9 * * *      rm -f /usr/local/nagiosxi/html/includes/components/autodiscovery/jobs/73aj2t.xml; touch /usr/local/nagiosxi/html/includes/components/autodiscovery/jobs/73aj2t.watch; sudo /usr/bin/php /usr/local/nagiosxi/html/includes/components/autodiscovery/scripts/autodiscover_new.php --addresses="x.x.x.x/24"  --exclude="x.x.x.x" --output=/usr/local/nagiosxi/html/includes/components/autodiscovery/jobs/73aj2t.xml --watch=/usr/local/nagiosxi/html/includes/components/autodiscovery/jobs/73aj2t.watch --onlynew=0 --debug=1 --detectos=1 --detecttopo=1  --system-dns=1 > /usr/local/nagiosxi/html/includes/components/autodiscovery/jobs/73aj2t.out 2>&1 & echo $! > /dev/null 2>&1

and the only errors I see are the one caused by emails separated by semicolon instead of colon in scheduled reports section.

I tried to restart nagios, reboot server - no change.

[ssax]

Please try running this command and post any output:
- Make sure to add the IPs back in

Code: Select all

su -s /bin/sh apache -c 'rm -f /usr/local/nagiosxi/html/includes/components/autodiscovery/jobs/73aj2t.xml; touch /usr/local/nagiosxi/html/includes/components/autodiscovery/jobs/73aj2t.watch; sudo /usr/bin/php /usr/local/nagiosxi/html/includes/components/autodiscovery/scripts/autodiscover_new.php --addresses="x.x.x.x/24"  --exclude="x.x.x.x" --output=/usr/local/nagiosxi/html/includes/components/autodiscovery/jobs/73aj2t.xml --watch=/usr/local/nagiosxi/html/includes/components/autodiscovery/jobs/73aj2t.watch --onlynew=0 --debug=1 --detectos=1 --detecttopo=1  --system-dns=1 > /usr/local/nagiosxi/html/includes/components/autodiscovery/jobs/73aj2t.out'

[stf_792]

running as root

got few permission denied.

Code: Select all

touch: cannot touch ‘/usr/local/nagiosxi/html/includes/components/autodiscovery/jobs/73aj2t.watch’: Permission denied

Code: Select all

sh: /usr/local/nagiosxi/html/includes/components/autodiscovery/jobs/73aj2t.out: Permission denied

[ssax]

Please run these commads and see if it resolves the issue:

Code: Select all

chown nagios.nagios /usr/local/nagiosxi/html/includes/components/autodiscovery/*
chown apache.apache /usr/local/nagiosxi/html/includes/components/autodiscovery/jobs/*

Let us know the results.

[stf_792]

ran chown, restarted processes, manual u run autodiscovery job, job completed. But did I lost all the history?

Only last ran job now show up in in Job: section. where before you can go back and pick a job from the list.

[ssax]

We may need to dig into the database on this, please send in an email to [email protected] with a descriptive subject and detailed body with a link back to this thread so that we can get a remote session setup.

Thank you

[stf_792]

let me wait 24 hours, so all task have chance to run. Will see what happened after.