Unable to start nagios - no errors

[emartine]

Running on RHEL 6.7 with gearmand2 the nagios web interface is telling me that the monitoring engine is not running. So I attempted to start it by clicking on the play sign and it doesn't spit out any errors. I logged on to the server

Ran this command --> service nagios status
output --> nagios is not running

Ran this command --> service nagios start
output --> Starting nagios: done.


But this is still not running. I verified the configuration and I have 3 warnings but no errors.


Warning: Service 'Memory Usage' on host 'server2' has a notification interval less than its check interval! Notifications are only re-sent after checks are made, so the effective notification interval will be that of the check interval.
Checked 258 services.
Warning: Host 'serveresg2' has no default contacts or contactgroups defined!
Warning: Host 'localhost' has no default contacts or contactgroups defined!


I ran through /var/log/messages and found no errors. Any other place I am missing where I might find an error?

[lmiltchev]

How did you install gearmand2? Did you follow our documentation?

https://assets.nagios.com/downloads/nag ... ios_XI.pdf

Run the following commands and show the output in code wraps:

Code: Select all

service gearmand restart
service nagios restart
grep gearman /usr/local/nagios/etc/nagios.cfg
grep live /usr/local/nagios/etc/nagios.cfg
ps -ef | grep [g]earman
tail /usr/local/nagios/var/nagios.log

[emartine]

# service gearmand restart
Stopping gearmand: [ OK ]
Starting gearmand: [ OK ]

# service nagios restart
Running configuration check...done.
Stopping nagios: /etc/init.d/nagios: line 67: kill: (14721) - No such process
done.
Starting nagios: done.

# grep gearman /usr/local/nagios/etc/nagios.cfg
broker_module=/usr/lib64/mod_gearman2/mod_gearman2.o config=/etc/mod_gearman2/module.conf eventhandler=no

# grep live /usr/local/nagios/etc/nagios.cfg

# ps -ef | grep [g]earman
nagios 8343 1 0 10:29 ? 00:00:00 /usr/bin/mod_gearman2_worker -d --config=/etc/mod_gearman2/worker.conf --pidfile=/var/mod_gearman2/mod_gearman_worker.pid
nagios 10849 8343 0 14:51 ? 00:00:00 /usr/bin/mod_gearman2_worker -d --config=/etc/mod_gearman2/worker.conf --pidfile=/var/mod_gearman2/mod_gearman_worker.pid
nagios 18488 8343 0 15:11 ? 00:00:00 /usr/bin/mod_gearman2_worker -d --config=/etc/mod_gearman2/worker.conf --pidfile=/var/mod_gearman2/mod_gearman_worker.pid
nagios 20967 8343 0 15:17 ? 00:00:00 /usr/bin/mod_gearman2_worker -d --config=/etc/mod_gearman2/worker.conf --pidfile=/var/mod_gearman2/mod_gearman_worker.pid
nagios 23222 8343 0 15:23 ? 00:00:00 /usr/bin/mod_gearman2_worker -d --config=/etc/mod_gearman2/worker.conf --pidfile=/var/mod_gearman2/mod_gearman_worker.pid
nagios 26630 8343 0 15:32 ? 00:00:00 /usr/bin/mod_gearman2_worker -d --config=/etc/mod_gearman2/worker.conf --pidfile=/var/mod_gearman2/mod_gearman_worker.pid
nagios 30434 8343 0 15:42 ? 00:00:00 /usr/bin/mod_gearman2_worker -d --config=/etc/mod_gearman2/worker.conf --pidfile=/var/mod_gearman2/mod_gearman_worker.pid
nagios 31081 8343 0 15:44 ? 00:00:00 /usr/bin/mod_gearman2_worker -d --config=/etc/mod_gearman2/worker.conf --pidfile=/var/mod_gearman2/mod_gearman_worker.pid
gearmand 31811 1 0 15:46 ? 00:00:00 /usr/sbin/gearmand -d --worker-wakeup=10 --retention-file=/tmp/gearmand.retention -q retention --log-file=/var/log/gearmand/gearmand.log
nagios 31999 8343 0 15:46 ? 00:00:00 /usr/bin/mod_gearman2_worker -d --config=/etc/mod_gearman2/worker.conf --pidfile=/var/mod_gearman2/mod_gearman_worker.pid


# tail /usr/local/nagios/var/nagios.log
[1460061988] ndomod registered for contact notification data'
[1460061988] ndomod registered for acknowledgement data'
[1460061988] ndomod registered for state change data'
[1460061988] ndomod registered for contact status data'
[1460061988] ndomod registered for adaptive contact data'
[1460061988] Event broker module '/usr/local/nagios/bin/ndomod.o' initialized successfully.
[1460061988] Warning: Service 'Memory Usage' on host 'server2' has a notification interval less than its check interval! Notifications are only re-sent after checks are made, so the effective notification interval will be that of the check interval.
[1460061988] Warning: Host 'server1' has no default contacts or contactgroups defined!
[1460061988] Warning: Host 'localhost' has no default contacts or contactgroups defined!
[1460061988] Successfully launched command file worker with pid 31878

[lmiltchev]

Does nagios start when you comment out the gearman broker module line?

Code: Select all

# broker_module=/usr/lib64/mod_gearman2/mod_gearman2.o config=/etc/mod_gearman2/module.conf eventhandler=no

Code: Select all

service nagios stop
killall nagios
service nagios start

You didn't tell us how you installed gearmand2. Did you follow our documentation?

[emartine]

I followed the nagios documentation to install it.

By the way there is another broker line in nagios.cfg:

broker_module=/usr/local/nagios/bin/ndomod.o config_file=/usr/local/nagios/etc/ndomod.cfg

Commenting the gearman broker line starts it fine.

]# service nagios start
Starting nagios: done.
]# service nagios status
nagios (pid 4682) is running...


Oddly enough even after uncommenting the broker line nagios comes up. I didn't see any nagios.cfg process running before. Kill all nagios process command didn't do anything.
Note that now that I have it running I am experiencing the same problem I have on another system where I am unable to submit a command command through the web.

[bheden]

In /etc/mod_gearman2/module.conf AND /etc/mod_gearman2/worker.conf files can you change the line

Code: Select all

debug=0

to

Code: Select all

debug=1

Then:

Code: Select all

service gearmand restart
service nagios restart
service mod-gearman2-worker restart

Finally, can you show us the output (after performing those steps) from the following:

Code: Select all

cat /usr/local/nagios/var/nagios.log | grep gearman
cat /var/log/mod_gearman2/mod_gearman_neb.log
cat /var/log/mod_gearman_worker.log

[emartine]

Code: Select all

cat /usr/local/nagios/var/nagios.log | grep gearman
[1460126741] Event broker module '/usr/lib64/mod_gearman2/mod_gearman2.o' deinitialized successfully.
[1460126742] mod_gearman: initialized version 2.1.1 (libgearman 0.33)
[1460126742] Event broker module '/usr/lib64/mod_gearman2/mod_gearman2.o' initialized successfully.
[1460130362] Event broker module '/usr/lib64/mod_gearman2/mod_gearman2.o' deinitialized successfully.
[1460130363] mod_gearman: initialized version 2.1.1 (libgearman 0.33)
[1460130363] Event broker module '/usr/lib64/mod_gearman2/mod_gearman2.o' initialized successfully.
[1460130521] Event broker module '/usr/lib64/mod_gearman2/mod_gearman2.o' deinitialized successfully.
[1460130522] mod_gearman: initialized version 2.1.1 (libgearman 0.33)
[1460130522] Event broker module '/usr/lib64/mod_gearman2/mod_gearman2.o' initialized successfully.
[1460130711] Event broker module '/usr/lib64/mod_gearman2/mod_gearman2.o' deinitialized successfully.
[1460130712] mod_gearman: initialized version 2.1.1 (libgearman 0.33)
[1460130712] Event broker module '/usr/lib64/mod_gearman2/mod_gearman2.o' initialized successfully.
[1460131130] Event broker module '/usr/lib64/mod_gearman2/mod_gearman2.o' deinitialized successfully.
[1460131131] mod_gearman: initialized version 2.1.1 (libgearman 0.33)
[1460131131] Event broker module '/usr/lib64/mod_gearman2/mod_gearman2.o' initialized successfully.

[emartine]

Code: Select all

cat /var/log/mod_gearman2/mod_gearman_neb.log 
[2016-04-08 10:58:51][4309][DEBUG] --------------------------------
[2016-04-08 10:58:51][4309][DEBUG] configuration:
[2016-04-08 10:58:51][4309][DEBUG] log level:                       1
[2016-04-08 10:58:51][4309][DEBUG] log mode:                        file (1)
[2016-04-08 10:58:51][4309][DEBUG] queue by cust var:               no
[2016-04-08 10:58:51][4309][DEBUG] debug result:                    no
[2016-04-08 10:58:51][4309][DEBUG] result_worker:                   1
[2016-04-08 10:58:51][4309][DEBUG] do_hostchecks:                   yes
[2016-04-08 10:58:51][4309][DEBUG] route_eventhandler_like_checks:  no
[2016-04-08 10:58:51][4309][DEBUG] result_queue:                    check_results
[2016-04-08 10:58:51][4309][DEBUG]
[2016-04-08 10:58:51][4309][DEBUG] server:                          localhost:4730
[2016-04-08 10:58:51][4309][DEBUG]
[2016-04-08 10:58:51][4309][DEBUG]
[2016-04-08 10:58:51][4309][DEBUG] perfdata:                        no
[2016-04-08 10:58:51][4309][DEBUG] perfdata mode:                   overwrite
[2016-04-08 10:58:51][4309][DEBUG] hosts:                           yes
[2016-04-08 10:58:51][4309][DEBUG] services:                        yes
[2016-04-08 10:58:51][4309][DEBUG] eventhandler:                    no
[2016-04-08 10:58:51][4309][DEBUG]
[2016-04-08 10:58:51][4309][DEBUG] encryption:                      yes
[2016-04-08 10:58:51][4309][DEBUG] keyfile:                         no
[2016-04-08 10:58:51][4309][DEBUG] encryption key:                  set
[2016-04-08 10:58:51][4309][DEBUG] accept clear result:             no
[2016-04-08 10:58:51][4309][DEBUG] transport mode:                  aes-256+base64
[2016-04-08 10:58:51][4309][DEBUG] use uniq jobs:                   yes
[2016-04-08 10:58:51][4309][DEBUG] --------------------------------
[2016-04-08 10:58:51][4309][DEBUG] finished initializing
[2016-04-08 10:58:51][4309][DEBUG] registered neb callbacks

after 10:58 it looks the same as below.

......
cat /var/log/mod_gearman2/mod_gearman_neb.log
[2016-04-08 11:06:05][6125][DEBUG] received job for queue service: test9x5server - CPU Load
[2016-04-08 11:06:05][6125][DEBUG] service: 'test9x5server' - 'CPU Load', next_check is at 2016-04-08 11:06:05, latency so far: 0
[2016-04-08 11:06:05][6125][DEBUG] service job completed: test9x5server CPU Load: 2
[2016-04-08 11:06:06][6125][DEBUG] received job for queue service: nrpe9x5checkperiod - Total Processes
[2016-04-08 11:06:06][6125][DEBUG] service: 'nrpe9x5checkperiod' - 'Total Processes', next_check is at 2016-04-08 11:06:06, latency so far: 0
[2016-04-08 11:06:06][6125][DEBUG] service job completed: nrpe9x5checkperiod Total Processes: 1
[2016-04-08 11:06:07][6125][DEBUG] received job for queue service: server44- Uptime
[2016-04-08 11:06:07][6125][DEBUG] service: server55- 'Uptime', next_check is at 2016-04-08 11:06:07, latency so far: 0
[2016-04-08 11:06:07][6125][DEBUG] service job completed: server44Uptime: 0
[2016-04-08 11:06:08][6125][DEBUG] received job for queue service: server33 - DNS IP Match
[2016-04-08 11:06:08][6125][DEBUG] service: 'server33' - 'DNS IP Match', next_check is at 2016-04-08 11:06:08, latency so far: 0
[2016-04-08 11:06:08][6125][DEBUG] service job completed: server33 DNS IP Match: 0
[2016-04-08 11:06:09][6125][DEBUG] received job for queue service: server44- MountPoint - M:Star_ReportLog
[2016-04-08 11:06:09][6125][DEBUG] service: server55- 'MountPoint - M:Star_ReportLog', next_check is at 2016-04-08 11:06:09, latency so far: 0
[2016-04-08 11:06:09][6125][DEBUG] service job completed: server44MountPoint - M:Star_ReportLog: 0
[2016-04-08 11:06:10][6125][DEBUG] received job for queue service: server2 - Exchange_RPC_User_Count
[2016-04-08 11:06:10][6125][DEBUG] service: 'server2' - 'Exchange_RPC_User_Count', next_check is at 2016-04-08 11:06:10, latency so far: 0
[2016-04-08 11:06:10][6125][DEBUG] service job completed: server2 Exchange_RPC_User_Count: 3
[2016-04-08 11:06:11][6125][DEBUG] received job for queue service: test2 - Current Load
[2016-04-08 11:06:11][6125][DEBUG] service: 'test2' - 'Current Load', next_check is at 2016-04-08 11:06:11, latency so far: 0
[2016-04-08 11:06:11][6125][DEBUG] service job completed: test2 Current Load: 2
[2016-04-08 11:06:12][6125][DEBUG] received job for queue service: hamster - Eventlog
[2016-04-08 11:06:12][6125][DEBUG] service: 'hamster' - 'Eventlog', next_check is at 2016-04-08 11:06:12, latency so far: 0
[2016-04-08 11:06:12][6125][DEBUG] service job completed: hamster Eventlog: 0
[2016-04-08 11:06:13][6125][DEBUG] received job for queue service: test4 - Current Users
[2016-04-08 11:06:13][6125][DEBUG] service: 'test4' - 'Current Users', next_check is at 2016-04-08 11:06:13, latency so far: 0
[2016-04-08 11:06:14][6125][DEBUG] received job for queue service: test66 - CPU Load
[2016-04-08 11:06:14][6125][DEBUG] service: 'test66' - 'CPU Load', next_check is at 2016-04-08 11:06:14, latency so far: 0
[2016-04-08 11:06:14][6125][DEBUG] service job completed: test66 CPU Load: 0
[2016-04-08 11:06:14][6125][DEBUG] received job for queue host: server2
[2016-04-08 11:06:14][6125][DEBUG] host: 'server2', next_check is at 2016-04-08 11:06:14, latency so far: 0
[2016-04-08 11:06:14][6125][DEBUG] received job for queue host: nrpe9x5checkperiod
[2016-04-08 11:06:14][6125][DEBUG] host: 'nrpe9x5checkperiod', next_check is at 2016-04-08 11:06:14, latency so far: 0
[2016-04-08 11:06:14][6125][DEBUG] received job for queue host: test9x5server
[2016-04-08 11:06:14][6125][DEBUG] host: 'test9x5server', next_check is at 2016-04-08 11:06:14, latency so far: 0
[2016-04-08 11:06:14][6125][DEBUG] host job completed: test9x5server: 3
[2016-04-08 11:06:14][6125][DEBUG] host job completed: server2: 0
[2016-04-08 11:06:15][6125][DEBUG] host job completed: nrpe9x5checkperiod: 3
[2016-04-08 11:06:15][6125][DEBUG] received job for queue service: nrpe9x5checkperiod - File System Space
[2016-04-08 11:06:15][6125][DEBUG] service: 'nrpe9x5checkperiod' - 'File System Space', next_check is at 2016-04-08 11:06:15, latency so far: 0
[2016-04-08 11:06:16][6125][DEBUG] service job completed: nrpe9x5checkperiod File System Space: 1
[2016-04-08 11:06:16][6125][DEBUG] service job completed: test4 Current Users: 2
[2016-04-08 11:06:16][6125][DEBUG] received job for queue service: server2 - Exchange Connection Count
[2016-04-08 11:06:16][6125][DEBUG] service: 'server2' - 'Exchange Connection Count', next_check is at 2016-04-08 11:06:16, latency so far: 0
[2016-04-08 11:06:17][6125][DEBUG] service job completed: server2 Exchange Connection Count: 3
[2016-04-08 11:06:17][6125][DEBUG] received job for queue service: localhost - ExternalCommandsUsed 1mn
[2016-04-08 11:06:17][6125][DEBUG] service: 'localhost' - 'ExternalCommandsUsed 1mn', next_check is at 2016-04-08 11:06:18, latency so far: -1
[2016-04-08 11:06:18][6125][DEBUG] service job completed: localhost ExternalCommandsUsed 1mn: 0
[2016-04-08 11:06:19][6125][DEBUG] received job for queue service: ARCHITECT - Ping
[2016-04-08 11:06:19][6125][DEBUG] service: 'ARCHITECT' - 'Ping', next_check is at 2016-04-08 11:06:19, latency so far: 0
[2016-04-08 11:06:20][6125][DEBUG] received job for queue service: server2 - Active_Virtual_Memory_in_MB
[2016-04-08 11:06:20][6125][DEBUG] service: 'server2' - 'Active_Virtual_Memory_in_MB', next_check is at 2016-04-08 11:06:20, latency so far: 0
[2016-04-08 11:06:20][6125][DEBUG] service job completed: server2 Active_Virtual_Memory_in_MB: 0
[2016-04-08 11:06:21][6125][DEBUG] received job for queue service: server44- IIS World Wide Web Publishing Service
[2016-04-08 11:06:21][6125][DEBUG] service: server55- 'IIS World Wide Web Publishing Service', next_check is at 2016-04-08 11:06:21, latency so far: 0
[2016-04-08 11:06:21][6125][DEBUG] service job completed: server44IIS World Wide Web Publishing Service: 3
[2016-04-08 11:06:22][6125][DEBUG] received job for queue service: AliasLongFixingTester - Current Load
[2016-04-08 11:06:22][6125][DEBUG] service: 'AliasLongFixingTester' - 'Current Load', next_check is at 2016-04-08 11:06:22, latency so far: 0
[2016-04-08 11:06:22][6125][DEBUG] service job completed: AliasLongFixingTester Current Load: 1
[2016-04-08 11:06:23][6125][DEBUG] received job for queue service: server44- Memory Usage
[2016-04-08 11:06:23][6125][DEBUG] service: server55- 'Memory Usage', next_check is at 2016-04-08 11:06:23, latency so far: 0
[2016-04-08 11:06:23][6125][DEBUG] service job completed: server44Memory Usage: 0
[2016-04-08 11:06:24][6125][DEBUG] received job for queue host: AliasLongFixingTester
[2016-04-08 11:06:24][6125][DEBUG] host: 'AliasLongFixingTester', next_check is at 2016-04-08 11:06:24, latency so far: 0
[2016-04-08 11:06:24][6125][DEBUG] received job for queue host: test88
[2016-04-08 11:06:24][6125][DEBUG] host: 'test88', next_check is at 2016-04-08 11:06:24, latency so far: 0
[2016-04-08 11:06:24][6125][DEBUG] received job for queue host: test4
[2016-04-08 11:06:24][6125][DEBUG] host: 'test4', next_check is at 2016-04-08 11:06:24, latency so far: 0
[2016-04-08 11:06:24][6125][DEBUG] host job completed: test88: 0
[2016-04-08 11:06:24][6125][DEBUG] received job for queue host: test99
[2016-04-08 11:06:24][6125][DEBUG] host: 'test99', next_check is at 2016-04-08 11:06:24, latency so far: 0
[2016-04-08 11:06:25][6125][DEBUG] host job completed: AliasLongFixingTester: 3
[2016-04-08 11:06:25][6125][DEBUG] received job for queue service: server2 - Memory Usage
[2016-04-08 11:06:25][6125][DEBUG] service: 'server2' - 'Memory Usage', next_check is at 2016-04-08 11:06:25, latency so far: 0
[2016-04-08 11:06:25][6125][DEBUG] service job completed: server2 Memory Usage: 0
[2016-04-08 11:06:26][6125][DEBUG] received job for queue service: test2 - Current Users
[2016-04-08 11:06:26][6125][DEBUG] service: 'test2' - 'Current Users', next_check is at 2016-04-08 11:06:26, latency so far: 0
[2016-04-08 11:06:26][6125][DEBUG] service job completed: test2 Current Users: 2
[2016-04-08 11:06:27][6125][DEBUG] received job for queue service: server44- Disk - C
[2016-04-08 11:06:27][6125][DEBUG] service: server55- 'Disk - C', next_check is at 2016-04-08 11:06:27, latency so far: 0
[2016-04-08 11:06:27][6125][DEBUG] service job completed: server44Disk - C: 0
[2016-04-08 11:06:27][6125][DEBUG] host job completed: test4: 2
[2016-04-08 11:06:28][6125][DEBUG] received job for queue service: server2 - RPC_HTTP_Connection_Count
[2016-04-08 11:06:28][6125][DEBUG] service: 'server2' - 'RPC_HTTP_Connection_Count', next_check is at 2016-04-08 11:06:28, latency so far: 0
[2016-04-08 11:06:28][6125][DEBUG] service job completed: server2 RPC_HTTP_Connection_Count: 3
[2016-04-08 11:06:29][6125][DEBUG] service job completed: ARCHITECT Ping: 2
[2016-04-08 11:06:29][6125][DEBUG] received job for queue service: localhost - ActiveHostChecks 1mn
[2016-04-08 11:06:29][6125][DEBUG] service: 'localhost' - 'ActiveHostChecks 1mn', next_check is at 2016-04-08 11:06:29, latency so far: 0
[2016-04-08 11:06:29][6125][DEBUG] service job completed: localhost ActiveHostChecks 1mn: 0
[2016-04-08 11:06:30][6125][DEBUG] received job for queue service: nrpe9x5checkperiod - Swap
[2016-04-08 11:06:30][6125][DEBUG] service: 'nrpe9x5checkperiod' - 'Swap', next_check is at 2016-04-08 11:06:30, latency so far: 0
[2016-04-08 11:06:30][6125][DEBUG] service job completed: nrpe9x5checkperiod Swap: 1
[2016-04-08 11:06:31][6125][DEBUG] received job for queue service: nrpe9x5checkperiod - Current Load
[2016-04-08 11:06:31][6125][DEBUG] service: 'nrpe9x5checkperiod' - 'Current Load', next_check is at 2016-04-08 11:06:31, latency so far: 0
[2016-04-08 11:06:31][6125][DEBUG] service job completed: nrpe9x5checkperiod Current Load: 1
[2016-04-08 11:06:32][6125][DEBUG] received job for queue service: localhost - ActiveServiceChecks 1mn
[2016-04-08 11:06:32][6125][DEBUG] service: 'localhost' - 'ActiveServiceChecks 1mn', next_check is at 2016-04-08 11:06:32, latency so far: 0
[2016-04-08 11:06:32][6125][DEBUG] service job completed: localhost ActiveServiceChecks 1mn: 0
[2016-04-08 11:06:33][6125][DEBUG] received job for queue service: nagiosxi - Current Load
[2016-04-08 11:06:33][6125][DEBUG] service: 'nagiosxi' - 'Current Load', next_check is at 2016-04-08 11:06:33, latency so far: 0
[2016-04-08 11:06:33][6125][DEBUG] service job completed: nagiosxi Current Load: 2
[2016-04-08 11:06:34][6125][DEBUG] received job for queue service: localhost - PassiveHostChecks 1mn
[2016-04-08 11:06:34][6125][DEBUG] service: 'localhost' - 'PassiveHostChecks 1mn', next_check is at 2016-04-08 11:06:34, latency so far: 0
[2016-04-08 11:06:34][6125][DEBUG] service job completed: localhost PassiveHostChecks 1mn: 0
[2016-04-08 11:06:34][6125][DEBUG] received job for queue host: nagiosxi
[2016-04-08 11:06:34][6125][DEBUG] host: 'nagiosxi', next_check is at 2016-04-08 11:06:34, latency so far: 0
[2016-04-08 11:06:34][6125][DEBUG] received job for queue host: nrpe9x5checkperiod
[2016-04-08 11:06:34][6125][DEBUG] host: 'nrpe9x5checkperiod', next_check is at 2016-04-08 11:06:34, latency so far: 0
[2016-04-08 11:06:34][6125][DEBUG] received job for queue host: ARCHITECT
[2016-04-08 11:06:34][6125][DEBUG] host: 'ARCHITECT', next_check is at 2016-04-08 11:06:34, latency so far: 0
[2016-04-08 11:06:34][6125][DEBUG] received job for queue host: server2
[2016-04-08 11:06:34][6125][DEBUG] host: 'server2', next_check is at 2016-04-08 11:06:34, latency so far: 0
[2016-04-08 11:06:34][6125][DEBUG] received job for queue host: test2
[2016-04-08 11:06:34][6125][DEBUG] host: 'test2', next_check is at 2016-04-08 11:06:34, latency so far: 0
[2016-04-08 11:06:34][6125][DEBUG] host job completed: nagiosxi: 0
[2016-04-08 11:06:34][6125][DEBUG] host job completed: test2: 0
[2016-04-08 11:06:34][6125][DEBUG] host job completed: server2: 0
[2016-04-08 11:06:35][6125][DEBUG] host job completed: test99: 2
[2016-04-08 11:06:35][6125][DEBUG] host job completed: nrpe9x5checkperiod: 3
[2016-04-08 11:06:35][6125][DEBUG] received job for queue service: server44- Eventlog
[2016-04-08 11:06:35][6125][DEBUG] service: server55- 'Eventlog', next_check is at 2016-04-08 11:06:35, latency so far: 0
[2016-04-08 11:06:35][6125][DEBUG] service job completed: server44Eventlog: 0
[2016-04-08 11:06:36][6125][DEBUG] received job for queue service: test99 - Uptime
[2016-04-08 11:06:36][6125][DEBUG] service: 'test99' - 'Uptime', next_check is at 2016-04-08 11:06:36, latency so far: 0
[2016-04-08 11:06:37][6125][DEBUG] received job for queue service: serverrr - Current Load
[2016-04-08 11:06:37][6125][DEBUG] service: 'serverrr' - 'Current Load', next_check is at 2016-04-08 11:06:37, latency so far: 0
[2016-04-08 11:06:37][6125][DEBUG] service job completed: serverrr Current Load: 0
[2016-04-08 11:06:38][6125][DEBUG] received job for queue service: localhost - PassiveServiceChecks 1mn
[2016-04-08 11:06:38][6125][DEBUG] service: 'localhost' - 'PassiveServiceChecks 1mn', next_check is at 2016-04-08 11:06:38, latency so far: 0
[2016-04-08 11:06:38][6125][DEBUG] service job completed: localhost PassiveServiceChecks 1mn: 0
[2016-04-08 11:06:39][6125][DEBUG] received job for queue service: nagiosxi - Current Users
[2016-04-08 11:06:39][6125][DEBUG] service: 'nagiosxi' - 'Current Users', next_check is at 2016-04-08 11:06:39, latency so far: 0
[2016-04-08 11:06:39][6125][DEBUG] service job completed: nagiosxi Current Users: 2
[2016-04-08 11:06:40][6125][DEBUG] received job for queue host: test66
[2016-04-08 11:06:40][6125][DEBUG] host: 'test66', next_check is at 2016-04-08 11:06:40, latency so far: 0
[2016-04-08 11:06:40][6125][DEBUG] host job completed: test66: 0
[2016-04-08 11:06:41][6125][DEBUG] received job for queue service: hamster - Ping
[2016-04-08 11:06:41][6125][DEBUG] service: 'hamster' - 'Ping', next_check is at 2016-04-08 11:06:41, latency so far: 0
[2016-04-08 11:06:41][6125][DEBUG] service job completed: hamster Ping: 0
[2016-04-08 11:06:42][6125][DEBUG] received job for queue service: nrpe9x5checkperiod - Ping
[2016-04-08 11:06:42][6125][DEBUG] service: 'nrpe9x5checkperiod' - 'Ping', next_check is at 2016-04-08 11:06:42, latency so far: 0
[2016-04-08 11:06:43][6125][DEBUG] service job completed: nrpe9x5checkperiod Ping: 3
[2016-04-08 11:06:43][6125][DEBUG] received job for queue service: test99 - Memory Usage
[2016-04-08 11:06:43][6125][DEBUG] service: 'test99' - 'Memory Usage', next_check is at 2016-04-08 11:06:43, latency so far: 0
[2016-04-08 11:06:44][6125][DEBUG] received job for queue service: dragon - CPU Load
[2016-04-08 11:06:44][6125][DEBUG] service: 'dragon' - 'CPU Load', next_check is at 2016-04-08 11:06:44, latency so far: 0
[2016-04-08 11:06:44][6125][DEBUG] host job completed: ARCHITECT: 2
[2016-04-08 11:06:45][6125][DEBUG] received job for queue service: testchanger- Processor_Timing
[2016-04-08 11:06:45][6125][DEBUG] service: 'testchanger' - 'Processor_Timing', next_check is at 2016-04-08 11:06:45, latency so far: 0
[2016-04-08 11:06:46][6125][DEBUG] service job completed: test99 Uptime: 2
[2016-04-08 11:06:46][6125][DEBUG] received job for queue service: server44- SQL Server Agent
[2016-04-08 11:06:46][6125][DEBUG] service: server55- 'SQL Server Agent', next_check is at 2016-04-08 11:06:47, latency so far: -1
[2016-04-08 11:06:46][6125][DEBUG] service job completed: testchangerProcessor_Timing: 0
[2016-04-08 11:06:46][6125][DEBUG] service job completed: server44SQL Server Agent: 0
[2016-04-08 11:06:48][6125][DEBUG] received job for queue service: hamster - Check log3
[2016-04-08 11:06:48][6125][DEBUG] service: 'hamster' - 'Check log3', next_check is at 2016-04-08 11:06:48, latency so far: 0
[2016-04-08 11:06:48][6125][DEBUG] service job completed: hamster Check log3: 2
[2016-04-08 11:06:49][6125][DEBUG] received job for queue service: serverrr - Swap
[2016-04-08 11:06:49][6125][DEBUG] service: 'serverrr' - 'Swap', next_check is at 2016-04-08 11:06:49, latency so far: 0
[2016-04-08 11:06:49][6125][DEBUG] service job completed: serverrr Swap: 0
[2016-04-08 11:06:50][6125][DEBUG] received job for queue service: hamster - Memory Physical RAM Usage
[2016-04-08 11:06:50][6125][DEBUG] service: 'hamster' - 'Memory Physical RAM Usage', next_check is at 2016-04-08 11:06:50, latency so far: 0
[2016-04-08 11:06:50][6125][DEBUG] service job completed: hamster Memory Physical RAM Usage: 0
[2016-04-08 11:06:51][6125][DEBUG] received job for queue service: server2 - HTTP_PROXY_Unique_Users
[2016-04-08 11:06:51][6125][DEBUG] service: 'server2' - 'HTTP_PROXY_Unique_Users', next_check is at 2016-04-08 11:06:51, latency so far: 0
[2016-04-08 11:06:51][6125][DEBUG] service job completed: server2 HTTP_PROXY_Unique_Users: 3
[2016-04-08 11:06:53][6125][DEBUG] received job for queue service: test2 - Ping
[2016-04-08 11:06:53][6125][DEBUG] service: 'test2' - 'Ping', next_check is at 2016-04-08 11:06:53, latency so far: 0
[2016-04-08 11:06:53][6125][DEBUG] service job completed: test2 Ping: 0
[2016-04-08 11:06:53][6125][DEBUG] service job completed: test99 Memory Usage: 2
[2016-04-08 11:06:54][6125][DEBUG] received job for queue service: test66 - Disk - C
[2016-04-08 11:06:54][6125][DEBUG] service: 'test66' - 'Disk - C', next_check is at 2016-04-08 11:06:54, latency so far: 0
[2016-04-08 11:06:54][6125][DEBUG] service job completed: test66 Disk - C: 0
[2016-04-08 11:06:54][6125][DEBUG] service job completed: dragon CPU Load: 2
[2016-04-08 11:06:54][6125][DEBUG] received job for queue service: testchanger- HTTP_PROXY_Unique_Users
[2016-04-08 11:06:54][6125][DEBUG] service: 'testchanger' - 'HTTP_PROXY_Unique_Users', next_check is at 2016-04-08 11:06:54, latency so far: 0
[2016-04-08 11:06:54][6125][DEBUG] received job for queue host: dragon
[2016-04-08 11:06:54][6125][DEBUG] host: 'dragon', next_check is at 2016-04-08 11:06:54, latency so far: 0
[2016-04-08 11:06:54][6125][DEBUG] received job for queue host: test99
[2016-04-08 11:06:54][6125][DEBUG] host: 'test99', next_check is at 2016-04-08 11:06:54, latency so far: 0
[2016-04-08 11:06:54][6125][DEBUG] received job for queue host: server2
[2016-04-08 11:06:54][6125][DEBUG] host: 'server2', next_check is at 2016-04-08 11:06:54, latency so far: 0
[2016-04-08 11:06:54][6125][DEBUG] received job for queue host: hamster
[2016-04-08 11:06:54][6125][DEBUG] host: 'hamster', next_check is at 2016-04-08 11:06:54, latency so far: 0
[2016-04-08 11:06:54][6125][DEBUG] host job completed: dragon: 0
[2016-04-08 11:06:54][6125][DEBUG] service job completed: testchangerHTTP_PROXY_Unique_Users: 0
[2016-04-08 11:06:55][6125][DEBUG] host job completed: server2: 0
[2016-04-08 11:06:55][6125][DEBUG] host job completed: hamster: 0
[2016-04-08 11:06:55][6125][DEBUG] received job for queue service: sharechanger- Multi Address Ping
[2016-04-08 11:06:55][6125][DEBUG] service: 'sharechanger' - 'Multi Address Ping', next_check is at 2016-04-08 11:06:55, latency so far: 0
[2016-04-08 11:06:56][6125][DEBUG] received job for queue service: nagiosxi - Total Processes
[2016-04-08 11:06:56][6125][DEBUG] service: 'nagiosxi' - 'Total Processes', next_check is at 2016-04-08 11:06:56, latency so far: 0
[2016-04-08 11:06:56][6125][DEBUG] service job completed: nagiosxi Total Processes: 2
[2016-04-08 11:06:57][6125][DEBUG] received job for queue service: test9x5server - Ping
[2016-04-08 11:06:57][6125][DEBUG] service: 'test9x5server' - 'Ping', next_check is at 2016-04-08 11:06:57, latency so far: 0
[2016-04-08 11:06:57][6125][DEBUG] service job completed: test9x5server Ping: 3
[2016-04-08 11:06:58][6125][DEBUG] received job for queue service: server2 - Uptime
[2016-04-08 11:06:58][6125][DEBUG] service: 'server2' - 'Uptime', next_check is at 2016-04-08 11:06:58, latency so far: 0
[2016-04-08 11:06:58][6125][DEBUG] service job completed: server2 Uptime: 0
[2016-04-08 11:06:59][6125][DEBUG] received job for queue service: hamster - Disk - C
[2016-04-08 11:06:59][6125][DEBUG] service: 'hamster' - 'Disk - C', next_check is at 2016-04-08 11:06:59, latency so far: 0
[2016-04-08 11:06:59][6125][DEBUG] service job completed: hamster Disk - C: 0
[2016-04-08 11:07:00][6125][DEBUG] received job for queue service: serverrr - Total Processes
[2016-04-08 11:07:00][6125][DEBUG] service: 'serverrr' - 'Total Processes', next_check is at 2016-04-08 11:07:00, latency so far: 0
[2016-04-08 11:07:00][6125][DEBUG] service job completed: serverrr Total Processes: 0
[2016-04-08 11:07:01][6125][DEBUG] received job for queue service: test2 - Total Processes
[2016-04-08 11:07:01][6125][DEBUG] service: 'test2' - 'Total Processes', next_check is at 2016-04-08 11:07:01, latency so far: 0
[2016-04-08 11:07:01][6125][DEBUG] service job completed: test2 Total Processes: 2
[2016-04-08 11:07:01][6125][DEBUG] service job completed: sharechangerMulti Address Ping: 0
[2016-04-08 11:07:02][6125][DEBUG] received job for queue service: testchanger- Exchange_RPC_Average_Latency
[2016-04-08 11:07:02][6125][DEBUG] service: 'testchanger' - 'Exchange_RPC_Average_Latency', next_check is at 2016-04-08 11:07:02, latency so far: 0
[2016-04-08 11:07:02][6125][DEBUG] service job completed: testchangerExchange_RPC_Average_Latency: 0

[emartine]

Code: Select all

cat /var/log/mod_gearman2/mod_gearman_worker.log | grep 2016-04-08
[2016-04-08 10:58:51][12318][ERROR] worker error: flush(Broken pipe) lost connection to server during send -> libgearman/connection.cc:761
[2016-04-08 10:59:01][8343][INFO ] mod_gearman worker exited
[2016-04-08 10:59:02][4515][DEBUG] --------------------------------
[2016-04-08 10:59:02][4515][DEBUG] configuration:
[2016-04-08 10:59:02][4515][DEBUG] log level:                       1
[2016-04-08 10:59:02][4515][DEBUG] log mode:                        file (1)
[2016-04-08 10:59:02][4515][DEBUG] identifier:                      <NAGIOSTESTHOSTFQDN>
[2016-04-08 10:59:02][4515][DEBUG] pidfile:                         /var/mod_gearman2/mod_gearman_worker.pid
[2016-04-08 10:59:02][4515][DEBUG] logfile:                         /var/log/mod_gearman2/mod_gearman_worker.log
[2016-04-08 10:59:02][4515][DEBUG] job max num:                     1000
[2016-04-08 10:59:02][4515][DEBUG] job max age:                     0
[2016-04-08 10:59:02][4515][DEBUG] job timeout:                     60
[2016-04-08 10:59:02][4515][DEBUG] min worker:                      5
[2016-04-08 10:59:02][4515][DEBUG] max worker:                      50
[2016-04-08 10:59:02][4515][DEBUG] spawn rate:                      1
[2016-04-08 10:59:02][4515][DEBUG] fork on exec:                    no
[2016-04-08 10:59:02][4515][DEBUG]
[2016-04-08 10:59:02][4515][DEBUG] embedded perl:                   yes
[2016-04-08 10:59:02][4515][DEBUG] use_epn_implicitly:              no
[2016-04-08 10:59:02][4515][DEBUG] use_perl_cache:                  yes
[2016-04-08 10:59:02][4515][DEBUG] p1_file:                         /usr/share/mod_gearman2/mod_gearman_p1.pl
[2016-04-08 10:59:02][4515][DEBUG]
[2016-04-08 10:59:02][4515][DEBUG] server:                          localhost:4730
[2016-04-08 10:59:02][4515][DEBUG]
[2016-04-08 10:59:02][4515][DEBUG]
[2016-04-08 10:59:02][4515][DEBUG] hosts:                           yes
[2016-04-08 10:59:02][4515][DEBUG] services:                        yes
[2016-04-08 10:59:02][4515][DEBUG] eventhandler:                    yes
[2016-04-08 10:59:02][4515][DEBUG]
[2016-04-08 10:59:02][4515][DEBUG] encryption:                      yes
[2016-04-08 10:59:02][4515][DEBUG] keyfile:                         no
[2016-04-08 10:59:02][4515][DEBUG] encryption key:                  set
[2016-04-08 10:59:02][4515][DEBUG] transport mode:                  aes-256+base64
[2016-04-08 10:59:02][4515][DEBUG] use uniq jobs:                   yes
[2016-04-08 10:59:02][4515][DEBUG] --------------------------------
[2016-04-08 10:59:02][4534][INFO ] mod_gearman worker daemon started with pid 4534
[2016-04-08 10:59:02][4534][DEBUG] Version 2.1.1
[2016-04-08 10:59:02][4534][DEBUG] running on libgearman 0.33
[2016-04-08 10:59:02][4534][DEBUG] pid file /var/mod_gearman2/mod_gearman_worker.pid written
[2016-04-08 10:59:02][4534][DEBUG] main process started
[2016-04-08 10:59:02][4539][DEBUG] child started with pid: 4539
[2016-04-08 10:59:02][4538][DEBUG] child started with pid: 4538
[2016-04-08 10:59:02][4540][DEBUG] child started with pid: 4540
[2016-04-08 10:59:02][4537][DEBUG] child started with pid: 4537
[2016-04-08 10:59:02][4536][DEBUG] child started with pid: 4536
[2016-04-08 10:59:02][4535][DEBUG] child started with pid: 4535
[2016-04-08 10:59:03][4538][DEBUG] got service job: citrixtest9x5period - Citrix Services Manager Service
[2016-04-08 10:59:04][4540][DEBUG] got service job: sharechanger - Eventlog
[2016-04-08 10:59:05][4537][DEBUG] got service job: sharechanger - Uptime
[2016-04-08 10:59:07][4536][DEBUG] got service job: appstuff - Current Users
[2016-04-08 10:59:07][4539][DEBUG] got service job: localhost - ActiveServiceChecks 1mn
[2016-04-08 10:59:08][4540][DEBUG] got service job: sharechanger - Ping
[2016-04-08 10:59:10][4538][DEBUG] got service job: dragon - Disk - C
[2016-04-08 10:59:11][4536][DEBUG] got service job: shareinfinite- Disk - C
[2016-04-08 10:59:12][4537][DEBUG] got service job: citrixtest9x5period - Citrix Independent Management Architecture Service
[2016-04-08 10:59:14][4539][DEBUG] got service job: shareinfinite- CPU Load
[2016-04-08 10:59:15][4540][DEBUG] got service job: exchangeserver - Exchange_RPC_User_Count
[2016-04-08 10:59:16][4537][DEBUG] got service job: nagiosxihost - File System Space
[2016-04-08 10:59:17][4540][DEBUG] got service job: citrixtest9x5period - Citrix Group Policy Engine
[2016-04-08 10:59:19][4537][DEBUG] got service job: citrixtest9x5period - Citrix XTE Server
[2016-04-08 10:59:20][4539][DEBUG] got service job: appstuff - File System Space
[2016-04-08 10:59:20][4539][DEBUG] got host job: citrixtest9x5period
[2016-04-08 10:59:20][4536][DEBUG] got host job: nagiosxihost
[2016-04-08 10:59:21][4538][DEBUG] got service job: dragon - Memory Usage
[2016-04-08 10:59:22][4540][DEBUG] got service job: exchangeserver22 - Exchange_Pending_Ping_Count
[2016-04-08 10:59:22][4536][DEBUG] got service job: appstuff - check_log3_kpi_SQL
[2016-04-08 10:59:23][4537][DEBUG] got service job: citrixtest9x5period - Citrix MFCOM Service
[2016-04-08 10:59:25][4537][DEBUG] got service job: hamster - CPU Usage Counter
[2016-04-08 10:59:26][4540][DEBUG] got service job: exchangeserver - Active_Virtual_Memory_in_MB
[2016-04-08 10:59:27][4537][DEBUG] got service job: nagiosxihost - SSH
[2016-04-08 10:59:27][4540][DEBUG] got service job: exchangeserver - Exchange_RPC_Connection_Count
[2016-04-08 10:59:29][4537][DEBUG] got service job: hamster - CPU Load NRPE 80 180 1440
[2016-04-08 10:59:30][4540][DEBUG] got service job: exchangeserver22 - CPU Load
[2016-04-08 10:59:30][4540][DEBUG] got host job: exchangeserver
[2016-04-08 10:59:31][4537][DEBUG] got service job: server777 - Disk - M
[2016-04-08 10:59:32][4537][DEBUG] got service job: localhost - ExternalCommandsUsed 1mn
[2016-04-08 10:59:33][4679][DEBUG] child started with pid: 4679
[2016-04-08 10:59:34][4537][DEBUG] got service job: hamster - CPU Load
[2016-04-08 10:59:35][4540][DEBUG] got service job: AliasLongFixingTester - Current Load
[2016-04-08 10:59:36][4536][DEBUG] got service job: exchangeserver22 - Exchange_RPC_Connection_Count
[2016-04-08 10:59:37][4538][DEBUG] got service job: localhost - ActiveHostChecks 1mn
[2016-04-08 10:59:37][4538][DEBUG] got service job: dragon - Uptime
[2016-04-08 10:59:38][4537][DEBUG] got service job: nagiosxihost - Swap
[2016-04-08 10:59:39][4536][DEBUG] got service job: ping9x5check_period - Ping
[2016-04-08 10:59:40][4539][DEBUG] got host job: fakeserver
[2016-04-08 10:59:40][4536][DEBUG] got host job: ping9x5check_period
[2016-04-08 10:59:40][4540][DEBUG] got host job: nagiosxihost
[2016-04-08 10:59:40][4537][DEBUG] got host job: AliasLongFixingTester
[2016-04-08 10:59:41][4540][DEBUG] got service job: nrpe9x5checkperiod - Current Users
[2016-04-08 10:59:42][4536][DEBUG] got service job: citrixtest9x5period - Ping
[2016-04-08 10:59:42][4539][DEBUG] got service job: localhost - PassiveHostChecks 1mn
[2016-04-08 10:59:43][4537][DEBUG] got service job: someserver999- File System Space
[2016-04-08 10:59:43][4537][DEBUG] check exited with exit code > 3. Exit: 255
[2016-04-08 10:59:43][4537][DEBUG] stdout:
[2016-04-08 10:59:45][4537][DEBUG] got service job: nagiosxihost - Ping
[2016-04-08 10:59:46][4539][DEBUG] got service job: nrpe9x5checkperiod - SSH
[2016-04-08 10:59:47][4539][DEBUG] got service job: hamster - CPU Load NRPE
[2016-04-08 10:59:47][4536][DEBUG] got service job: citrixtest9x5period - Remote Desktop Services
[2016-04-08 10:59:48][4540][DEBUG] got service job: hamster - Uptime
[2016-04-08 10:59:49][4539][DEBUG] got service job: printer9x5checks - Ping
[2016-04-08 10:59:50][4537][DEBUG] got service job: windows9x5check_period - Uptime
[2016-04-08 10:59:50][4538][DEBUG] got service job: dragon - Ping
[2016-04-08 10:59:50][4540][DEBUG] got service job: <SOMEIP>- CPU Usage for VMHost
[2016-04-08 10:59:50][4540][DEBUG] Using Embedded Perl interpreter for: /usr/local/nagios/libexec/check_esx3.pl
[2016-04-08 10:59:50][4539][DEBUG] got service job: <SOMEIP>- Datastore usage for VMHost
[2016-04-08 10:59:50][4539][DEBUG] Using Embedded Perl interpreter for: /usr/local/nagios/libexec/check_esx3.pl
[2016-04-08 10:59:50][4536][DEBUG] got service job: <SOMEIP>- Networking for VMHost
[2016-04-08 10:59:50][4536][DEBUG] Using Embedded Perl interpreter for: /usr/local/nagios/libexec/check_esx3.pl
[2016-04-08 10:59:51][4537][DEBUG] got service job: SQLSERVER - MSSQL Buffer Hit Ratio
[2016-04-08 10:59:51][4538][DEBUG] got service job: SQLSERVER - MSSQL Checkpoint Pages Per Sec
[2016-04-08 10:59:51][4538][DEBUG] got service job: SQLSERVER - MSSQL Page Splits Per Sec
[2016-04-08 10:59:51][4537][DEBUG] got service job: SQLSERVER - MSSQL Readaheads Per Sec
[2016-04-08 10:59:51][4538][DEBUG] got host job: windows9x5check_period
[2016-04-08 10:59:51][4537][DEBUG] got host job: printer9x5checks
[2016-04-08 10:59:51][4540][DEBUG] got service job: swordfish - DNS Resolution
[2016-04-08 10:59:55][4538][DEBUG] got service job: exchangeserver22 - Memory Usage
[2016-04-08 10:59:56][4537][DEBUG] got service job: localhost - PassiveServiceChecks 1mn
[2016-04-08 10:59:57][4540][DEBUG] got service job: shareinfinite- Multi Address Ping
[2016-04-08 10:59:58][4539][DEBUG] got service job: appstuff - SSH
[2016-04-08 10:59:59][4537][DEBUG] got service job: windows9x5check_period - Disk - C
[2016-04-08 11:00:00][4536][DEBUG] got host job: <someotherip>
[2016-04-08 11:00:00][4538][DEBUG] got host job: server332
[2016-04-08 11:00:00][4537][DEBUG] got service job: swordfish - Ping
[2016-04-08 11:00:01][4538][DEBUG] got service job: someserver999- Check Disk
[2016-04-08 11:00:01][4538][DEBUG] check exited with exit code > 3. Exit: 255
[2016-04-08 11:00:01][4538][DEBUG] stdout:
[2016-04-08 11:00:02][4537][DEBUG] got service job: windows9x5check_period - Eventlog
[2016-04-08 11:00:03][4538][DEBUG] got service job: exchangeserver22 - Ping
[2016-04-08 11:00:04][4964][DEBUG] child started with pid: 4964
[2016-04-08 11:00:04][4540][DEBUG] got service job: server777 - SQL Server
[2016-04-08 11:00:05][4540][DEBUG] got service job: server777 - Ping
[2016-04-08 11:00:06][4539][DEBUG] got service job: localhost - ActiveServiceChecks 1mn
[2016-04-08 11:00:07][4537][DEBUG] got service job: server777 - CPU Load
[2016-04-08 11:00:08][4537][DEBUG] got service job: windows9x5check_period - Memory Usage
[2016-04-08 11:00:09][4538][DEBUG] got host job: server2012
[2016-04-08 11:00:10][4537][DEBUG] got service job: swordfish - SSL Certificate
[2016-04-08 11:00:10][4980][DEBUG] child started with pid: 4980
[2016-04-08 11:00:10][4539][DEBUG] got host job: swordfish
[2016-04-08 11:00:10][4540][DEBUG] got host job: windows9x5check_period
[2016-04-08 11:00:10][4537][DEBUG] got host job: serverrrrr21
[2016-04-08 11:00:11][4540][DEBUG] got service job: windows9x5check_period - CPU Load
[2016-04-08 11:00:12][4537][DEBUG] got service job: nrpe9x5checkperiod - Total Processes
[2016-04-08 11:00:13][4537][DEBUG] got service job: server777 - Uptime
[2016-04-08 11:00:14][4539][DEBUG] got service job: swordfish - DNS IP Match
[2016-04-08 11:00:15][4537][DEBUG] got service job: server777 - MountPoint - M:Star_ReportLog
[2016-04-08 11:00:16][4537][DEBUG] got service job: exchangeserver22 - Exchange_RPC_User_Count
[2016-04-08 11:00:17][4540][DEBUG] got service job: someserver999- Current Load
[2016-04-08 11:00:17][4540][DEBUG] check exited with exit code > 3. Exit: 255
[2016-04-08 11:00:17][4540][DEBUG] stdout:
[2016-04-08 11:00:18][4537][DEBUG] got service job: hamster - Eventlog
[2016-04-08 11:00:19][4537][DEBUG] got service job: serverxyzz- Current Users
[2016-04-08 11:00:20][4540][DEBUG] got service job: sharechanger - CPU Load
[2016-04-08 11:00:20][4538][DEBUG] got host job: exchangeserver22
[2016-04-08 11:00:20][4980][DEBUG] got host job: nrpe9x5checkperiod
[2016-04-08 11:00:20][4536][DEBUG] got host job: localhost
[2016-04-08 11:00:21][4536][DEBUG] got service job: nrpe9x5checkperiod - File System Space
[2016-04-08 11:00:22][4537][DEBUG] check exited with exit code > 3. Exit: 255
[2016-04-08 11:00:22][4537][DEBUG] stdout:
[2016-04-08 11:00:22][4539][DEBUG] got service job: exchangeserver22 - Exchange Connection Count
[2016-04-08 11:00:23][4540][DEBUG] got service job: ARCHITECT - Ping
[2016-04-08 11:00:24][4539][DEBUG] got service job: exchangeserver22 - Active_Virtual_Memory_in_MB
[2016-04-08 11:00:26][4536][DEBUG] got service job: server777 - IIS World Wide Web Publishing Service
[2016-04-08 11:00:27][4538][DEBUG] got service job: server777 - Memory Usage
[2016-04-08 11:00:28][4539][DEBUG] got service job: someserver999- Current Users
[2016-04-08 11:00:28][4539][DEBUG] check exited with exit code > 3. Exit: 255
[2016-04-08 11:00:28][4539][DEBUG] stdout:
[2016-04-08 11:00:30][4538][DEBUG] got service job: exchangeserver22 - RPC_HTTP_Connection_Count
[2016-04-08 11:00:30][4980][DEBUG] got host job: serverrrrr21
[2016-04-08 11:00:30][4539][DEBUG] got host job: server777
[2016-04-08 11:00:30][4536][DEBUG] got host job: se1rverrrrr21
[2016-04-08 11:00:31][4537][DEBUG] got service job: localhost - ExternalCommandsUsed 1mn
[2016-04-08 11:00:32][4537][DEBUG] got service job: nrpe9x5checkperiod - Swap
[2016-04-08 11:00:34][4980][DEBUG] got service job: AliasLongFixingTester - Current Load
[2016-04-08 11:00:35][5095][DEBUG] child started with pid: 5095
[2016-04-08 11:00:35][4538][DEBUG] got service job: localhost - ActiveHostChecks 1mn
[2016-04-08 11:00:36][4540][DEBUG] got service job: nagiosxihost - Current Load
[2016-04-08 11:00:38][4537][DEBUG] got host job: sharechanger
[2016-04-08 11:00:40][4537][DEBUG] got service job: localhost - PassiveHostChecks 1mn
[2016-04-08 11:00:40][4536][DEBUG] got host job: nagiosxihost
[2016-04-08 11:00:40][4540][DEBUG] got host job: AliasLongFixingTester
[2016-04-08 11:00:41][4980][DEBUG] got service job: server777 - Eventlog
[2016-04-08 11:00:42][4538][DEBUG] got service job: server2012 - Uptime
[2016-04-08 11:00:43][4537][DEBUG] got service job: appstuff - Current Load
[2016-04-08 11:00:44][4536][DEBUG] got service job: nagiosxihost - Current Users
[2016-04-08 11:00:45][4539][DEBUG] got service job: hamster - Ping
[2016-04-08 11:00:46][4980][DEBUG] got service job: nrpe9x5checkperiod - Ping
[2016-04-08 11:00:49][4537][DEBUG] got service job: server2012 - Memory Usage
[2016-04-08 11:00:50][4536][DEBUG] got service job: dragon - CPU Load
[2016-04-08 11:00:50][4540][DEBUG] got service job: <SOMEIP>- Input / Output for VMHost
[2016-04-08 11:00:50][4540][DEBUG] Using Embedded Perl interpreter for: /usr/local/nagios/libexec/check_esx3.pl
[2016-04-08 11:00:50][4539][DEBUG] got service job: <SOMEIP>- Services for VMHost
[2016-04-08 11:00:50][4539][DEBUG] Using Embedded Perl interpreter for: /usr/local/nagios/libexec/check_esx3.pl
[2016-04-08 11:00:50][4980][DEBUG] got service job: server332 - MSSQL Average Wait Time
[2016-04-08 11:00:51][4540][DEBUG] got service job: server332 - MSSQL Log Shrinks
[2016-04-08 11:00:51][4539][DEBUG] got service job: server332 - MSSQL Target Pages Per Sec
[2016-04-08 11:00:51][4980][DEBUG] got service job: localhost - PassiveServiceChecks 1mn
[2016-04-08 11:00:52][4980][DEBUG] got service job: hamster - Check log3
[2016-04-08 11:00:53][4539][DEBUG] got service job: appstuff - Swap
[2016-04-08 11:00:54][4540][DEBUG] got service job: exchangeserver22 - HTTP_PROXY_Unique_Users
[2016-04-08 11:00:55][4539][DEBUG] got service job: hamster - Memory Physical RAM Usage
[2016-04-08 11:00:55][4538][DEBUG] got service job: someserver999- Ping
[2016-04-08 11:00:56][4540][DEBUG] got service job: sharechanger - Disk - C
[2016-04-08 11:00:57][4540][DEBUG] got service job: exchangeserver - HTTP_PROXY_Unique_Users
[2016-04-08 11:00:58][4539][DEBUG] got service job: nagiosxihost - Total Processes
[2016-04-08 11:00:59][4538][DEBUG] got service job: windows9x5check_period - Ping
[2016-04-08 11:01:00][4980][DEBUG] got service job: hamster - Disk - C
[2016-04-08 11:01:00][4537][DEBUG] got host job: dragon
[2016-04-08 11:01:00][4538][DEBUG] got host job: server2012
[2016-04-08 11:01:00][4536][DEBUG] got host job: nagiosxihost
[2016-04-08 11:01:00][4540][DEBUG] got host job: exchangeserver22
[2016-04-08 11:01:00][4980][DEBUG] got host job: hamster
[2016-04-08 11:01:01][4537][DEBUG] got service job: exchangeserver22 - Uptime
[2016-04-08 11:01:02][4540][DEBUG] got service job: appstuff - Ping
[2016-04-08 11:01:03][4536][DEBUG] got service job: localhost - ActiveServiceChecks 1mn
[2016-04-08 11:01:04][4537][DEBUG] got service job: appstuff - Total Processes
[2016-04-08 11:01:05][4540][DEBUG] got service job: someserver999- Total Processes
[2016-04-08 11:01:05][4540][DEBUG] check exited with exit code > 3. Exit: 255
[2016-04-08 11:01:05][4540][DEBUG] stdout:
[2016-04-08 11:01:06][4536][DEBUG] got service job: exchangeserver - Exchange_RPC_Average_Latency
[2016-04-08 11:01:06][5394][DEBUG] child started with pid: 5394
[2016-04-08 11:01:07][4539][DEBUG] got service job: swordfish - HTTP
[2016-04-08 11:01:08][4980][DEBUG] got service job: serverxyzz- Ping
[2016-04-08 11:01:09][4536][DEBUG] got service job: exchangeserver - Exchange_OWA_Unique_User_Count
[2016-04-08 11:01:10][4536][DEBUG] got service job: serverxyzz- Current Load
[2016-04-08 11:01:10][4537][DEBUG] got host job: serverrrrr21
[2016-04-08 11:01:11][4536][DEBUG] check exited with exit code > 3. Exit: 255
[2016-04-08 11:01:11][4536][DEBUG] stdout:
[2016-04-08 11:01:12][4536][DEBUG] got service job: exchangeserver - Ping
[2016-04-08 11:01:13][4537][DEBUG] got service job: exchangeserver22 - Exchange_User_Count
[2016-04-08 11:01:14][4540][DEBUG] got service job: exchangeserver - Uptime
[2016-04-08 11:01:15][4536][DEBUG] got service job: sharechanger - Memory Usage
[2016-04-08 11:01:17][4539][DEBUG] got service job: exchangeserver22 - Disk - C
[2016-04-08 11:01:17][4540][DEBUG] got service job: exchangeserver - Exchange Connection Count
[2016-04-08 11:01:18][4536][DEBUG] got host job: servepvs
[2016-04-08 11:01:19][4539][DEBUG] got service job: server777 - MountPoint - M:Star_Report1
[2016-04-08 11:01:20][4539][DEBUG] got service job: server2012 - Ping
[2016-04-08 11:01:20][4980][DEBUG] got host job: se1rverrrrr21
[2016-04-08 11:01:20][4536][DEBUG] got host job: exchangeserver22
[2016-04-08 11:01:20][4537][DEBUG] got service job: hamster - Memory Paging File Usage
[2016-04-08 11:01:21][4537][DEBUG] got service job: hamster - Memory Usage
[2016-04-08 11:01:22][4540][DEBUG] got service job: exchangeserver22 - Exchange_RPC_Average_Latency
[2016-04-08 11:01:24][4536][DEBUG] got service job: exchangeserver22 - Processor_Timing
[2016-04-08 11:01:25][4540][DEBUG] got service job: citrixtest9x5period - Citrix Print Manager Service
[2016-04-08 11:01:26][4537][DEBUG] got service job: exchangeserver - Disk - C
[2016-04-08 11:01:27][4536][DEBUG] got service job: exchangeserver - Eventlog
[2016-04-08 11:01:28][4537][DEBUG] got service job: exchangeserver - Exchange_User_Count
[2016-04-08 11:01:30][4537][DEBUG] got service job: server2012 - CPU Usage
[2016-04-08 11:01:30][4538][DEBUG] got host job: server2012
[2016-04-08 11:01:31][4536][DEBUG] got service job: localhost - ExternalCommandsUsed 1mn
[2016-04-08 11:01:32][4540][DEBUG] got service job: serverxyzz- Total System Space
[2016-04-08 11:01:33][4536][DEBUG] got service job: exchangeserver - CPU Load
[2016-04-08 11:01:34][4536][DEBUG] got service job: AliasLongFixingTester - Current Load
[2016-04-08 11:01:35][4540][DEBUG] check exited with exit code > 3. Exit: 255
[2016-04-08 11:01:35][4540][DEBUG] stdout:
[2016-04-08 11:01:35][4536][DEBUG] got service job: citrixtest9x5period - CPU Load
[2016-04-08 11:01:37][4540][DEBUG] got service job: localhost - ActiveHostChecks 1mn
[2016-04-08 11:01:37][5510][DEBUG] child started with pid: 5510
[2016-04-08 11:01:38][4536][DEBUG] got service job: exchangeserver22 - Eventlog
[2016-04-08 11:01:39][4539][DEBUG] got service job: exchangeserver - Memory Usage
[2016-04-08 11:01:40][4536][DEBUG] got service job: exchangeserver22 - Exchange_OWA_Unique_User_Count
[2016-04-08 11:01:40][4537][DEBUG] got host job: exchangeserver22
[2016-04-08 11:01:40][4540][DEBUG] got host job: se1rverrrrr21
[2016-04-08 11:01:41][4537][DEBUG] got service job: hamster - CPU Load NRPE Counters
[2016-04-08 11:01:43][4538][DEBUG] got service job: localhost - PassiveHostChecks 1mn
[2016-04-08 11:01:44][4540][DEBUG] got service job: server777 - MountPoint - M:Star_Report1 FREE
[2016-04-08 11:01:46][4536][DEBUG] got service job: localhost - PassiveServiceChecks 1mn
[2016-04-08 11:01:47][4536][DEBUG] got service job: server332 - MSSQL Log Truncations
[2016-04-08 11:01:48][4540][DEBUG] got service job: serverxyzz- Swap
[2016-04-08 11:01:50][4537][DEBUG] got service job: citrixtest9x5period - Uptime
[2016-04-08 11:01:50][4537][DEBUG] got service job: SQLSERVER - MSSQL Lock Timeouts Per Sec
[2016-04-08 11:01:50][4538][DEBUG] got service job: SQLSERVER - MSSQL Lock Wait Times
[2016-04-08 11:01:50][4536][DEBUG] got service job: SQLSERVER - MSSQL Lock Waits Per Sec
[2016-04-08 11:01:50][4980][DEBUG] got service job: SQLSERVER - MSSQL Page Looks Per Sec
[2016-04-08 11:01:50][4539][DEBUG] got service job: SQLSERVER - MSSQL Page Reads Per Sec
[2016-04-08 11:01:50][4537][DEBUG] got service job: localhost - AvgHostExecTime
[2016-04-08 11:01:50][4536][DEBUG] got service job: localhost - AvgServiceExecTime
[2016-04-08 11:01:50][4538][DEBUG] got service job: localhost - ExternalCommandsUsed 5mn
[2016-04-08 11:01:50][4980][DEBUG] got service job: localhost - HighCommandBufferUsage
[2016-04-08 11:01:51][4539][DEBUG] got service job: localhost - PassiveHostChecks 15mn
[2016-04-08 11:01:51][4537][DEBUG] got service job: serverxyzz- Total Zombie Processes
[2016-04-08 11:01:51][4540][DEBUG] check exited with exit code > 3. Exit: 255
[2016-04-08 11:01:51][4540][DEBUG] stdout:
[2016-04-08 11:01:51][4537][DEBUG] check exited with exit code > 3. Exit: 255
[2016-04-08 11:01:51][4537][DEBUG] stdout:
[2016-04-08 11:01:52][4536][DEBUG] got service job: server2012 - IIS Web Server
[2016-04-08 11:01:53][4539][DEBUG] got service job: citrixtest9x5period - Eventlog
[2016-04-08 11:01:54][4537][DEBUG] got service job: server2012 - Drive C: Disk Usage
[2016-04-08 11:01:55][4540][DEBUG] got service job: citrixtest9x5period - Citrix XML Service
[2016-04-08 11:01:56][4538][DEBUG] got host job: appstuff
[2016-04-08 11:01:58][4540][DEBUG] got service job: shareinfinite- Memory Usage
[2016-04-08 11:01:59][4538][DEBUG] got service job: localhost - ActiveServiceChecks 1mn
[2016-04-08 11:02:00][4538][DEBUG] got service job: citrixtest9x5period - Citrix Services Manager Service
[2016-04-08 11:02:00][4980][DEBUG] got host job: SP13BPR01DAR
[2016-04-08 11:02:00][4540][DEBUG] got host job: tibbeqa01dar
[2016-04-08 11:02:01][4539][DEBUG] got service job: sharechanger - Eventlog
[2016-04-08 11:02:02][4540][DEBUG] got service job: sharechanger - Uptime
[2016-04-08 11:02:03][4540][DEBUG] got service job: appstuff - Current Users
[2016-04-08 11:02:04][4538][DEBUG] got service job: sharechanger - Ping
[2016-04-08 11:02:05][4536][DEBUG] got service job: dragon - Disk - C
[2016-04-08 11:02:06][4537][DEBUG] got service job: shareinfinite- Disk - C
[2016-04-08 11:02:07][4540][DEBUG] got service job: citrixtest9x5period - Citrix Independent Management Architecture Service
[2016-04-08 11:02:08][4539][DEBUG] got service job: shareinfinite- Uptime
[2016-04-08 11:02:08][5775][DEBUG] child started with pid: 5775
[2016-04-08 11:02:09][4537][DEBUG] got service job: shareinfinite- Eventlog
[2016-04-08 11:02:10][4540][DEBUG] got service job: shareinfinite- CPU Load
[2016-04-08 11:02:11][4537][DEBUG] got service job: exchangeserver - Exchange_RPC_User_Count
[2016-04-08 11:02:12][4540][DEBUG] got service job: nagiosxihost - File System Space
[2016-04-08 11:02:13][4537][DEBUG] got service job: citrixtest9x5period - Citrix Group Policy Engine
[2016-04-08 11:02:14][4537][DEBUG] got service job: citrixtest9x5period - Citrix XTE Server
[2016-04-08 11:02:15][4539][DEBUG] got service job: dragon - Eventlog
[2016-04-08 11:02:16][4537][DEBUG] got service job: appstuff - File System Space
[2016-04-08 11:02:18][4537][DEBUG] got service job: dragon - Memory Usage
[2016-04-08 11:02:19][4536][DEBUG] got service job: appstuff - check_log3_kpi_SQL
[2016-04-08 11:02:20][4540][DEBUG] got service job: exchangeserver22 - Exchange_Pending_Ping_Count
[2016-04-08 11:02:20][4540][DEBUG] got host job: exchangeserver22
[2016-04-08 11:02:20][4538][DEBUG] got service job: citrixtest9x5period - Citrix MFCOM Service

[bheden]

Can we turn the debug level up to 3 on the worker and get more output from the worker log, please?

Also, just to make sure everything went smoothly, can I see the output of:

Code: Select all

yum list installed | grep gearman

and

Code: Select all

iptables -L

I'm assuming that the server and worker are on the same server, but if they aren't: I'd like to see the output from iptables for both server and worker.

Thanks!