All mysql nagios tables are Empty

[deepak_symphony]

Did changes ,, but still same.

mysql> use nagios;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> select count(*) from nagios_services;
+----------+
| count(*) |
+----------+
| 0 |
+----------+
1 row in set (0.00 sec)

mysql> SELECT * FROM nagios_objects;
Empty set (0.00 sec)

[scottwilkerson]

You may also need to restart nagios

Code: Select all

service nagios restart

finally, are you getting error logs in the nagios.log?

Code: Select all

tail -f /usr/local/nagios/var/nagios.log

Or in syslog on your system?

Code: Select all

cat /var/log/messages|grep ndo2db

[deepak_symphony]

lnx0202:root# tail -f /usr/local/nagios/var/nagios.log
[1424053669] ndomod registered for adaptive contact data'
[1424053669] Event broker module '/usr/local/nagios/bin/ndomod.o' initialized successfully.
[1424053671] Successfully launched command file worker with pid 26589
[1424053690] ndomod: Error writing to data sink! Some output may get lost...
[1424053690] ndomod: Please check remote ndo2db log, database connection or SSL Parameters
[1424053706] ndomod: Still unable to reconnect to data sink. 0 items lost, 147 queued items to flush.
[1424053722] ndomod: Still unable to reconnect to data sink. 0 items lost, 343 queued items to flush.
[1424053738] ndomod: Still unable to reconnect to data sink. 0 items lost, 477 queued items to flush.
[1424053755] ndomod: Successfully reconnected to data sink! 0 items lost, 615 queued items to flush.
[1424053755] ndomod: Successfully flushed 615 queued items to data sink.

[deepak_symphony]

Feb 15 20:14:30 lnx0202 ndo2db: Warning: Retrying message send. This can occur because you have too few messages allowed or too few total bytes allowed in message queues. You are currently using 128000 of 32768 messages and 131072000 of 131072000 bytes in the queue. See README for kernel tuning options.
Feb 15 20:14:50 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:14:50 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:15:10 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:15:10 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:15:30 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:15:30 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:15:50 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:15:50 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:16:10 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:16:10 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:16:30 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:16:30 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:16:50 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:16:50 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:17:10 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:17:10 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:17:30 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:17:30 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:17:50 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:17:50 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:18:10 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:18:10 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:18:30 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:18:30 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:18:50 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:18:50 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:19:10 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:19:10 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:19:30 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:19:30 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:19:50 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:19:50 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:20:10 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:20:10 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:20:30 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:20:30 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:20:50 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:20:50 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:21:10 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:21:10 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:21:30 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:21:30 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:21:50 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:21:50 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:22:10 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:22:10 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:22:30 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:22:30 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:22:50 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:22:50 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:23:10 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:23:10 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:23:30 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:23:30 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:23:50 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:23:50 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:24:10 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:24:10 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:24:30 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:24:30 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:24:50 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:24:50 lnx0202 ndo2db: Warning: Retrying message send. This can occur because you have too few messages allowed or too few total bytes allowed in message queues. You are currently using 128000 of 32768 messages and 131072000 of 131072000 bytes in the queue. See README for kernel tuning options.
Feb 15 20:25:10 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:25:10 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:25:30 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:25:30 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:25:50 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:25:50 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:26:10 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:26:10 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:26:30 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:26:30 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:26:50 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:26:50 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:27:10 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:27:10 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:27:30 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:27:30 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:27:50 lnx0202 ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may neeed to be tuned. See README.
Feb 15 20:27:50 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 15 20:28:10 lnx0202 nagios: ndomod: Please check remote ndo2db log, database connection or SSL Parameters

[deepak_symphony]

Here the details:
# Controls the default maxmimum size of a mesage queue
kernel.msgmnb = 131072000

# Controls the maximum size of a message, in bytes
kernel.msgmax = 131072000

# Controls the maximum shared segment size, in bytes
kernel.shmmax = 68719476736

# Controls the maximum number of shared memory segments, in pages
kernel.shmall = 4294967296

[scottwilkerson]

dno2db doesn't appear to be connected to the MySQL DB

restart ndo again

Code: Select all

restart ndo2db restart

then run

Code: Select all

tail -f /var/log/messages|grep ndo2db

And if it still doesn't work, post your ndo2db.cfg

[deepak_symphony]

Still its same issue:

Logs:
lnx0202:root# tail -f /usr/local/nagios/var/nagios.log
[1424226194] ndomod registered for acknowledgement data'
[1424226194] ndomod registered for contact status data'
[1424226194] ndomod registered for adaptive contact data'
[1424226194] Event broker module '/usr/local/nagios/bin/ndomod.o' initialized successfully.
[1424226196] Successfully launched command file worker with pid 8052
[1424226211] ndomod: Still unable to connect to data sink. 0 items lost, 818 queued items to flush.
[1424226227] ndomod: Still unable to connect to data sink. 0 items lost, 974 queued items to flush.
[1424226243] ndomod: Still unable to connect to data sink. 0 items lost, 1183 queued items to flush.
[1424226259] ndomod: Successfully connected to data sink. 1338 queued items to flush.
[1424226260] ndomod: Successfully flushed 1338 queued items to data sink.
__________________________________________________________________________________

Feb 17 20:21:58 lnx0202 nagios: ndomod: Please check remote ndo2db log, database connection or SSL Parameters
Feb 17 20:24:19 lnx0202 ndo2db: Warning: Retrying message send. This can occur because you have too few messages allowed or too few total bytes allowed in message queues. You are currently using 128000 of 32768 messages and 131072000 of 131072000 bytes in the queue. See README for kernel tuning options.
Feb 17 20:24:19 lnx0202 ndo2db: Message sent to queue.
Feb 17 20:24:19 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 17 20:24:19 lnx0202 ndo2db: Message sent to queue.
Feb 17 20:24:19 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 17 20:24:19 lnx0202 ndo2db: Message sent to queue.
Feb 17 20:24:19 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 17 20:24:19 lnx0202 ndo2db: Message sent to queue.
Feb 17 20:24:19 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 17 20:24:19 lnx0202 ndo2db: Message sent to queue.
Feb 17 20:24:19 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 17 20:24:19 lnx0202 ndo2db: Message sent to queue.
Feb 17 20:24:19 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 17 20:24:19 lnx0202 ndo2db: Message sent to queue.
Feb 17 20:24:19 lnx0202 ndo2db: Warning: queue send error, retrying...
Feb 17 20:24:19 lnx0202 ndo2db: Message sent to queue.
Feb 17 20:24:19 lnx0202 ndo2db: Warning: queue send error, retrying...

[deepak_symphony]

lnx0202:root# egrep -v "^#|^$" ndo2db.cfg
lock_file=/usr/local/nagios/var/ndo2db.lock
ndo2db_user=nagios
ndo2db_group=nagios
socket_type=unix
socket_name=/usr/local/nagios/var/ndo.sock
tcp_port=5668
use_ssl=0
db_servertype=mysql
db_host=10.106.10.197
db_port=3306
db_name=nagios
db_prefix=nagios_
db_user=nagios
db_pass=nagios
max_timedevents_age=1440
max_systemcommands_age=10080
max_servicechecks_age=10080
max_hostchecks_age=10080
max_eventhandlers_age=44640
max_externalcommands_age=44640
max_notifications_age=44640
max_contactnotifications=44640
max_contactnotificationmethods=44640
max_logentries_age=129600
max_acknowledgements_age=44640
debug_level=2
debug_verbosity=2
debug_file=/usr/local/nagios/var/ndo2db.debug
max_debug_file_size=1000000

[deepak_symphony]

Also we observed that alert notification are not sent by nagios, if we restart then it will send that day and next day no alerts from nagios.

[deepak_symphony]

Please help us on priority as we are planing to move nagios for production environment once this issue get resolved.