Acknowldgement is not working

[absarcompass]

Hi ssax,
thanks for that, have PMd you the profile.

@tacolover101
I used the same article to change the values in sysctl.conf
below is the result of using that article.

# sysctl -p
net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
error: "net.bridge.bridge-nf-call-ip6tables" is an unknown key
error: "net.bridge.bridge-nf-call-iptables" is an unknown key
error: "net.bridge.bridge-nf-call-arptables" is an unknown key
kernel.msgmnb = 262144000
kernel.msgmax = 262144000
kernel.shmmax = 4294967295
kernel.shmall = 268435456
kernel.msgmni = 512000

[ssax]

Please edit your /usr/local/nagios/etc/ndo2db.cfg and change these settings to your new settings:

Code: Select all

db_servertype=mysql
db_host=localhost
db_port=3306

db_user=ndoutils
db_pass=n@gweb

Then run these commands and see if it resolves your issue:

Code: Select all

service nagios stop
service ndo2db restart
service nagios start

Let us know the results.

Thank you

[absarcompass]

@ssax
just to confirm you want me to change config even though I have offloaded the DB??

[absarcompass]

you want me to revert offload? or want me to check if these fields have newer values off remote DB server?
i left the old values in config by commenting them out.

[absarcompass]

restarted nagios and it didn't work. no multiple nagios process in ipcs but queue maxing
i don't think so it is realted to offloaded DB, is there anything else i can check. only thing i remember is that it started happening right after when i upgraded XI to version 5.5.2

[absarcompass]

it is taking longer now previoiusly services were returning the monitoring checks in and around an hour but now it is taking more, around 2 hours and more
Oct 8 10:21:37 nagiosxi ndo2db: Warning: queue send error, retrying...
Oct 8 10:21:38 nagiosxi ndo2db: Message sent to queue.
Oct 8 10:21:38 nagiosxi ndo2db: Warning: queue send error, retrying...
Oct 8 10:21:39 nagiosxi ndo2db: Message sent to queue.
Oct 8 10:21:39 nagiosxi ndo2db: Warning: queue send error, retrying...
Oct 8 10:21:40 nagiosxi ndo2db: Message sent to queue.
Oct 8 10:21:40 nagiosxi ndo2db: Warning: queue send error, retrying...
Oct 8 10:21:41 nagiosxi ndo2db: Message sent to queue.
Oct 8 10:21:41 nagiosxi ndo2db: Warning: queue send error, retrying...

I have increased the values in sysctl.conf but it is not helping

net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
error: "net.bridge.bridge-nf-call-ip6tables" is an unknown key
error: "net.bridge.bridge-nf-call-iptables" is an unknown key
error: "net.bridge.bridge-nf-call-arptables" is an unknown key
kernel.msgmnb = 393216000
kernel.msgmax = 393216000
kernel.shmmax = 4294967295
kernel.shmall = 268435456
kernel.msgmni = 512000

the time when queue is retrying nagios has no cpu load in top
like this

top - 10:23:52 up 60 days, 19:04, 2 users, load average: 0.05, 3.10, 8.51
Tasks: 219 total, 1 running, 218 sleeping, 0 stopped, 0 zombie
Cpu(s): 38.1%us, 9.7%sy, 0.0%ni, 44.8%id, 7.1%wa, 0.0%hi, 0.3%si, 0.0%st
Mem: 5985396k total, 3554384k used, 2431012k free, 161800k buffers
Swap: 2064376k total, 44668k used, 2019708k free, 1334504k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
47743 nagios 20 0 59640 11m 1064 S 17.6 0.2 1:09.94 ndo2db
47742 nagios 20 0 49840 1696 984 S 3.9 0.0 0:10.56 ndo2db
47735 nagios 20 0 753m 331m 161m S 2.0 5.7 0:16.57 nagios

but once queue is cleared the system cpu load shoot to higher side
around load average: 18.06, 15.59, 10.83

[ssax]

I was saying that you need to update that file with the offloaded DB information, it's a requirement.

Please follow from page 10 of this PDF onwards to validate you've changed all the required files:

https://assets.nagios.com/downloads/nag ... Server.pdf

[absarcompass]

Hi ssax,
yup have go through the doc again and have double check configs. I can run NagiosXI web very fine
its just that when ever nagios service is restarted from web or command line or new settings applied from web it stops service check and greyed out the hosts assigned to the services on web. and it was not happening before upgrade to 5.5.2

can you please confirm if nagios deamon writes to nagios.cfg all the time
below is the output of iotop for last 5 minutes

Total DISK READ: 0.00 B/s | Total DISK WRITE: 215.33 M/s
TID PRIO USER DISK READ DISK WRITE> SWAPIN IO COMMAND
1822 be/4 nagios 0.00 B 18.51 G 0.00 % 16.58 % nagios -d /usr/local/nagios/etc/nagios.cfg

I can not figure out why every time I restart nagios service or apply settings it stops working for at least more than an hour.

[ssax]

Yeah, it shouldn't be writing to it all the time. Very strange.

Please PM me a fresh copy of your profile, you can download it from Admin > System Profile > Download Profile and upload it to the ticket by clicking the "choose item" link at the bottom of the menu.​ Make sure to wait until the file is finished uploading before clicking the Post Reply button.

[ssax]

Additionally, are you a customer? If you are, can you login and create a ticket so that we can get a remote setup?

https://support.nagios.com/tickets/