Page 9 of 13
Re: NDO2DB Issue out of the blue
Posted: Wed Aug 26, 2015 12:46 pm
by jfrickson
No, patch the original. But use this patch instead. I messed up the other one.
Re: NDO2DB Issue out of the blue
Posted: Wed Aug 26, 2015 12:50 pm
by BanditBBS
jfrickson wrote:No, patch the original. But use this patch instead. I messed up the other one.
Done and looking better
Code: Select all
[nagios@iss-chi-nag09 var]$ ps -ef|grep ndo
nagios 2000 1 0 12:47 ? 00:00:00 /usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2db.cfg
nagios 2062 2000 1 12:48 ? 00:00:00 /usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2db.cfg
nagios 2063 2062 21 12:48 ? 00:00:04 /usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2db.cfg
nagios 2084 1950 0 12:48 pts/2 00:00:00 grep ndo
[nagios@iss-chi-nag09 var]$ ll
total 298656
-rw-r--r-- 1 nagios nagios 76 Aug 26 12:48 debug_2000_.log
-rw-r--r-- 1 nagios nagios 5378622 Aug 26 12:48 debug_2062_.log
-rw-r--r-- 1 nagios nagios 48943887 Aug 26 12:48 debug_2063_.log
-rw-r--r-- 1 nagios nagios 9472224 Aug 26 12:33 ndo2db.debug
-rw-r--r-- 1 nagios nagios 20002925 Aug 26 12:33 ndo2db.debug.old
-rw-r--r-- 1 nagios nagios 5 Aug 26 12:47 ndo2db.lock
-rw-r--r-- 1 root root 222007026 Aug 21 13:44 ndo2db.log
I'll update when it crashes/hangs
p.s. I liked the "Bleh...sorry" unedited message, lol
EDIT: It still seems to be only the same pids so the files are growing rather rapidly, the 2063 one is 500MB already
Re: NDO2DB Issue out of the blue
Posted: Wed Aug 26, 2015 1:12 pm
by jfrickson
BanditBBS wrote:
...
EDIT: It still seems to be only the same pids so the files are growing rather rapidly, the 2063 one is 500MB already
I was thinking they would turn over faster.
This one should keep less than 1,000 lines in the current file, and keep a previous log for the previous 1,000 lines.
Re: NDO2DB Issue out of the blue
Posted: Wed Aug 26, 2015 1:17 pm
by BanditBBS
jfrickson wrote:BanditBBS wrote:
...
EDIT: It still seems to be only the same pids so the files are growing rather rapidly, the 2063 one is 500MB already
I was thinking they would turn over faster.
This one should keep less than 1,000 lines in the current file, and keep a previous log for the previous 1,000 lines.
Ok, this one is now in place, we'll see what happens! With this I can also keep my event handler running too since the pids will change and the old logs will remain
Re: NDO2DB Issue out of the blue
Posted: Wed Aug 26, 2015 4:15 pm
by hsmith
BanditBBS wrote:jfrickson wrote:BanditBBS wrote:
...
EDIT: It still seems to be only the same pids so the files are growing rather rapidly, the 2063 one is 500MB already
I was thinking they would turn over faster.
This one should keep less than 1,000 lines in the current file, and keep a previous log for the previous 1,000 lines.
Ok, this one is now in place, we'll see what happens! With this I can also keep my event handler running too since the pids will change and the old logs will remain
You know the drill, let us know what you come up with!
Re: NDO2DB Issue out of the blue
Posted: Wed Aug 26, 2015 7:51 pm
by BanditBBS
PM'ing Trevor and jfrickson these files:
Code: Select all
-rw-r--r-- 1 nagios nagios 76 Aug 26 13:15 debug_6938_.log
-rw-r--r-- 1 nagios nagios 72366 Aug 26 17:46 debug_6948_.log
-rw-r--r-- 1 nagios nagios 204854 Aug 26 17:46 debug_6948_.log.prev
-rw-r--r-- 1 nagios nagios 12624549 Aug 26 17:46 debug_6949_.log
-rw-r--r-- 1 nagios nagios 9673526 Aug 26 17:41 debug_6949_.log.prev
Re: NDO2DB Issue out of the blue
Posted: Thu Aug 27, 2015 8:05 am
by BanditBBS
I have 2 more sets of files I can send if needed, just let me know!
Re: NDO2DB Issue out of the blue
Posted: Thu Aug 27, 2015 9:31 am
by jfrickson
BanditBBS wrote:I have 2 more sets of files I can send if needed, just let me know!
I don't think I will need them. I should have a fix for you in a little while, probably under an hour.
Re: NDO2DB Issue out of the blue
Posted: Thu Aug 27, 2015 10:57 am
by jfrickson
Apply the attached patch to the original ndo2db.c and run it. I'm pretty sure this will solve the problem.
Re: NDO2DB Issue out of the blue
Posted: Thu Aug 27, 2015 11:14 am
by BanditBBS
jfrickson wrote:Apply the attached patch to the original ndo2db.c and run it. I'm pretty sure this will solve the problem.
Well, I have this in, we'll see what happens. I must mention this though as well....based on the tail end of some of the debug logs generated when it got in its infinite loop I fixed the 2 checks that were causing a ton of longoutput and : from being sent, so, if that was the cause, we may not see it again anyway. But I had to do it! I'll let you know if it seems resolved(I'll call it resolved after it doesn't happen again in 3 days)