NDO2DB Issue out of the blue
Re: NDO2DB Issue out of the blue
It hasn't been an issue in 27 hours. Not since I did the make and make install on the debug version. Once I stopped that and put the original in and restarted its been running great every since. I'll of course update this on Monday again.
2 of XI5.6.14 Prod/DR/DEV - Nagios LogServer 2 Nodes
See my projects on the Exchange at BanditBBS - Also check out my Nagios stuff on my personal page at Bandit's Home and at github
See my projects on the Exchange at BanditBBS - Also check out my Nagios stuff on my personal page at Bandit's Home and at github
Re: NDO2DB Issue out of the blue
45 hour mark and if failed...was just a good Saturday apparently. Nice thing though, I used the action component for the first time and create a script to fix(restart ndo and kill extra queues) it upon a click.
So, patiently waiting any feedback from the log, beta/alpha of a patch or any other idea.
EDIT: It has crashed 3 times this morning(Monday).
So, patiently waiting any feedback from the log, beta/alpha of a patch or any other idea.
EDIT: It has crashed 3 times this morning(Monday).
2 of XI5.6.14 Prod/DR/DEV - Nagios LogServer 2 Nodes
See my projects on the Exchange at BanditBBS - Also check out my Nagios stuff on my personal page at Bandit's Home and at github
See my projects on the Exchange at BanditBBS - Also check out my Nagios stuff on my personal page at Bandit's Home and at github
-
jfrickson
Re: NDO2DB Issue out of the blue
I looked over the log over the weekend and couldn't find anything suspicious.
Let's try something. Perform the same process as before, but instead of uncommenting all of the defines starting at line 72, just uncomment the first line from:
to:
Go to line 218. There you should see the following:
at the end of that list, add the following lines:
These are other signals that will cause the process to terminate.
Run ndo2db stand-alone as before, but you don't need to redirect stdout to a file. When the process dies, you should get a message saying:
with the 99 replaced with some number. Let me know what (if anything) gets printed.
Let's try something. Perform the same process as before, but instead of uncommenting all of the defines starting at line 72, just uncomment the first line from:
Code: Select all
/*#define DEBUG_NDO2DB 1*/ /* don't daemonize */Code: Select all
#define DEBUG_NDO2DB 1 /* don't daemonize */Code: Select all
signal(SIGQUIT,ndo2db_parent_sighandler);
signal(SIGTERM,ndo2db_parent_sighandler);
signal(SIGINT,ndo2db_parent_sighandler);
signal(SIGSEGV,ndo2db_parent_sighandler);
signal(SIGFPE,ndo2db_parent_sighandler);
signal(SIGCHLD,ndo2db_parent_sighandler);
Code: Select all
signal(SIGHUP,ndo2db_parent_sighandler);
signal(SIGILL,ndo2db_parent_sighandler);
signal(SIGABRT,ndo2db_parent_sighandler);
signal(SIGPIPE,ndo2db_parent_sighandler);
signal(SIGALRM,ndo2db_parent_sighandler);
signal(SIGUSR1,ndo2db_parent_sighandler);
signal(SIGUSR2,ndo2db_parent_sighandler);
signal(SIGBUS,ndo2db_parent_sighandler);
signal(SIGPOLL,ndo2db_parent_sighandler);
signal(SIGPROF,ndo2db_parent_sighandler);
signal(SIGSYS,ndo2db_parent_sighandler);
signal(SIGTRAP,ndo2db_parent_sighandler);
signal(SIGVTALRM,ndo2db_parent_sighandler);
signal(SIGXCPU,ndo2db_parent_sighandler);
signal(SIGXFSZ,ndo2db_parent_sighandler);
signal(SIGIO,ndo2db_parent_sighandler);
signal(SIGPWR,ndo2db_parent_sighandler);
Run ndo2db stand-alone as before, but you don't need to redirect stdout to a file. When the process dies, you should get a message saying:
Code: Select all
Caught the Signal 99 but don't care about this.Re: NDO2DB Issue out of the blue
Sure, thanks, I'll get right on this! (just need to figure out how to keep my terminal session active as it times out after inactivity - I'm sure there is a setting, lol).
2 of XI5.6.14 Prod/DR/DEV - Nagios LogServer 2 Nodes
See my projects on the Exchange at BanditBBS - Also check out my Nagios stuff on my personal page at Bandit's Home and at github
See my projects on the Exchange at BanditBBS - Also check out my Nagios stuff on my personal page at Bandit's Home and at github
Re: NDO2DB Issue out of the blue
tmux or screen will at least keep the session active on the server so you can reconnect and not lose any work.BanditBBS wrote:Sure, thanks, I'll get right on this! (just need to figure out how to keep my terminal session active as it times out after inactivity - I'm sure there is a setting, lol).
Former Nagios employee
Re: NDO2DB Issue out of the blue
I'm wondering, if you set NDO2DEBUG directive and it does not daemonize, doesn't that stop it from using IPC and cause everything to run in one process?
Grumpy Olde IT Guy
-
jfrickson
Re: NDO2DB Issue out of the blue
It does run in just one process, but it still uses IPC.rseiwert wrote:I'm wondering, if you set NDO2DEBUG directive and it does not daemonize, doesn't that stop it from using IPC and cause everything to run in one process?
Re: NDO2DB Issue out of the blue
Quick end of day update: Been running with modified ndo2db since 11:30ish(Central) and still running strong. Will update whenever it crashes.
2 of XI5.6.14 Prod/DR/DEV - Nagios LogServer 2 Nodes
See my projects on the Exchange at BanditBBS - Also check out my Nagios stuff on my personal page at Bandit's Home and at github
See my projects on the Exchange at BanditBBS - Also check out my Nagios stuff on my personal page at Bandit's Home and at github
-
jfrickson
Re: NDO2DB Issue out of the blue
These things never crash when you want them to. Thanks for keeping on top of it!
Re: NDO2DB Issue out of the blue
Nothing useful. It was at 100% and not doing its job so I can to close it manually, so its not crashing to give any codejfrickson wrote:These things never crash when you want them to. Thanks for keeping on top of it!
EDIT: Went back to original binary. Its not crashing to give us anything, it just spikes to 100% and stops doing anything. Its crashed 2 more times since last night
2 of XI5.6.14 Prod/DR/DEV - Nagios LogServer 2 Nodes
See my projects on the Exchange at BanditBBS - Also check out my Nagios stuff on my personal page at Bandit's Home and at github
See my projects on the Exchange at BanditBBS - Also check out my Nagios stuff on my personal page at Bandit's Home and at github