Page 2 of 2

Re: Instance status different in console and GUI

Posted: Wed Mar 18, 2015 11:29 am
by jolson
Thank you for that, much appreciated. Any chance that selinux is running on either server?

Code: Select all

sestatus
If so, let's try turning it off temporarily and seeing if that helps:

Code: Select all

setenforce 0
Let us know. Thanks!

Re: Instance status different in console and GUI

Posted: Thu Mar 19, 2015 7:29 am
by pccwglobalit
hi any updates on this? thanks

Re: Instance status different in console and GUI

Posted: Thu Mar 19, 2015 9:46 am
by tmcdonald
pccwglobalit wrote:hi any updates on this? thanks
Did you check the things in jolson's post?

Re: Instance status different in console and GUI

Posted: Fri Mar 20, 2015 1:28 am
by pccwglobalit
all selinuxs are disabled.

Re: Instance status different in console and GUI

Posted: Fri Mar 20, 2015 12:46 pm
by jolson
Okay, it seems we're at a point where basic troubleshooting isn't leading us down the correct path. I'll need some more information so I better understand your setup.
Some questions I'd like you to answer in a detailed way are as follows:
When was this NLS cluster installed and setup? Was it working previously?
When did this stop working and were there any events that led up to it?

It looks like commands from the WebGUI are not reaching the cluster. This can mean a variety of things, but I'd like you to try the following:

Make sure cron is running on every server:

Code: Select all

systemctl status crond
Ensure that the nagios account isn't expired:

Code: Select all

chage -l nagios
Report cron logs to ensure it's functioning:

Code: Select all

tail -n20 /var/log/cron.d/log
Let me know. Thank you.

Re: Instance status different in console and GUI

Posted: Sun Mar 22, 2015 8:59 pm
by pccwglobalit
Okay, it seems we're at a point where basic troubleshooting isn't leading us down the correct path. I'll need some more information so I better understand your setup.
Some questions I'd like you to answer in a detailed way are as follows:
When was this NLS cluster installed and setup? Was it working previously?
When did this stop working and were there any events that led up to it?
we setup the NLS cluster on dec. it was working previously until we add nfs to server and change uid of nagios.
It looks like commands from the WebGUI are not reaching the cluster. This can mean a variety of things, but I'd like you to try the following:

Make sure cron is running on every server:
the crond is running

Code: Select all

systemctl status crond
Ensure that the nagios account isn't expired:
crond.service - Command Scheduler
Loaded: loaded (/usr/lib/systemd/system/crond.service; enabled)
Active: active (running) since Wed 2015-03-11 11:10:11 GMT; 1 weeks 4 days ago
Main PID: 8115 (crond)
CGroup: /system.slice/crond.service
__8115 /usr/sbin/crond -n

Mar 11 11:10:11 nls1-tmh2.it.pccwglobal.com systemd[1]: Started Command Sched...
Mar 11 11:10:11 nls1-tmh2.it.pccwglobal.com crond[8115]: (CRON) INFO (RANDOM_...
Mar 11 11:10:11 nls1-tmh2.it.pccwglobal.com crond[8115]: (CRON) INFO (running...
Mar 11 11:10:11 nls1-tmh2.it.pccwglobal.com crond[8115]: (CRON) INFO (@reboot...
Hint: Some lines were ellipsized, use -l to show in full.[/color]

Code: Select all

chage -l nagios
Report cron logs to ensure it's functioning:
the password is setup never expired

Code: Select all

Mar 23 01:48:01 nls1-tmh2 CROND[31296]: (nagios) CMD (/usr/bin/php -q /var/www/html/nagioslogserver/www/index.php jobs > /usr/local/nagioslogserver/var/jobs.log 2>&1)
Mar 23 01:48:01 nls1-tmh2 CROND[31297]: (nagios) CMD (/usr/bin/php -q /var/www/html/nagioslogserver/www/index.php poller > /usr/local/nagioslogserver/var/poller.log 2>&1)
Mar 23 01:49:01 nls1-tmh2 CROND[31366]: (nagios) CMD (/usr/bin/php -q /var/www/html/nagioslogserver/www/index.php poller > /usr/local/nagioslogserver/var/poller.log 2>&1)
Mar 23 01:49:01 nls1-tmh2 CROND[31367]: (nagios) CMD (/usr/bin/php -q /var/www/html/nagioslogserver/www/index.php jobs > /usr/local/nagioslogserver/var/jobs.log 2>&1)
Mar 23 01:50:01 nls1-tmh2 CROND[31540]: (root) CMD (/usr/lib64/sa/sa1 1 1)
Mar 23 01:50:01 nls1-tmh2 CROND[31541]: (nagios) CMD (/usr/bin/php -q /var/www/html/nagioslogserver/www/index.php jobs > /usr/local/nagioslogserver/var/jobs.log 2>&1)
Mar 23 01:50:01 nls1-tmh2 CROND[31542]: (nagios) CMD (/usr/bin/php -q /var/www/html/nagioslogserver/www/index.php poller > /usr/local/nagioslogserver/var/poller.log 2>&1)
Mar 23 01:51:01 nls1-tmh2 CROND[31584]: (nagios) CMD (/usr/bin/php -q /var/www/html/nagioslogserver/www/index.php jobs > /usr/local/nagioslogserver/var/jobs.log 2>&1)
Mar 23 01:51:01 nls1-tmh2 CROND[31585]: (nagios) CMD (/usr/bin/php -q /var/www/html/nagioslogserver/www/index.php poller > /usr/local/nagioslogserver/var/poller.log 2>&1)
Mar 23 01:52:01 nls1-tmh2 CROND[31654]: (nagios) CMD (/usr/bin/php -q /var/www/html/nagioslogserver/www/index.php poller > /usr/local/nagioslogserver/var/poller.log 2>&1)
Mar 23 01:52:01 nls1-tmh2 CROND[31655]: (nagios) CMD (/usr/bin/php -q /var/www/html/nagioslogserver/www/index.php jobs > /usr/local/nagioslogserver/var/jobs.log 2>&1)
Mar 23 01:53:01 nls1-tmh2 CROND[31794]: (nagios) CMD (/usr/bin/php -q /var/www/html/nagioslogserver/www/index.php poller > /usr/local/nagioslogserver/var/poller.log 2>&1)
Mar 23 01:53:01 nls1-tmh2 CROND[31795]: (nagios) CMD (/usr/bin/php -q /var/www/html/nagioslogserver/www/index.php jobs > /usr/local/nagioslogserver/var/jobs.log 2>&1)
Mar 23 01:54:01 nls1-tmh2 CROND[31935]: (nagios) CMD (/usr/bin/php -q /var/www/html/nagioslogserver/www/index.php jobs > /usr/local/nagioslogserver/var/jobs.log 2>&1)
Mar 23 01:54:01 nls1-tmh2 CROND[31936]: (nagios) CMD (/usr/bin/php -q /var/www/html/nagioslogserver/www/index.php poller > /usr/local/nagioslogserver/var/poller.log 2>&1)

Updating Cluster Hosts File
Updating Elasticsearch with instance...
sudo: unknown uid 8005: who are you?
sudo: unknown uid 8005: who are you?
Updating Cluster Hosts File
Updating Elasticsearch with instance...
sudo: unknown uid 8005: who are you?
sudo: unknown uid 8005: who are you?

Let me know. Thank you.

Re: Instance status different in console and GUI

Posted: Mon Mar 23, 2015 9:16 am
by jolson
Can you change the UID of nagios back to what it was originally as a test, and then restart elasticsearch and logstash?

Code: Select all

usermod -u <UID> nagios

Code: Select all

systemctl restart elasticsearch

Code: Select all

systemctl restart logstash

Re: Instance status different in console and GUI

Posted: Tue Mar 24, 2015 11:19 am
by pccwglobalit
After change usermod, now seems working. will change another one and confirm the status. thanks for your help.

Re: Instance status different in console and GUI

Posted: Tue Mar 24, 2015 11:34 am
by jolson
No problem - I will wait for your confirmation before locking this thread. Thanks!