Whoops!, was working fine, then it lost its configs

Support forum for Nagios Core, Nagios Plugins, NCPA, NRPE, NSCA, NDOUtils and more. Engage with the community of users including those using the open source solutions.
AcademyNag
Posts: 19
Joined: Wed Jun 12, 2013 2:26 pm
Location: Houston, Texas
Contact:

Re: Whoops!, was working fine, then it lost its configs

Post by AcademyNag »

Update-
It 'sort-of' happened again... I get the same first message, I am ssh'd into the server:
- as root, # ps -fe | grep nagios show that it is running, and it is making it's server/hosts checks.
- as nagios, > nagios -v /usr/local/nagios/etc/nagios.cfg shows NO errors, 1 warning:
Warning: Host 'padbogn02' has no services associated with it! (doesn't matter for that server atm...)
> tailf var/nagios.log shows normal

Its the web interface that can not see the service/host status'
Where do I go from here? what can I provide?

also-
# /etc/init.d/nagios status
nagios (pid 8684) is running...
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: Whoops!, was working fine, then it lost its configs

Post by scottwilkerson »

Are you logging in as a user that is a contact for any of the hosts on the system?

If not as slansing had mentioned before, you need to add them to the appropriate items in the cgi.cfg, ie.

Code: Select all

authorized_for_all_hosts=nagiosadmin,yourusernamehere
Former Nagios employee
Creator:
ahumandesign.com
enneagrams.com
AcademyNag
Posts: 19
Joined: Wed Jun 12, 2013 2:26 pm
Location: Houston, Texas
Contact:

Re: Whoops!, was working fine, then it lost its configs

Post by AcademyNag »

OK,,, after I reset the system back to befor this occurred, - all was well, however, with no config change, it stopped, what you are referring to I will definately do, but, that is a symptom, I need to fix the cause, thats where i am looking. I will "manually" put my username there, though it is supposed to check by group, and provide the results and any logs, confs, or outputs yu wish I just want to get to the issue here.
and again-- Thank you for your replies and any/all help

fyi all users log in via their username authenticated by ldap, and some groups have more access than others of course.
slansing
Posts: 7698
Joined: Mon Apr 23, 2012 4:28 pm
Location: Travelling through time and space...

Re: Whoops!, was working fine, then it lost its configs

Post by slansing »

though it is supposed to check by group, and provide the results and any logs, confs, or outputs yu wish I just want to get to the issue here.
As scott and I had both mentioned the way this feature works is you HAVE to have your user added to the correct lines here. True, you can add a contact to hosts/services to give them viewing access but that does not have to do with this.. to my knowledge. You are probably seeing some users have access to certain hosts/services, that us because their contact is assigned to them.
AcademyNag
Posts: 19
Joined: Wed Jun 12, 2013 2:26 pm
Location: Houston, Texas
Contact:

Re: Whoops!, was working fine, then it lost its configs

Post by AcademyNag »

For now- I went back to the working snapshot- so I have a working system, in order to find what is causing this issue, i have created the following: (on my own box, not the server)

Gathering Info
Crontab-
55 * * * * root /root/rsync-bak.sh

rsync-bak.sh-
#### omitted variables...............
# Actions-
rsync -ave ssh plapnag01e:/var/log/apache2 /root/naglog/apache2nag01 2>> $TEMPLOG
rsync -ave ssh plapnag01e:/usr/local/nagios/var/nagios.log /root/naglog/ 2>> $TEMPLOG
rsync -ave ssh plapnag01e:/usr/local/nagios/var/archives/nagios-06*2013-00.log /root/naglog/ 2>> $TEMPLOG
rsync -ave ssh plapnag01e:/usr/local/nagios/etc/*.cfg /root/naglog/ 2>> $TEMPLOG
tar -czvf naglog-$TDATE.tar.gz naglog

#### omitted variables...............

Contents of naglog:

actions.txt
apache2nag01
cgi.cfg
nagios-06-19-2013-00.log
nagios-06-20-2013-00.log
nagios-06-21-2013-00.log
nagios-06-22-2013-00.log
nagios-06-23-2013-00.log
nagios.cfg
nagios.log
nag-working-smiller.png
resource.cfg

naglog/apache2nag01:
apache2

naglog/apache2nag01/apache2:
access_log
access_log-20130618.bz2
access_log-20130619.bz2
access_log-20130621.bz2
access_log-20130622.bz2
access_log-20130625.bz2
error_log
error_log-20130618.bz2
rcapache2.out

I can attach this tarball, or any part of it at any time. If I need to add anything to the rsync- I can. The "nag-working-smiller.png" is a SS of the working system cgi, note the 'logged in as smiller'. If/When it fails again, I will compare the logs/cfg's, and- I'll take a SS of the Failed state.
I can also produce the entire "rsync-bak.sh" if you have questions, or would just like a giggle at my 'mad-bash-skills' :P (or lack thereof)
abrist
Red Shirt
Posts: 8334
Joined: Thu Nov 15, 2012 1:20 pm

Re: Whoops!, was working fine, then it lost its configs

Post by abrist »

Let us know if the issue recurs.
Former Nagios employee
"It is turtles. All. The. Way. Down. . . .and maybe an elephant or two."
VI VI VI - The editor of the Beast!
Come to the Dark Side.
AcademyNag
Posts: 19
Joined: Wed Jun 12, 2013 2:26 pm
Location: Houston, Texas
Contact:

Re: Whoops!, was working fine, then it lost its configs

Post by AcademyNag »

It started again, attaching screenshot, let me know what other logs to attach- I already did a diff of the .cfg files in etc...
nagios is running, and is checking the systems it monitors, just not alerting or notifying and no cgi- see attachment-
Attachments
I get this screen no matter
I get this screen no matter
abrist
Red Shirt
Posts: 8334
Joined: Thu Nov 15, 2012 1:20 pm

Re: Whoops!, was working fine, then it lost its configs

Post by abrist »

Does your config verify?

Code: Select all

/usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg
Is nagios running?

Code: Select all

service nagios status
Former Nagios employee
"It is turtles. All. The. Way. Down. . . .and maybe an elephant or two."
VI VI VI - The editor of the Beast!
Come to the Dark Side.
AcademyNag
Posts: 19
Joined: Wed Jun 12, 2013 2:26 pm
Location: Houston, Texas
Contact:

Re: Whoops!, was working fine, then it lost its configs

Post by AcademyNag »

abrist ,
Thank you for your time and efforts, I found the issue--
It was a faulty configuration between the keyboard and the chair- (ID10T-error)
the '/usr' partition was full- so when changes were made and you tried to write the configs- it tried, then errored, and left the cgi not knowing what to do with it :(
(Who watches the watcher? :P )

Thanks again for all your help and efforts!
I'll go away quietly now....... :D
Locked