Page 1 of 2
Stuck in Applying Changes
Posted: Wed May 17, 2017 9:48 am
by SavaSC
Hello,
I'm having issues with my instance of NagiosXI v5.4.3 not applying changes. It just sits there with the eternal "Waiting for Configuration Verification...." showing. It never fails out, even over night, just shows that one screen.
How do I check to see what is causing this?
Thanks!
Re: Stuck in Applying Changes
Posted: Wed May 17, 2017 9:52 am
by dwhitfield
https://support.nagios.com/kb/article.php?id=34 will probably take care of it.
If not, can you PM me your Profile? You can download it by going to Admin > System Config > System Profile and click the ***Download Profile*** button towards the top. If for whatever reason you *cannot* download the profile, please put the output of View System Info (5.3.4+, Show Profile if older) in the thread (that will at least get us some info). This will give us access to many of the logs we would otherwise ask for individually. If security is a concern, you can unzip the profile take out what you like, and then zip it up again. We may end up needing something you remove, but we can ask for that specifically.
After you PM the profile, please update this thread. Updating this thread is the only way for it to show back up on our dashboard.
Re: Stuck in Applying Changes
Posted: Wed May 17, 2017 10:40 am
by SavaSC
I think I found the issue, /dev/mapper/nagios_vg00-usr_local_lv is full.
This shows to be mounted to /usr/local. What do I need to clean up to clear some space?
Thanks!
Re: Stuck in Applying Changes
Posted: Wed May 17, 2017 10:44 am
by dwhitfield
Impossible to say at this point. Increasing the VM size may be of use:
https://assets.nagios.com/downloads/nag ... M-Disk.pdf
Also,
https://support.nagios.com/kb/article.php?id=26 has some pointers of disk space. I'm not sure if /var/log is going to be super useful though if it's /usr/local. Instead, I'd use
du -a /usr/local | sort -n -r | head -n 10
Please let us know if you have any questions on the consequences of moving specific data.
Re: Stuck in Applying Changes
Posted: Wed May 17, 2017 3:25 pm
by SavaSC
Well, I went through and deleted everything before 2016 of the archive data in /usr/local/nagios/var/archives. That got me back 3% of that disk space. Nagios immediately sighed with relief.
As we'll be migrating off of this server soon, hopefully I won't be needing any more space before then.
Thanks for your help. You can close this thread.
Re: Stuck in Applying Changes
Posted: Wed May 17, 2017 3:31 pm
by dwhitfield
I'm happy to close, but I'm curious how much 3% is and how soon you are migrating off. If you look at the size of some of your recent log files, it would be pretty simple math to figure out if you need to delete more or increase space before the migration date.
I'm aware this assumes you know when the migration is actually happen and that may not be in your control, but I offer the thought in case it saves you some pain down the road.
Re: Stuck in Applying Changes
Posted: Thu May 18, 2017 8:56 am
by SavaSC
Currently, that mount point is showing 317MB free. (I have attached a screenshot.) The switch will be within the next couple of weeks. I already have the new box built, just jumping through management hoops to migrate off of one onto the other.
Re: Stuck in Applying Changes
Posted: Thu May 18, 2017 9:28 am
by dwhitfield
The following will give you the average size of your nagios logs per day in KB.
Code: Select all
cd /usr/local/nagios/var/archives
ls -lh | gawk '{sum += $5; n++;} END {print sum/n;}'
If you want bytes, just remove the
h. Those are definitely not the only logs, but it should give you an obvious answer if you do not have enough size.
For that particularly mount point,
/usr/local/nagios/share/perfdata is likely to be a big one, but the perfdata is stuck in individual directories, so the command above won't be of much help.
du will be useful for the directories, but it's not going to tell you how much is being added per day like the logs do.
Re: Stuck in Applying Changes
Posted: Thu May 18, 2017 10:19 am
by SavaSC
Hmmmm..... You bring in an important point: There is data in the /usr/local/nagios/share/perfdata on machines that are no longer active. Is there an easy way to remove any inactive machine's data from this location? Sure, I could go through by hand. But, wow, that would be a very arduous process.
Re: Stuck in Applying Changes
Posted: Thu May 18, 2017 10:33 am
by dwhitfield
SavaSC wrote:Is there an easy way to remove any inactive machine's data from this location?
As with many questions regarding XI...it depends. It doesn't seem particularly complicated, but I don't have a ready-made script for it, so it could certainly be *easier*.
What I'd do is
diff the output of
ll/usr/local/nagios/etc/hosts with the output of
ll /usr/local/nagios/share/perfdata. You can get rid of all of the .cfg from the
ll/usr/local/nagios/etc/hosts output with
basename, but you'd need to write a loop to do that.
You'll need to modify this a bit, but once you have your list, something like the following should get rid of the directories no longer in use.
Code: Select all
#!/bin/bash
keep_no=$(( $1+1 ))
directory="./mydirec/"
cd $directory
rm -rf `find . -maxdepth 1 -mindepth 1 -type d -printf '%f\n'| sort -nr | tail -n +$keep_no`
cd -