Page 1 of 2

Nagios disk full and conf changes doesnt apply

Posted: Mon Mar 03, 2014 10:00 am
by Loul42
Hello

Our partition / is full because of /usr/local/nagios/var/spool/perfdata (performance data)
Since we moved the content to another folder in /data we had issue with nagios components (see picture)

And now / is full again .
I guess we need to free some space on /
Also we can't add new hosts conf changes don't apply..

Should we use mount –o bind on /data to make some space on / ?
and then if we restart nagios everything should go back to normal ?

Thanks in advance

Re: Nagios disk full and conf changes doesnt apply

Posted: Mon Mar 03, 2014 11:45 am
by slansing
Yes, XI is looking at those directories which you likely removed/moved to a different location, you are going to need to move your performance data back to where it was, so we can trouble shoot why it was so full, it is likely that your performance data was not being processes, how much space have you allocated to your XI server? If it is on a VM try this:

http://assets.nagios.com/downloads/nagi ... M_Disk.pdf

What is the error you are seeing when applying configuration? When you mouse over those component icons, what do they tell you?

Re: Nagios disk full and conf changes doesnt apply

Posted: Tue Mar 04, 2014 3:44 am
by Loul42
What I didn’t tell you it’s that it’s again full so I can’t put it back as / is already full
We moved it in another folder at the beginning.
So I don’t know what could we do.
Where are the nagios backup that should I save somewhere ?

When I click on apply configuration
The log end like this :

Read object config files okay...

Running pre-flight check on configuration data...

Checking services...
Checked 2135 services.
Checking hosts...
Checked 227 hosts.
Checking host groups...
Checked 28 host groups.
Checking service groups...
Checked 0 service groups.
Checking contacts...
Checked 39 contacts.
Checking contact groups...
Checked 9 contact groups.
Checking service escalations...
Checked 0 service escalations.
Checking service dependencies...
Checked 0 service dependencies.
Checking host escalations...
Checked 0 host escalations.
Checking host dependencies...
Checked 0 host dependencies.
Checking commands...
Checked 124 commands.
Checking time periods...
Checked 20 time periods.
Checking for circular paths between hosts...
Checking for circular host and service dependencies...
Checking global event handlers...
Checking obsessive compulsive processor commands...
Checking misc settings...

Total Warnings: 0
Total Errors: 0

Things look okay - No serious problems were detected during the pre-flight check
-And something is missing there-
Because nothing applies

When I mouse over on
Performance Grapher : NPCD not running
Database BackEnd : ndo2db not running but subsystem locked
Database Maintenance : Last Run 1d7h40 ago
Event Data : Last Event Data transfer was 24d 22h ago
Command SUbSystem : Last Run 20h ago
Cleaner : Last Run 9h ago
Nonstop Operations Manager : Last run 7h39
System Statistique : Last updated 9h42 ago

Re: Nagios disk full and conf changes doesnt apply

Posted: Tue Mar 04, 2014 7:22 am
by Loul42
also my nagios.cfg is empty !!
I dont know why

Do you know where I can foind a backup of this file ?

Thanks

Re: Nagios disk full and conf changes doesnt apply

Posted: Tue Mar 04, 2014 3:39 pm
by lmiltchev
Is this a VM? Do you have any snapshots that you can revert to? If this is not an option, you may try to restore from an old backup. Do you have any backups in the "/store/backups/nagiosxi/"? You will probably need to spin a new VM, transfer your backup from the old server, and restore from the backup.

http://assets.nagios.com/downloads/nagi ... ing_XI.pdf

Re: Nagios disk full and conf changes doesnt apply

Posted: Wed Mar 05, 2014 3:06 am
by Loul42
Yes it's a VM but we don't have any recent backup of the vm
Also i got Nothing in "/store/backups/nagiosxi"

the only thing i have is
data\store\backups\mysql\
data\store\backups\postgresql\

With folders inside like : monthly daily weekly and i got also these folders : nagios nagioql nagiosxi mysql and some others

Can I backup nagios with that ?


Thanks

Re: Nagios disk full and conf changes doesnt apply

Posted: Wed Mar 05, 2014 11:21 am
by sreinhardt
Yes, copying those, plugins, and your rrd files off the system would basically be a full backup. Then you can proceed with the document to extend the VM disk size.

Re: Nagios disk full and conf changes doesnt apply

Posted: Thu Mar 06, 2014 12:08 pm
by Loul42
Thanks I copied it

But how with this files can i backup Nagios ?
Do you have documentation ?
So iI only have to save plugins and rrd files ? Where can i find rrd files ?
Host are already saved ?

Thanks a lot

Re: Nagios disk full and conf changes doesnt apply

Posted: Thu Mar 06, 2014 4:11 pm
by slansing
You can use this to grab as much as possible:

http://assets.nagios.com/downloads/nagi ... ing_XI.pdf

You could also alter the script to grab specific things, and leave others.

Re: Nagios disk full and conf changes doesnt apply

Posted: Tue Mar 11, 2014 5:21 am
by Loul42
Hello

Can I grab everything except mysql and postgresql backup (today). And then add mysql and postgres January backup to the zip and then restore this zip.
Will it work ? (As I want my system be backupped as it was in January, but I only have mysql and postgre backupped)