Snapshots & Maintenance off of a Ext4 Partition?

[tvoll]

I have created a separate ext4 partition specifically for the snapshots and maintenance feature on Nagios Log Server.
Is this acceptable? Unfortunately, even though the drive is mounted and has correct permissions for the nagios user, none of the snapshots are being created; however, the system command for snapshot_maintenance shows that it has been running successfully as planned.

[ssax]

Yes, that should work, but it's recommended to be stored off-server in the event of a disaster, it also need to be mounted/accessible on all nodes.

Please send a screenshot of your Snapshot and Maintenance settings as well.

Please post the output of the permissions:

Code: Select all

ls -l /path/to/your/dir

[tvoll]

Code: Select all

[tvollscw@logserver ~]$ ls -la /mnt/logs-backup/
total 8
drwxrwxr-x  2 nagios nagios 4096 Mar 25 14:04 .
drwxr-xr-x. 4 root   root   4096 Mar 25 14:04 ..

Screenshot of Config Attached

[ssax]

If you go to Admin > Command Subsystem and click the Reset All Jobs button, do they start showing up?

[tvoll]

If you go to Admin > Command Subsystem and click the Reset All Jobs button, do they start showing up?

No. Still showing 'No backups yet!' under the Snapshots tab on the Snapshots & Maintenance page.

[ssax]

Is this system in a cluster with multiple nodes? The reason I ask is because the backup location will need to be available on all nodes.

Please send me a copy of your profile, you can download it from Admin > System Status by clicking the Download System Profile button.

[tvoll]

Only on one node. For some reason, when I try to download the system profile, it loads for 30 minutes and doesn't do anything..

Is this system in a cluster with multiple nodes? The reason I ask is because the backup location will need to be available on all nodes.

Please send me a copy of your profile, you can download it from Admin > System Status by clicking the Download System Profile button.

[ssax]

Do you see anything in your /var/log/httpd/error_log or /var/log/httpd/ssl_error_log that could be related?

What about in these directories, any really big files?

Code: Select all

/var/log/elasticsearch
/var/log/logstash

If so, what do they contain?

Code: Select all

tail -n100 /path/to/log

Do you see any errors in /var/log/cron or /var/log/messages that could be related?

What is the output of these commands?

Code: Select all

chage -l nagios
grep nag /etc/group

[tvoll]

No big files in /var/log/elasticsearch, but consistently large files in the logstash log folder. Current size being over 40GB.
Last 100 lines copied and attached to this post.

I did not see any noticeable errors in the cron or messages file(s).

[ssax]

Please run this command as root (and leave it running):

Code: Select all

tail -Fn0 /usr/local/nagioslogserver/var/jobs.log

Then go to Admin > System > Command Subsystem and click Run next to snapshots_maintenance

Wait for the job to run/fail, send us the full output of the tail command.