multiple snapshots occurring overnight

[bpennin]

Nagios XI Version 5.6.6 on Rhel 7

We are seeing multiple unexplained snapshots occurring around the same time every night. I have searched through the audit logs, system logs, Nagios XI logs, and the forum with no luck. I have attached screenshot. I increased our number of snapshots allowed until we figure this out.

[jbrunkow]

Do you see any backup scheduled in the Scheduled Backups page of XI? You can find the page by navigating to the Admin section using the bar on top > then clicking Scheduled Backups under the System Backups section on the left pane.

What information was recorded to the Nagios log during that time frame?

Code: Select all

/usr/local/nagios/var/nagios.log

You could also look in the following log for that same time to see if it contains any clues.

Code: Select all

/usr/local/nagiosxi/var/eventman.log

It never hurts to send ( PM ) us a profile as well. You can download the profile by navigating to System Profile under System Config on the left pane > and clicking the Download Profile button on that page.

I hope that helps! Let me know if you have any more questions.

[bpennin]

No backups are scheduled during those times and nothing stands out in the logs. Is there a way we could look for something coming in from the API? I have a feeling it is API related...we have some batch-type processes to add and remove hosts but they are scheduled for noon and midnight. Also, we had 3 more snapshots at 840 ish again last night.

I tried to run a system profile and get the following:
PROFILE BUILD FAILED
Array
(
)
CODE: 1

[bpennin]

I ran the profile from the command line to get around what looks like sudo file errors.

[ssax]

See here for the profile build failed issue:

https://support.nagios.com/kb/article.p ... ategory=44

The file you attached is just your configs, the profile.zip that the web UI generates includes a lot of additional stuff that we need to see as well.

Please check in Admin > Audit Log around that time, do you see anything related? That's where you'd see it occurring.

If you don't see it in there you should be able to look in your apache logs in one of these files it should show the requests and where they came from:

Code: Select all

/var/log/httpd/access_log
/var/log/httpd/request_log
/var/log/httpd/ssl_access_log
/var/log/httpd/ssl_request_log

Since the snapshot times are so close together this would likely be intentional or called through the API as you were suspecting. The above info should clear up where it came from.

[jbrunkow]

Apparently,

This problem is occurring because the required sudoers entries are missing...

Please take a look at the following knowledge base article for detailed instructions on how to resolve this error.
PROFILE BACKUP FAILED KB

COMMANDS =

Code: Select all

cd /tmp
wget https://assets.nagios.com/downloads/nagiosxi/5/xi-5.6.3.tar.gz
tar xzf xi-5.6.3.tar.gz nagiosxi/nagiosxi/nagiosxi.sudoers --strip-components 2
grep -v NAGIOSXI /etc/sudoers > /etc/sudoers.new
mv -f /etc/sudoers.new /etc/sudoers
rm -rf /etc/sudoers.d/nagiosxi
sed -i 's/^Defaults    requiretty/#Defaults    requiretty/g' /etc/sudoers
cat /tmp/nagiosxi.sudoers >> /etc/sudoers
chmod 440 /etc/sudoers

[bpennin]

Nothing in the apache logs at those times. I had the developer run his API while I watched and could see his apply config. I do not see any apply configs during those 8pm time frames in any log - including audit log in Nagios. We do have Nagios Fusion in our environment - could that be doing anything like this? We have a log server cluster, and 2 other Nagios servers fused to it. I did get the profile to run and its included. Still confused.

[bpennin]

Here is the profile


Moderator's Note: The profile has been shared with the support team but has been removed from the public forum.

[jbrunkow]

Thankyou for including that. I will examine the profile to see if I can determine why it is inexplicably creating midnight backups.

[bpennin]

Any status update?