Maintenance run has no effect

[tonnag]

Hi,

At home I am running a free nagios log server which is more than enough there. It states "Your license key is valid and activated"

Now, Under "Snapshot & Maintenance" I specified

Code: Select all

Optimize Indexes older than 	 2 days
Close indexes older than 	14 days
Delete indexes older than 	32 days
Repository to store snapshots in 	You must first create a repository on the right.
Delete snapshots older than 	You must first create a repository on the right.
Enable Maintenance and Snapshots	Yes 
Last modified 	Mon, 22 Apr 2019 17:31:04 +0200

System Jobs status says it does run

Job ID Job Status Last Run Status Last Run Time Frequency Next Run Time Type Actions
cleanup_cmdsubsys Waiting SUCCESS 04/28/2019 17:18:46 1 hour 04/28/2019 18:18:46 System Edit Run
backups Waiting SUCCESS 04/27/2019 23:35:41 1 day 04/28/2019 23:35:41 System Edit Run
snapshots_maintenance Waiting SUCCESS 04/28/2019 06:34:11 1 day 04/29/2019 06:34:11 System Edit Run
run_all_alerts Waiting SUCCESS 04/28/2019 17:59:32 20 seconds 04/28/2019 17:59:52 System Edit Run
run_update_check Waiting SUCCESS 04/27/2019 23:36:41 1 day 04/28/2019 23:36:41 System Edit Run

but no indexes get closed nor deleted. Did I miss something here ?

[scottwilkerson]

How many indexes do you have open with dates less than today?

[tonnag]

that will be around 45

[scottwilkerson]

If you go to Admin -> Audit Log
and filter the Type to MAINTENANCE

Do you see any data there?

[tonnag]

Yes i do:

Created By Type Created At Message Instance ID Source
System MAINTENANCE Mon, 29 Apr 2019 18:02:01 +0200 Deleting indexes more than 32 day(s) old. Nagios Log Server
System MAINTENANCE Mon, 29 Apr 2019 18:02:01 +0200 Closing indexes 14 day(s) old. Nagios Log Server
System MAINTENANCE Mon, 29 Apr 2019 18:02:01 +0200 Optimizing indexes 2 day(s) old. Nagios Log Server

and Index Overview:
47 Indices
newest: logstash-2019.04.30
oldest: logstash-2019.03.15

[scottwilkerson]

Can I have you run the following in the CLI

Code: Select all

tail -f /usr/local/nagioslogserver/var/jobs.log

after doing so, Go to Admin -> Command Subsystem
click run next to snapshots_maintenance

and watch the log we are tailing looking for errors.

This may take a minute or 2 to complete

If there are any errors, post them back here

[tonnag]

Error:

Code: Select all

-----
/usr/local/nagioslogserver/scripts/curator.sh: 3: /usr/local/nagioslogserver/scripts/curator.sh: curator: not found
-----

chaged /usr/local/nagioslogserver/scripts/curator.sh to

Code: Select all

#!/bin/sh

# curator "$@"
/usr/local/bin/curator "$@"

and that seems to work. Got 32 indexes of which 14 are open

/etc/os-release

PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
NAME="Debian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

[scottwilkerson]

Glad you were able to resolve the issue.

It looks line /usr/local/bin might have bee removed from the PATH at least for the nagios user

Can you show the output of

Code: Select all

sudo su nagios -c "echo $PATH"

[tonnag]

Thnx Scott.

Doesn't look like the root cause,

Code: Select all

$ which curator
/usr/local/bin/curator

$ sudo su nagios -c "echo $PATH"
/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games

but as you said, it works now....i'll sort out the details later, the work-around is fine for now. Thnx for the help.

[scottwilkerson]

Thnx Scott.

Doesn't look like the root cause,

Code: Select all

$ which curator
/usr/local/bin/curator

$ sudo su nagios -c "echo $PATH"
/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games

but as you said, it works now....i'll sort out the details later, the work-around is fine for now. Thnx for the help.

Glad to help

Locking thread