NLSBackup Snapshots always IN PROGRESS

[teirekos]

As seen in attached screenshot my NLSBackup Snapshots from the 1st day that started has created only one snapshot and is always "IN Progress"?
Is this normal behavior?

[scottwilkerson]

What version of Nagios Log Server are you running?

How many instances are in the cluster?

Does every instance have access to the share /Backups ?

[teirekos]

I am running the latest i.e. ver 1.3

My cluster has 2 instances.

The /Backup is a local file system in my first server (instance). Both instances are in the same subnet.
Shouldn't though the 1st instance be able to create the snapshot?

But indeed in the 2nd instance I get the following:

[2015-03-01 07:01:51,217][WARN ][snapshots ] [845bc07c-ed91-4920-8e23-747c9cc699f5] failed to create snapshot [NLSBackup:logsta
sh-2015.01.31]
org.elasticsearch.repositories.RepositoryMissingException: [NLSBackup] missing
at org.elasticsearch.repositories.RepositoriesService.repository(RepositoriesService.java:266)
at org.elasticsearch.snapshots.SnapshotsService.beginSnapshot(SnapshotsService.java:264)
at org.elasticsearch.snapshots.SnapshotsService.access$600(SnapshotsService.java:88)
at org.elasticsearch.snapshots.SnapshotsService$1$1.run(SnapshotsService.java:202)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)

How does the 2nd instance tries to reach the missing repository i.e. how can I test that my 2nd instance can write in my 1st instance filesystem (if I understood well).

Thanx a lot.

[scottwilkerson]

A repository MUST be on a shared filesystem that ALL instances in the cluster have access to.

[teirekos]

What kind of share? Samba, ftp?

[jolson]

As long as the share can be mounted on your nodes, the type should not be extremely relevant. A samba share would be fine as long as it is mounted in the same place across all of your nodes. This mount location is what you would specify under "Administration > Backup & Maintenance".

More information can be found at the following URL:
http://assets.nagios.com/downloads/nagi ... enance.pdf

Please let us know if you encounter any issues.

[teirekos]

Thanx for your clarification. I've mounted the filesystem. Now I'll wait for a successful backup.

[tmcdonald]

Let us know how that turns out.

[teirekos]

I had to restart the server in order for my backup to start working.
is there any way to "restart" the backup schedule only?

[jolson]

"The backup process utilizes an elasticsearch addon called curator that performs all of the maintenance on existing indexes as well as creating the backups for all non-current day indexes."

This means that the backup process is controlled by Elasticsearch. I spun up my own instance of NLS for testing this case, and have used the following to get backups to begin:
-apply configuration from the UI
-restart elasticsearch service on both servers
-force the backup via command subsystem
-backups did not begin appearing in the /tmp directory (which I was using a test backup mount)
-noticed that the /tmp directory had root:root permissions - changed the permissions to nagios:nagios.
-ran an 'Apply Config' from the NLC GUI
-backups began appearing properly in /tmp/

I suggest checking your permissions on your mountpoint, be sure that the nagios user has access, and run an Apply Configuration.

Best,


Jesse