NLS Single node instance - cluster status is RED

[lukedevon]

Hi

I have a single node NLS and I configured it according to the following guide.

https://assets.nagios.com/downloads/nag ... oyment.pdf

But currently I am facing for a data inconsistency issue. Even though rsyslog injecting the logs to nls, I am unable to view them over the dashboard.

Just to troubleshoot , I have followed following guide too.

https://support.nagios.com/kb/article.php?id=90

{
"cluster_name" : "0488e2d8-30d1-4a98-b7bb-17b691908a74",
"status" : "red",
"timed_out" : false,
"number_of_nodes" : 1,
"number_of_data_nodes" : 1,
"active_primary_shards" : 931,
"active_shards" : 931,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 1,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 2,
"number_of_in_flight_fetch" : 0
}

curl -s -XGET http://localhost:9200/_cat/shards?v | egrep 'UNASSIGNED|INITIALIZING'

nagioslogserver_log 1 p UNASSIGNED

Currently cluster status is "Red". Please see the attached image.

Status.PNG

Could you please help me to fix this issue?

Thank you
Luke.

[cdienger]

Please PM me profile from the system. It can be gathered under Admin > System > System Status > Download System Profile or from the command line with:

/usr/local/nagioslogserver/scripts/profile.sh

This will create /tmp/system-profile.tar.gz.

Note that this file can be very large and may not be able to be uploaded. This is usually due to the logs in the Logstash and/or Elasticseach directories found in it. If it is too large, please open the profile, extract these directories/files and send them separately.

[lukedevon]

Hi

I already PM you the system profile.

Thanks
Luke.

[lukedevon]

Hi

I uploaded the files. but I'm not sure I have correctly uploaded. If the files have not been uploaded correctly, Please let me know what logs you need to be captured from the server side ? also please let me know what commands I have to execute.

If the indexes have been corrupted, how can we re-indexing those all the corrupted indexes?

Thanks
Luke

[cdienger]

The unassigned primary shard for the nagioslogserver_log is the reason you see the RED status. Restarting elasticsearch _may_ help fix it:

Code: Select all

service elasticsearch restat

If this doesn't help then you can try restoring from a backup. From page 8 of https://assets.nagios.com/downloads/nag ... Server.pdf:

To restore the system backup execute the following commands in the terminal session:

cd /usr/local/nagioslogserver/scripts/
./restore_backup.sh /store/backups/nagioslogserver/nagioslogserver.2017-05-10.1494373596.tar.gz

You can see that the backup file used in this example is nagioslogserver.2017-05-10.1494373596.tar.gz, you will need to change this to match the name of your system backup.

The files you tried to PM didn't make it through. If the above doesn't help, try sending again or upload them somewhere and share the link.