Core services - Could not authenticate, Invalid Token Given

[vAJ]

I've read through the other threads that have this message, nothing work for me.

Fresh source install on CentOS vm. Go to reconfigure data_dir to separate partition, move files from original location, start elastic search back up. All logins are gone, have to use the curl command to add "someuser" back in. System status shows both core services down with "Could not authenticate. Invalid Token Given"

On a side note, I really wish all of the product documentation was in wiki form, not individual PDFs. Do I need corner @FatherofNagios next month to get this in the works?

[jolson]

Go to reconfigure data_dir to separate partition, move files from original location

When you moved the files from the original location, are you certain that you moved *all* of them? I ask because the 'nagioslogserver' index is in control of the NLS login information, and if you don't move it before you start elasticsearch back up, elasticsearch is going to generate a new one without any credentials - this would result in the problem that you have.

Run the following command, and be sure that you have these indices: nagioslogserver, nagioslogserver_log, kibana-int.

Code: Select all

curl 'localhost:9200/_cluster/health?level=indices&pretty'

[tmcdonald]

On a side note, I really wish all of the product documentation was in wiki form, not individual PDFs. Do I need corner @FatherofNagios next month to get this in the works? :?:

We've had people fiercely for and against both PDF and wiki format for a long time. We're working on something, no need to corner anyone :)

[vAJ]

Code: Select all

[root@aus02gdnls01 logstash]# curl 'localhost:9200/_cluster/health?level=indices&pretty'
{
  "cluster_name" : "7f4cab6f-741e-454a-a588-027dce70bfcb",
  "status" : "yellow",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 6,
  "active_shards" : 6,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 6,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "indices" : {
    "nagioslogserver_log" : {
      "status" : "yellow",
      "number_of_shards" : 5,
      "number_of_replicas" : 1,
      "active_primary_shards" : 5,
      "active_shards" : 5,
      "relocating_shards" : 0,
      "initializing_shards" : 0,
      "unassigned_shards" : 5
    },
    "nagioslogserver" : {
      "status" : "yellow",
      "number_of_shards" : 1,
      "number_of_replicas" : 1,
      "active_primary_shards" : 1,
      "active_shards" : 1,
      "relocating_shards" : 0,
      "initializing_shards" : 0,
      "unassigned_shards" : 1
    }
  }
}

[vAJ]

From logstash.log:

Code: Select all

{:timestamp=>"2015-08-05T10:28:04.724000-0500", :message=>"Error: No config files found: /usr/local/nagioslogserver/logstash/etc/conf.d/*\nCan you make sure this path is a logstash config file?"}
{:timestamp=>"2015-08-05T10:28:04.731000-0500", :message=>"You may be interested in the '--configtest' flag which you can\nuse to validate logstash's configuration before you choose\nto restart a running system."}

I expect these as I haven't added any configs yet.

[jolson]

Code: Select all

[root@aus02gdnls01 logstash]# curl 'localhost:9200/_cluster/health?level=indices&pretty'
{
  "cluster_name" : "7f4cab6f-741e-454a-a588-027dce70bfcb",
  "status" : "yellow",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 6,
  "active_shards" : 6,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 6,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "indices" : {
    "nagioslogserver_log" : {
      "status" : "yellow",
      "number_of_shards" : 5,
      "number_of_replicas" : 1,
      "active_primary_shards" : 5,
      "active_shards" : 5,
      "relocating_shards" : 0,
      "initializing_shards" : 0,
      "unassigned_shards" : 5
    },
    "nagioslogserver" : {
      "status" : "yellow",
      "number_of_shards" : 1,
      "number_of_replicas" : 1,
      "active_primary_shards" : 1,
      "active_shards" : 1,
      "relocating_shards" : 0,
      "initializing_shards" : 0,
      "unassigned_shards" : 1
    }
  }
}

It looks like you only have 2 indices in your cluster - where is your new repository located? Can you verify that the 'nagios' user is capable of reading/writing to that repository?

Switch to 'nagios'.

Code: Select all

su - nagios

Cd to your new storage directory and make a file.

Code: Select all

cd /storage/dir
touch file
rm file

[vAJ]

Code: Select all

[root@aus02gdnls01 logstash]# su - nagios
This account is currently not available.

Didn't see that coming... Isn't fullinstall supposed to create nagios account?

[vAJ]

Code: Select all

nagios:x:498:497::/var/spool/nagios:/sbin/nologin

Ehhh, but it is there and set for nologin...

[jolson]

Please change nagios498:497::/var/spool/nagios:/sbin/nologin to nagios498:497::/var/spool/nagios:/bin/bash

At this point, you should be able to log in as the nagios user. It looks like the nagios user wasn't implemented properly.

[vAJ]

OK, I switch to nagios user and was able to create and remove files in /data01/elasticsearch/data/

ls of dir shows:
drwxr-xr-x 3 nagios nagios 4096 Aug 4 16:06 7f4cab6f-741e-454a-a588-027dce70bfcb
drwxr-xr-x 3 nagios nagios 4096 Aug 4 10:19 nodes