Nagios Support Forum

Hello @HIINNS

Thanks for following up, were you able to find the path location on the 'elasticsearch' service executable? I notice that on my test VM it is located in and can start with:

/etc/init.d/elasticsearch start

Circling back we see that you have a running java process with elasticsearch parms:

ps -aux | grep -Ei 'java'
nagios 16768 0.3 52.3 16131124 12843288 ? SLl Nov09 37:12 /bin/java -Xms11973m -Xmx11973m -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8 -Des.cluster.name=c0d6d20a-0a3c-4d14-9cda-1f2f4fcb6b55 -Des.node.name=bec4d3fc-0bab-49f5-88cf-fb1094c85cfd -Des.discovery.zen.ping.unicast.hosts=localhost -Des.path.repo=/ -Delasticsearch -Des.pidfile=/var/run/elasticsearch/elasticsearch.pid -Des.path.home=/usr/local/nagioslogserver/elasticsearch -cp :/usr/local/nagioslogserver/elasticsearch/lib/elasticsearch-1.7.6.jar:/usr/local/nagioslogserver/elasticsearch/lib/*:/usr/local/nagioslogserver/elasticsearch/lib/sigar/* -Des.default.path.home=/usr/local/nagioslogserver/elasticsearch -Des.default.path.logs=/var/log/elasticsearch -Des.default.path.data=/usr/local/nagioslogserver/elasticsearch/data -Des.default.path.work=/usr/local/nagioslogserver/tmp/elasticsearch -Des.default.path.conf=/usr/local/nagioslogserver/elasticsearch/config org.elasticsearch.bootstrap.Elasticsearch

The java process listed is running elasticsearch with parms, so we should see that elasticsearch is serving requests (using curl on Linux). To verify:
Let us know what you are seeing,
Perry

sudo curl -XGET http://localhost:9200/_status?pretty
Last login: Fri Nov 19 05:20:02 EST 2021 on cron
{
"_shards" : {
"total" : 0,
"successful" : 0,
"failed" : 0
},
"indices" : { }

sudo curl -XGET 'localhost:9200/_cluster/health?pretty'
Last login: Fri Nov 19 05:25:26 EST 2021 on pts/0
{
"cluster_name" : "c0d6d20a-0a3c-4d14-9cda-1f2f4fcb6b55",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 1,
"number_of_data_nodes" : 1,
"active_primary_shards" : 0,
"active_shards" : 0,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0
}

Hello @HIINNS

Thanks for following up with the results on the api's, which confirm that elasticsearch, and logstash are functioning set with status green.

Minus the elasticsearch, since that is running process java with parameters, let's verify that Apache and logstash are running:
or Verify that elasticsearch is running: That looks good, bring up the web console: https://yourhostaddress/nagioslogserver

Let us know how that looks,
Perry

sudo systemctl status httpd logstash
Last login: Mon Nov 22 05:30:04 EST 2021
● httpd.service - The Apache HTTP Server
Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: disabled)
Active: active (running) since Mon 2021-11-08 12:19:57 EST; 1 weeks 6 days ago
Docs: man:httpd(8)
man:apachectl(8)
Process: 9890 ExecReload=/usr/sbin/httpd $OPTIONS -k graceful (code=exited, status=0/SUCCESS)
Main PID: 18084 (httpd)
Status: "Total requests: 0; Current requests/sec: 0; Current traffic: 0 B/sec"
CGroup: /system.slice/httpd.service
├─ 9895 /usr/sbin/httpd -DFOREGROUND
├─ 9896 /usr/sbin/httpd -DFOREGROUND
├─ 9897 /usr/sbin/httpd -DFOREGROUND
├─ 9898 /usr/sbin/httpd -DFOREGROUND
├─ 9899 /usr/sbin/httpd -DFOREGROUND
└─18084 /usr/sbin/httpd -DFOREGROUND

Nov 22 03:34:01 nnagsr10 systemd[1]: Reloading The Apache HTTP Server.
Nov 22 03:34:02 nnagsr10 systemd[1]: Reloaded The Apache HTTP Server.

● logstash.service - LSB: Logstash
Loaded: loaded (/etc/rc.d/init.d/logstash; bad; vendor preset: disabled)
Active: active (exited) since Mon 2021-11-08 12:20:02 EST; 1 weeks 6 days ago
Docs: man:systemd-sysv-generator(8)
Process: 18156 ExecStart=/etc/rc.d/init.d/logstash start (code=exited, status=0/SUCCESS)

ps -aux | grep -Ei 'elasticsearch'
myuid 5334 0.0 0.0 112812 1008 pts/0 S+ 06:43 0:00 grep --color=auto -Ei elasticsearch
nagios 16768 0.3 52.4 16131124 12861728 ? SLl Nov09 63:24 /bin/java -Xms11973m -Xmx11973m -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8 -Des.cluster.name=c0d6d20a-0a3c-4d14-9cda-1f2f4fcb6b55 -Des.node.name=bec4d3fc-0bab-49f5-88cf-fb1094c85cfd -Des.discovery.zen.ping.unicast.hosts=localhost -Des.path.repo=/ -Delasticsearch -Des.pidfile=/var/run/elasticsearch/elasticsearch.pid -Des.path.home=/usr/local/nagioslogserver/elasticsearch -cp :/usr/local/nagioslogserver/elasticsearch/lib/elasticsearch-1.7.6.jar:/usr/local/nagioslogserver/elasticsearch/lib/*:/usr/local/nagioslogserver/elasticsearch/lib/sigar/* -Des.default.path.home=/usr/local/nagioslogserver/elasticsearch -Des.default.path.logs=/var/log/elasticsearch -Des.default.path.data=/usr/local/nagioslogserver/elasticsearch/data -Des.default.path.work=/usr/local/nagioslogserver/tmp/elasticsearch -Des.default.path.conf=/usr/local/nagioslogserver/elasticsearch/config org.elasticsearch.bootstrap.Elasticsearch

Hello @HINNS

That looks good, what do you see when you bring up the web console: https://yourhostaddress/nagioslogserver

• Are you able to view and add Log Sources?

Please send over the System Profile if see that you are unable to access the log server web console.

Hmmm… can't reach this page
myserver refused to connect.

Check PM, let me know if the file doesn't make it.

Hello @HIINNS

Thanks for following up, looks like the Profile did not make the trip. May need to use the split command to size down the compressed Profile.
Please send each 'part[x]' in a separate PM.

The following api command will provide connection feedback on the connection to the api backend on the apache:
Should receive a message error: true

{"error":true,"type":"authentication","message":"Could not authenticate. Invalid token given."}

Thanks,
Perry

sudo curl -XGET "http://xxx.xxx.xxx.xxx/nagioslogserver/ ... retty=true" -k
Last login: Tue Nov 30 08:20:01 EST 2021 on cron