Page 4 of 7
Re: Nagios user java command using over 200% CPU
Posted: Tue Feb 19, 2019 2:40 pm
by rferebee
@cdienger I PM'd you with an updated system profile
Elasticsearch died this morning due to a memory commit error. Appreciate your assistance.
Re: Nagios user java command using over 200% CPU
Posted: Tue Feb 19, 2019 4:29 pm
by cdienger
The system is running with the maximum amount of memory of which elasticsearch will automatically take half. This halving can actually still result in an allocation for elasticsearch that causes memory performance problems. Let's try setting this manually to rule out this problem. Edit /etc/sysconfig/elasticsearch and change line 7 from:
ES_HEAP_SIZE=$(expr $(free -m|awk '/^Mem:/{print $2}') / 2 )m
to:
ES_HEAP_SIZE=3100m
and then restart elasticsearch:
service elasticsearch restart
Re: Nagios user java command using over 200% CPU
Posted: Tue Feb 19, 2019 5:04 pm
by rferebee
Ok, I made the requested change. I'll keep you posted.
Re: Nagios user java command using over 200% CPU
Posted: Tue Feb 19, 2019 5:17 pm
by cdienger
Sounds good. I'll keep my fingers crossed.
Re: Nagios user java command using over 200% CPU
Posted: Wed Feb 20, 2019 10:22 am
by rferebee
I'm unsure why the two would be related, but since making that change yesterday I am unable to login to the console. It spins for a few minutes and then tells me my username does not exist.
I've rebooted the cluster to no avail. It still won't let me login.
Re: Nagios user java command using over 200% CPU
Posted: Wed Feb 20, 2019 1:04 pm
by cdienger
Are you using the nagiosadmin account or another one? Can you test with the nagiosadmin account? You can also try creating a new admin account just to see if that can get in by running:
[user]curl -XPUT '
http://localhost:9200/nagioslogserver/user/99' -d '{"username":"someuser","password":"c678bcf3b5138b9263a95c44d28097f22c2e02877193d2c25313478821d45c19","auth_type":"admin","email":"
[email protected]","language":"default","apiaccess":"1","apikey":"1396e08757545557073844695e5b64caa0bd3ad3","created":"2015-01-23 10:00:00","created_by":0,"default_dashboard":"/dashboard/elasticsearch/default"}'[/user]
this creates a user called "someuser" with a password of "nagiosls".
Check the status of the indices with:
curl 'localhost:9200/_cat/indices?pretty'
Re: Nagios user java command using over 200% CPU
Posted: Wed Feb 20, 2019 1:46 pm
by rferebee
I tried my LDAP account as well as the nagiosadmin account. Both responding with username doesn't exist.
Unfortunately, I had to roll back this change since this is a production system. I don't think checking the indices would help with anything at this point.
Re: Nagios user java command using over 200% CPU
Posted: Wed Feb 20, 2019 2:12 pm
by cdienger
Thanks for the update.
And regarding the profile for future reference, in cases where you can't login to the web ui, you can run "/usr/local/nagioslogserver/scripts/profile.sh" from the command line which will create system-profile.tar.gz.
Re: Nagios user java command using over 200% CPU
Posted: Tue Mar 05, 2019 4:44 pm
by rferebee
Will the Enterprise version of nxlog work with Log Server or can we only use the CE version?
Re: Nagios user java command using over 200% CPU
Posted: Tue Mar 05, 2019 5:19 pm
by cdienger
The enterprise edition should work as well.