Problems installing to bare metal Cent7

[bpizzutiWHI]

I'm trying to install bare metal instead of the VM appliance, mostly because I want to use some 4kn drives that I had around for the storage. Anyway, it appears logstash won't start, and the system now seems to have forgotten. Any clue what might be happening? And what's the easiest way to knock this down and start from scratch without doing an OS reload? (It's at a remote datacenter so that would be a pain).

[tmcdonald]

Was the OS installed cleanly and left unmodified before the LS install? If not, there could be some packages or configurations remaining that are conflicting. We generally recommend a clean install for this reason.

Otherwise, look into the RAM since Java-based applications and Elasticsearch in particular take a lot of memory. Anything in the syslogs?

[bpizzutiWHI]

Very clean install, we hadn't even set up NIC bonding yet. The only software that was on there was our NetVault backup software.

When I did the Nagios install, it was going through an HTTP/HTTPS proxy, so I'm not sure if that would have hurt anything.

Nothing of note in message or secure.

[cdienger]

How about the logs in /var/log/elasticsearch/ or /var/log/logstash/ ? Memory errors are usually seen in the elasticsearch log's but it's usually best to check both since a failure of one can impact the other. How much memory is on the system? The minimum is 2GB but I usually recommend at least the 8GB recommended if possible.

[bpizzutiWHI]

This thing's got 64 GB.

I'm not interested in recovering this install at this point. I just want to knock it down and start from scratch. Can I delete the path it's installed to, or are there services that will need to be stopped first?

[cdienger]

You can do this:

Code: Select all

service logstash stop
service elasticsearch stop
yum erase httpd
rm -rf /usr/local/nagioslogserver
rm -rf /var/www/html/nagioslogserver
rm /etc/rc.d/init.d/elasticsearch
rm /etc/rc.d/init.d/logstash
rm /etc/rsyslog.d/nagioslogserver.conf
rm /etc/cron.d/nagioslogserver
pip uninstall elasticsearch-curator==3.4.0
service crond restart

The other and recommended option would be a full OS install so that it's a completely clean system.

https://assets.nagios.com/downloads/nag ... Server.pdf

[bpizzutiWHI]

Ok, so I did a reinstall, got this in the logstash.log file:

{:timestamp=>"2017-06-29T12:40:49.078000-0400", :message=>"Error: No config files found: /usr/local/nagioslogserver/logstash/etc/conf.d/*\nCan you make sure this path is a logstash config file?"}
{:timestamp=>"2017-06-29T12:40:49.083000-0400", :message=>"You may be interested in the '--configtest' flag which you can\nuse to validate logstash's configuration before you choose\nto restart a running system."}

[cdienger]

Well that's going to be a problem.... there should be 3 files installed there: 000_inputs.conf, 500_filtesr.conf, and 999_outputs.conf.

Is the server from a corporate build? This sounds like there is something preventing the install from completely installing everything it needs.

https://support.nagios.com/kb/article/i ... -rhel.html some things to check.

-run umask. My lab machine shows 0022
-check /etc/passwd for the nagios user
-sudoers should contain:

User_Alias NAGIOSLOGSERVER=nagios
User_Alias NAGIOSLOGSERVERWEB=apache
NAGIOSLOGSERVER ALL = NOPASSWD:/etc/init.d/logstash start
NAGIOSLOGSERVER ALL = NOPASSWD:/etc/init.d/logstash stop
NAGIOSLOGSERVER ALL = NOPASSWD:/etc/init.d/logstash restart
NAGIOSLOGSERVER ALL = NOPASSWD:/etc/init.d/logstash reload
NAGIOSLOGSERVER ALL = NOPASSWD:/etc/init.d/logstash status
NAGIOSLOGSERVER ALL = NOPASSWD:/etc/init.d/elasticsearch start
NAGIOSLOGSERVER ALL = NOPASSWD:/etc/init.d/elasticsearch stop
NAGIOSLOGSERVER ALL = NOPASSWD:/etc/init.d/elasticsearch restart
NAGIOSLOGSERVER ALL = NOPASSWD:/etc/init.d/elasticsearch reload
NAGIOSLOGSERVER ALL = NOPASSWD:/etc/init.d/elasticsearch status
NAGIOSLOGSERVER ALL = NOPASSWD:/usr/local/nagioslogserver/scripts/change_timezone.sh
NAGIOSLOGSERVERWEB ALL = NOPASSWD:/etc/init.d/logstash start
NAGIOSLOGSERVERWEB ALL = NOPASSWD:/etc/init.d/logstash stop
NAGIOSLOGSERVERWEB ALL = NOPASSWD:/etc/init.d/logstash restart
NAGIOSLOGSERVERWEB ALL = NOPASSWD:/etc/init.d/logstash reload
NAGIOSLOGSERVERWEB ALL = NOPASSWD:/etc/init.d/logstash status
NAGIOSLOGSERVERWEB ALL = NOPASSWD:/etc/init.d/elasticsearch start
NAGIOSLOGSERVERWEB ALL = NOPASSWD:/etc/init.d/elasticsearch stop
NAGIOSLOGSERVERWEB ALL = NOPASSWD:/etc/init.d/elasticsearch restart
NAGIOSLOGSERVERWEB ALL = NOPASSWD:/etc/init.d/elasticsearch reload
NAGIOSLOGSERVERWEB ALL = NOPASSWD:/etc/init.d/elasticsearch status
NAGIOSLOGSERVERWEB ALL = NOPASSWD:/usr/local/nagioslogserver/scripts/get_logstash_ports.sh

# NEEDED TO ALLOW NAGIOS TO CHECK SERVICE STATUS
Defaults:nagios !requiretty
nagios ALL=NOPASSWD: /usr/local/nagios/libexec/check_init_service

# ASTERISK-SPECIFIC CHECKS
# NOTE: You can uncomment the following line if you are monitoring Asterisk locally
#nagios ALL=NOPASSWD: /usr/local/nagios/libexec/check_asterisk_sip_peers.sh, /usr/local/nagios/libexec/nagisk.pl, /usr/sbin/asterisk

[tacolover101]

@cdienger is right, sounds like something didn't write out correctly which could be further problematic.

the issue is that you're missing the standard configuration files that come with NLS, and should be in the /usr/local/nagioslogserver/logstash/etc/conf.d/ directory

this post has the files you'll need to make, then you should be able to run service logstash start - https://support.nagios.com/forum/viewto ... 58#p194729

[cdienger]

Thanks the help, tacolover101! Copying the files over will help towards getting logstash up and running but the underlying issue should be addressed since it could lead to other problems down the line.