Page 1 of 1

No new indices being created

Posted: Tue Jan 17, 2017 9:40 am
by CameronWP
Hello:

I recently migrated my Nagios Log server to a new server and had some issues but thought I had everything working. My ports are listening, my services are started and seemingly happy, my logstash log is empty, I have 5TB of space for logs and I can see that all of my hosts are reporting in based on the number of hosts indication on the home screen. Despite all of this, there are no events actually being captured and there are no new indices. I am at a loss as to what to check next.

Thanks in advance!

Re: No new indices being created

Posted: Tue Jan 17, 2017 9:52 am
by CameronWP
Ok, I am seeing this on a number of my hosts in the nxlog.log:

ERROR couldn't connect to tcp socket on 10.1.151.34:3515; No connection could be made because the target machine actively refused it.

Thanks!

Re: No new indices being created

Posted: Tue Jan 17, 2017 9:55 am
by CameronWP
FYI also:

sestatus
SELinux status: disabled
service firewalld status
Redirecting to /bin/systemctl status firewalld.service
● firewalld.service
Loaded: masked (/dev/null; bad)
Active: inactive (dead)

Re: No new indices being created

Posted: Tue Jan 17, 2017 10:37 am
by rkennedy
This sounds like a firewall issue somewhere. From another machine, what is the output of nmap 10.1.151.34?

Is iptables running by chance? Depending on the host on where you transferred, there could be another firewall in between that needs to be opened up.

Lastly, from the NLS machine, what is the full output of netstat -anp

Re: No new indices being created

Posted: Tue Jan 17, 2017 11:06 am
by CameronWP
Thanks for the reply! I think I got to the bottom of it, I was looking in the wrong place. My heap size was an issue for the logstash apparently. I set it to 1024M and restarted the service and it stopped crashing (that was the root cause of the other issues IO was having).

Thanks!

Re: No new indices being created

Posted: Tue Jan 17, 2017 12:23 pm
by mcapra
Awesome! If a Logstash instance is receiving a lot of traffic on startup, I can see where the heap might be overwhelmed.

Did you have additional questions regarding this issue, or can we close the thread and mark it as resolved?