trial issue, setup multiple sources, none showing up

[ucemike]

I've added 3 different hosts as log sources for a nagios log server (setup via OVF) and none of them are showing up in the dashboard.

ran the curl/setup-linux.sh on each client host.
next thing I did was verify that all 3 could talk to port 5544 (telnet to that port from the client, worked)
then, verify that rsyslog was sending packets to the nagios log server (tcpdump -n dst port 5544), they were

I am running linux in all cases
2.6.32-504.8.1.el6.centos.plus.x86_64
and
2.6.32-358.14.1.el6.x86_64

and rsyslogd 5.8.10.

Home section shows "Only receiving logs from 1 host." (same as it did when I first brought it up).

Am I missing some final configuration? I've watched the "how to add log source" tutorials and followed them completely as best I can tell.

[jolson]

It sounds like everything is set up and running properly - is logstash running on the NLS server?

Code: Select all

service logstash status

Code: Select all

tail /var/log/logstash/logstash.log

Did you use any special settings while running the setup.bash script?

[ucemike]

It sounds like everything is set up and running properly - is logstash running on the NLS server?

Code: Select all

service logstash status

Code: Select all

tail /var/log/logstash/logstash.log

Here is what I got.

Did you use any special settings while running the setup.bash script?

for the clients, no, is there another one on the OVF/nls side?

[jolson]

ucemike,

It doesn't look like your image came through - can you try that again please?

for the clients, no, is there another one on the OVF/nls side?

What I mean is which arguments did you use when running the bash script - the default arguments supplied from the 'Linux Source' page of NLS?

[ucemike]

ucemike,

It doesn't look like your image came through - can you try that again please?

Try this one
http://www.evernote.com/l/ASDzTdL8eM9OO ... I5SR8vMyw/

for the clients, no, is there another one on the OVF/nls side?

What I mean is which arguments did you use when running the bash script - the default arguments supplied from the 'Linux Source' page of NLS?

Yeap, I just copy/pasted. (bash setup-linux.sh -s nagios-log1.hostNameHere -p 5544)

[jolson]

Looks like logstash is missing its configuration. Could you run an 'Apply Config' from the Web GUI and re-start logstash please?

2015-05-06 12_06_36-Apply Configuration • Nagios Log Server.png

[ucemike]

Applied and also manually restarted and here is the result.

http://www.evernote.com/l/ASASX_RiAQJET ... z82pWvtvc/

So far nothing has shown up in the events or anymore than 1 loghosts.

[jolson]

Good - looks like logstash is up and running. Can you verify that logs are still flowing with another tcpdump from the NLS side of things? If you navigate to 'Dashboards' and select 'Last 5 minutes', does anything populate?

2015-05-06 12_42_45-Dashboard • Nagios Log Server.png

[ucemike]

tcpdump seems to be showing them coming in:

10.0.0.X = client
10.1.0.X = NLS
(these are not the real IPs)

17:51:28.898409 IP 10.0.0.X.49648 > 10.1.0.X.5544: Flags [.], seq 25996:27444, ack 1, win 115, options [nop,nop,TS val 108678152 ecr 9930631], length 1448


Unfortunately the dashboard still shows nothing (used last 5 min as detailed)

http://www.evernote.com/l/ASAm9LfD5K5NK ... TifyN8-M4/

[jolson]

Interesting. Let's take a look at your logstash configuration. Please run the following on the CLI and report the output to us:

Code: Select all

cat /usr/local/nagioslogserver/logstash/etc/conf.d/*

This should show us all of your logstash inputs, filters, and outputs - I would like to verify that they look correct.

How many nodes are in this cluster - is this a single node?
I'd like to see the output of this to ensure your output is configured properly:

Code: Select all

cat /usr/local/nagioslogserver/var/cluster_uuid

On your clients, you may wish to restart rsyslog to see if that has any effect:

Code: Select all

service rsyslog restart

Also, let's look at one of your clients rsyslog configurations to ensure it's put together correctly:

Code: Select all

cat /etc/rsyslog.conf
cat /etc/rsyslog.d/*log*