Dashboard shows strange host address value

[heskez]

Hi there,

I've got the virtual Nagios Log Server setup on vmware ESX.
What catches my eye in the first place,
when logs are sent by syslog, a switch for instance

- the "host address" display value shows "0:0:0:0:0:0:0:1" this seems a ipv6 address? Is there a way to show the ipv4 address?
- I expeceted to see the switch's ipaddress within that field, am I correct? Why is it the local loopback address?

Best,
Erik

[jolson]

- the "host address" display value shows "0:0:0:0:0:0:0:1" this seems a ipv6 address? Is there a way to show the ipv4 address?

The host address "0:0:0:0:0:0:0:1" is an IPv6 localhost address - the logs that you're seeing from that address are coming from Nagios Log Server itself.

If your switch logs were being received in Nagios Log Server, you could expect to see the switches host address under the 'host' field. My assumption is that your switch logs aren't currently making it into Nagios Log Server.

[heskez]

Thanks for your answer Jolson,

If it's the case syslog logging from switches doesn't make it to the nagios server.

How am I able to make sure my switch logs are arriving to the nagios system? I've setup following configuration in advance:

- configured nagios logstash service to run with root privileges, add udp 514 as input
- Allowed upd 514 on the nagios system's iptables
- put the switches in debug mode and set log destination to the nagios log server.

[jolson]

It sounds like you've followed the proper procedure. The best way to verify that log data is actually arriving from your switch is with a tcpdump. Lets give the following a try (on Nagios Log Server):

Code: Select all

yum install tcpdump
tcpdump -n dst port 514

After you have the tcpdump running, attempt to do something to the switch (log in and out of the switch, for example) that might generate a log event. Do you see the log event arrive at Nagios Log Server?

In some cases, a kernel update is required for proper IPv4 -> IPv6 translation. A yum update might resolve the issue if log events are indeed arriving.

It's also worth checking to ensure logstash is listening on port 514 properly:

Code: Select all

netstat -na | grep 514

[heskez]

Thanks for the advice Jolson.

I think the first issue I had is fixed by the update. I do see firewall logs coming in the Dashboard now with a ipv4 address in the host field.

The second issue is still there, TCPDUMP shows me that syslog events from that switch are coming in but they aren't being found within the Dashboard..

netstat -an | grep 514 shows nagios listens to port 514 udp as well tcp..


tcp 0 0 :::514 :::* LISTEN
udp 0 0 :::514 :::*

[heskez]

It sounds like you've followed the proper procedure. The best way to verify that log data is actually arriving from your switch is with a tcpdump. Lets give the following a try (on Nagios Log Server):

Code: Select all

yum install tcpdump
tcpdump -n dst port 514

After you have the tcpdump running, attempt to do something to the switch (log in and out of the switch, for example) that might generate a log event. Do you see the log event arrive at Nagios Log Server?

In some cases, a kernel update is required for proper IPv4 -> IPv6 translation. A yum update might resolve the issue if log events are indeed arriving.

It's also worth checking to ensure logstash is listening on port 514 properly:

Code: Select all

netstat -na | grep 514

And hereby output of TCPDUMP

Code: Select all

16:30:15.355706 IP x.x.x.x.syslog > y.y.y.y.syslog: SYSLOG user.info, length: 88

[jolson]

Are you certain that your switches are using UDP to send their information? Is it possible that they're sending TCP packets?

It's also worth verifying that UDP/514 is open in your firewall, and that there's no deny rule above that one. If logs reach a raw UDP input, they will appear in the Web GUI with no problems.

My assumptions are, in order of likelihood:
1. Switches are actually sending via TCP, not UDP.
2. Nagios Log Server firewall is not open properly.
3. The date of Nagios Log Server is set incorrectly, so switch events show up in either the future or the past.

Let me know what you find out - thanks!

[heskez]

Hi, port TCP 514 is also open.
About the time issue, that seems likely. How do I set the time and date on the nagios log server?
I did it like this:

Code: Select all

service ntpd stop

Code: Select all

ntpdate x.x.x.x

Code: Select all

chkconfig ntpdate on

Code: Select all

service ntpd start

Time and date seems allright now.

[heskez]

Update.

I ran the dashboard with refresh time interval at 5 sec. I removed a host with a filter which flooded the logging.
I logged on the switch and the syslog message appears immediately on the dashboard! So that part is also fine.

One thing that bothers me a bit is that after a reboot of the nagios server the system was down and I had to manual bring up the ETH0 interface. Is that a bug?

[hsmith]

One thing that bothers me a bit is that after a reboot of the nagios server the system was down and I had to manual bring up the ETH0 interface. Is that a bug?

No. Can you review the contents of /etc/sysconfig/network-scripts/ifcfg-eth0? You'll want ONBOOT=yes to be present.