Load balancing NLS nodes

This support forum board is for support questions relating to Nagios Log Server, our solution for managing and monitoring critical log data.
stecino
Posts: 248
Joined: Thu Mar 14, 2013 4:42 pm

Load balancing NLS nodes

Post by stecino »

Hello,

In my rsyslog forwarder rule I am currently pointing to an individual node to forward my logs to.
It is considered as single point of failure. If I replace this node (fqdn/IP address) with fqdn/IP address of an F5 VIP that points to the pool of 4 NLS nodes, will this work?

Theoretically this should work, just wanted to ask you guys first before I start setting things up

Thanks in advance.
jolson
Attack Rabbit
Posts: 2560
Joined: Thu Feb 12, 2015 12:40 pm

Re: Load balancing NLS nodes

Post by jolson »

This will work - but be sure not to re-write the source IP addresses of your sending hosts. If you re-write the IP addresses with the IP of your load-balancer, it will appear as if all of your logs are coming from the load balancer.
Twits Blog
Show me a man who lives alone and has a perpetually clean kitchen, and 8 times out of 9 I'll show you a man with detestable spiritual qualities.
stecino
Posts: 248
Joined: Thu Mar 14, 2013 4:42 pm

Re: Load balancing NLS nodes

Post by stecino »

jolson wrote:This will work - but be sure not to re-write the source IP addresses of your sending hosts. If you re-write the IP addresses with the IP of your load-balancer, it will appear as if all of your logs are coming from the load balancer.
In what case will re-write happen?
jolson
Attack Rabbit
Posts: 2560
Joined: Thu Feb 12, 2015 12:40 pm

Re: Load balancing NLS nodes

Post by jolson »

It depends on the load balancer in question. It may or may not be default behavior - you will have to check your F5 balancer to find out.
Twits Blog
Show me a man who lives alone and has a perpetually clean kitchen, and 8 times out of 9 I'll show you a man with detestable spiritual qualities.
stecino
Posts: 248
Joined: Thu Mar 14, 2013 4:42 pm

Re: Load balancing NLS nodes

Post by stecino »

jolson wrote:It depends on the load balancer in question. It may or may not be default behavior - you will have to check your F5 balancer to find out.
Got it, I will look into that
jolson
Attack Rabbit
Posts: 2560
Joined: Thu Feb 12, 2015 12:40 pm

Re: Load balancing NLS nodes

Post by jolson »

Sounds good. There's a very good discussion about this topic here if you'd like to read into it: http://support.nagios.com/forum/viewtop ... 38&t=33005

Let us know what you find out. Thanks!

Best,


Jesse
Twits Blog
Show me a man who lives alone and has a perpetually clean kitchen, and 8 times out of 9 I'll show you a man with detestable spiritual qualities.
stecino
Posts: 248
Joined: Thu Mar 14, 2013 4:42 pm

Re: Load balancing NLS nodes

Post by stecino »

jolson wrote:Sounds good. There's a very good discussion about this topic here if you'd like to read into it: http://support.nagios.com/forum/viewtop ... 38&t=33005

Let us know what you find out. Thanks!

Best,


Jesse
Thanks I will look into it. I am also told that I have X-forwarding set on F5 it should pass the logsource IP trough. I will update you once implemented.
jolson
Attack Rabbit
Posts: 2560
Joined: Thu Feb 12, 2015 12:40 pm

Re: Load balancing NLS nodes

Post by jolson »

Sounds great! I look forward to your results.

In addition to what has been stated, you should be aware of the 'cluster hostname' setting - this will allow you to implement a consistent hostname across your instances, if that's something that you'd like to set while you're doing load-balancing.
2015-06-15 16_12_42-Global Settings • Nagios Log Server - Firefox Developer Edition.png
You do not have the required permissions to view the files attached to this post.
Twits Blog
Show me a man who lives alone and has a perpetually clean kitchen, and 8 times out of 9 I'll show you a man with detestable spiritual qualities.
stecino
Posts: 248
Joined: Thu Mar 14, 2013 4:42 pm

Re: Load balancing NLS nodes

Post by stecino »

jolson wrote:Sounds great! I look forward to your results.

In addition to what has been stated, you should be aware of the 'cluster hostname' setting - this will allow you to implement a consistent hostname across your instances, if that's something that you'd like to set while you're doing load-balancing.
2015-06-15 16_12_42-Global Settings • Nagios Log Server - Firefox Developer Edition.png

So is this the fqdn of the VIP? I am not sure I follow.

Also I have setup the VIP, it's listening on 5544 port, profile is set to UDP. I updated one of the logsources, to point to the VIP. Datagram Stats for Recieved and Transmitted show that there is traffic, but Packets don't seem to be going out, as no more events are being recorded for that host. Something definitely being passed
jolson
Attack Rabbit
Posts: 2560
Joined: Thu Feb 12, 2015 12:40 pm

Re: Load balancing NLS nodes

Post by jolson »

Say you have a DNS name that resolves to all of your NLS boxes - nls1.nagios.local resolves to 192.168.1.1 -or- 192.168.1.2 -or- 192.168.1.3.

At this point, you would set the 'cluster hostname' to 'nls1.nagios.local'. This will configure all of the nodes in your cluster to be aware of that dns name - it will also set your alert messages to point to nls1.nagios.local instead of the default 127.0.0.1.

Does that make sense?
Also I have setup the VIP, it's listening on 5544 port, profile is set to UDP. I updated one of the logsources, to point to the VIP. Datagram Stats for Recieved and Transmitted show that there is traffic, but Packets don't seem to be going out, as no more events are being recorded for that host. Something definitely being passed
Are the packets from this host showing up on any of the NLS nodes? You can run a tcpdump to verify:

Code: Select all

yum install tcpdump
tcpdump -n host 192.168.x.x and dst port 5544
Where 192.168.x.x is the IP address of the sending host. You should run the above commands on each NLS node and see whether any of them are receiving traffic from the host in question.
Twits Blog
Show me a man who lives alone and has a perpetually clean kitchen, and 8 times out of 9 I'll show you a man with detestable spiritual qualities.
Locked