Q: What happens to rsyslogs when logserver goes offline?

This support forum board is for support questions relating to Nagios Log Server, our solution for managing and monitoring critical log data.
Locked
wyoder
Posts: 50
Joined: Mon Dec 29, 2014 3:21 pm
Location: Raleigh, NC
Contact:

Q: What happens to rsyslogs when logserver goes offline?

Post by wyoder »

Hello:

We want to take our single and lonely logserver down for maintenance. For now, there is no companion server in a cluster. Once this logserver goes down, there will be no place for the remote agents to send their logs. The question is, what will happen to the logs while our logserver is offline?
  • - Will the events occurring during this interval be lost? Hope not.
    - Will they be queued up and resume sending when the logserver goes back online? Hope so.
    - Since we're using TCP connections between the agents and the logserver, will the agents suffer congestion/constipation--holding their events in some huge buffer while waiting for the logserver to return to service?
Thanks in advance for your response.
--Bill
tmcdonald
Posts: 9117
Joined: Mon Sep 23, 2013 8:40 am

Re: Q: What happens to rsyslogs when logserver goes offline?

Post by tmcdonald »

This will all depend on the sending servers, and whether their logging daemons can spool. What remote syslog agents are you using?
Former Nagios employee
wyoder
Posts: 50
Joined: Mon Dec 29, 2014 3:21 pm
Location: Raleigh, NC
Contact:

Re: Q: What happens to rsyslogs when logserver goes offline?

Post by wyoder »

Hello, tmcdonald:

Thanks for your reponse.

Both types of our server senders (RHEL 6 and Ubuntu 12.04) are running Version 3.21.1 of the rsyslogd agent. So far, the Nagios logserver environment has been great--both reliable and useful. Is this enough information?

--Bill
User avatar
hsmith
Agent Smith
Posts: 3539
Joined: Thu Jul 30, 2015 11:09 am
Location: 127.0.0.1
Contact:

Re: Q: What happens to rsyslogs when logserver goes offline?

Post by hsmith »

I've normally witnessed the behavior where the logs are spooled for days assuming you're using TCP. UDP logs are generally just gone.
Former Nagios Employee.
me.
User avatar
eloyd
Cool Title Here
Posts: 2190
Joined: Thu Sep 27, 2012 9:14 am
Location: Rochester, NY
Contact:

Re: Q: What happens to rsyslogs when logserver goes offline?

Post by eloyd »

rsyslog will queue messages up to a point. That point is memory dependent. When it gets full, it blocks. This is the same as writing to local log file and running out of disk space. You can tell rsyslog to drop new messages if the queue fills up and/or to start queuing to local disk. Both of which are explained in rsyslog search engine results and beyond the scope of Nagios Log Server support.
Image
Eric Loyd • http://everwatch.global • 844.240.EVER • @EricLoyd
I'm a Nagios Fanatic! • Join our public Nagios Discord Server!
rkennedy
Posts: 6579
Joined: Mon Oct 05, 2015 11:45 am

Re: Q: What happens to rsyslogs when logserver goes offline?

Post by rkennedy »

Thanks @eloyd!

@wyoder - let us know if you have any further questions.
Former Nagios Employee
wyoder
Posts: 50
Joined: Mon Dec 29, 2014 3:21 pm
Location: Raleigh, NC
Contact:

Re: Q: What happens to rsyslogs when logserver goes offline?

Post by wyoder »

Thanks, Folks, for your help.

We took our logserver offline, upgraded it from RHEL6.7 to RHEL6.8, brought it back online, and everything seems fine.

Accordingly, please close this post at your convenience.

Best regards,
Bill Yoder
Locked