Import Sources from Files

[maartin.pii]

Hi All,

I have a particular scenario that I wanted to share with you in order to define a best practice for it.

- Nagios Log Server Version: Nagios Log Server • 1.4.4 •
- Cluster: No Cluster, just a Single Instance

My customer has purchased a license for Nagios Log Server in order to monitor a really mission critical application that is core for their busisness. They have identified some patterns that when they appear an action have to be taken asap.

The application is kind of legacy (can't remember the programming lenguage that it is developed on) and it runs on AIX. The problem here is that the application doesn't 'speak' with rsyslog protocol, so we have to import the log files by hand. For this I was thinking to use the 'shypper.py' script that comes with Nagios Log Server.

We have done some test on a lab environment and everything went ok, I wrote a query on json for the log pattern which detected the error and it was ok.

Now it's time to deploy Nagios log Server on prod environment and I am not really sure on which would be the better way or the best practice for importing the logs in order to be like a 'real time' monitoring. The customer's sysadmin told me that they can send me through scp or something like that the logs to Nagios Log Server, but what I was wondering is the following:

For ex:

1) They send me the logs every 5 minutes
1.a They send me the entire log from the app server to Nagios LS
1.b I upload it every 5 minutes

The question here is that if the log is the same but with the plus of the latest 5 minutes, Nagios LS realize that is the same information, discard that and import the new one only? Or I am having a duplicate log?

Which would be the better way to go through this? I know that this is not the best scenario. However, I have to think in some script that could resolve this.

Is there any best practice for scenarios like these?

Any kind of information will be thankfull.

Regards,

Juan

[mcapra]

Properly building rsyslog or syslog-ng on the remote machine is really the best option for this. shipper.py is good for one-off log file ingest, but not very good for something being provided regularly unless you 100% know that there aren't consistency issues with the files you are receiving.

If you can tell me what the system is currently using for it's syslog daemon (if anything), I can see if it can be configured to transport to Nagios Log Server.

Otherwise, if transporting the flat files is really the only option, I would suggest making a net share (from the AIX machine) that contains the log files and making that share available to the Nagios Log Server machine. Then all you would need to do is configure generic file inputs and logstash would/should take care of reconciling things.

[maartin.pii]

Thanks guys, I understand what you are saying. I know that this would not be the real use for what Log Server was build to. However, I have no choice but to do it on this way on this scenario.

You can close this thread.

Regards,