Page 1 of 1

Importing AWS Cloudwatch Logs into Log Server

Posted: Tue Nov 28, 2017 9:08 am
by quentinw
Hi,

Can anyone advise how they have done this or plan to? We have a fair number of Windows instances that we plan to leverage the already installed EC2 service to direct logs to Cloudwatch along with some syslog data. We are trying to avoid installing another agent like NXLog and using that to direct log data to NLS.

I'm not sure what our options are for getting the centralised Cloudwatch Log data into NLS? Does Nagios (or the community) have experience or recommendations how to proceed?

Cheers,

Quentin

Re: Importing AWS Cloudwatch Logs into Log Server

Posted: Tue Nov 28, 2017 4:22 pm
by kyang
It's only available with logstash 5.0 or greater which we do not use yet.

If you're able to get logs out of AWS, you could feed them into NLS with a generic input.

https://www.elastic.co/guide/en/logstas ... ugins.html

Re: Importing AWS Cloudwatch Logs into Log Server

Posted: Tue Nov 28, 2017 11:30 pm
by quentinw
Ok.. thanks.

I'll look at either exporting to S3 and then dumping to local disk for ingestion or using "aws log" CLI or https://github.com/jorgebastida/awslogs to read from Cloudwatch Logs directly and ingest via shipper.py.

Thanks,

Quentin

Re: Importing AWS Cloudwatch Logs into Log Server

Posted: Wed Nov 29, 2017 10:13 am
by kyang
Sounds good! Did you want us to leave this thread open?

Re: Importing AWS Cloudwatch Logs into Log Server

Posted: Wed Nov 29, 2017 10:55 am
by mcapra
I don't have a CloudWatch instance to test against, but it appears as though the plugin at least installs on the current version of NLS:

Code: Select all

[root@localhost ~]# /usr/local/nagioslogserver/logstash/bin/plugin install logstash-input-cloudwatch
The use of bin/plugin is deprecated and will be removed in a feature release. Please use bin/logstash-plugin.
Validating logstash-input-cloudwatch
Installing logstash-input-cloudwatch
Installation successful
[root@localhost ~]# /usr/local/nagioslogserver/logstash/bin/logstash -V
logstash 2.4.1

Re: Importing AWS Cloudwatch Logs into Log Server

Posted: Wed Nov 29, 2017 1:08 pm
by kyang
Thanks @mcapra!

OP, let us know if you have any more questions.