188000 + syslog entries a sec

[benhank]

And still running strong, no issues.

vector-good-job-symbol-29548614.jpg

[jolson]

Happy to hear it!

[tmcdonald]

188,000 reasons per second to use Nagios :)

[WillemDH]

Wow, I'm not even at 1000 / sec. But I just got confirmation I get 500 GB extra storage. so more is coming... :p Can i ask how many NLS nodes you have, how many storage they are consuming? retention period? Virtual servers? Or physical? How many cpu's? (Plus minus)
Tx!

[scottwilkerson]

Nice!

@benhank - Can you share how many instances in your cluster and what type of hardware you are running on?

Great story...

[benhank]

Code: Select all

221,444,583Documents
90.3GBPrimary Size
181.5GBTotal Size
2Data Instances
2972Total Shards
298Indices

Code: Select all

2Total Instances
0Client
2Master/Data
10Processors
0%Process CPU
36.12 GBMemory Used
154.62 MBSwap
2,368.26 GBTotal Storage
2,172.45 GBFree Storage
469.13 GBData Read
5,498.76 GBData Written
5,967.89 GBI/O Size

Code: Select all

Cluster Health
Status 	Green
Timed Out? 	false
# Instances 	2
# Data Instances 	2
Active Primary Shards 	1486
Active Shards 	2972
Relocating Shards 	0
Initializing Shards 	0
Unassigned Shards 	0

[scottwilkerson]

@benhank - I'm a little confused on your math. Was the 188k syslog entries a sec just a quick burst/spike?

What you posted only shows 221m documents indexed, which would be only ~ 19 hours of logs @ 188k/sec.

[benhank]

hey pal I am a sysadmin not a mathamatician... It's the thought that counts =D

[tmcdonald]

188k/s seems a little high, but for a 2-instance cluster it looks to be humming along quite nicely.

Now I need to do that manager thing and request that we go back to being boring and close up this thread. That cool?

[benhank]

yeah, but i want to apologize for the bad math. My main point was that we have a lot of stuff being processed by NLS, and it is working fine.
You guys made a product that delivers on its stated capabilities.
I spent months trying to get the ELK stack working. NLS was an easy install,as promised, and it makes setting up remote devices simple.
Kudos and good job to the team.
Ben