Nagios Support Forum

Posted: **Mon Dec 29, 2014 8:15 am**

Ok,

I will disable auto-reboot, & I will execute command when this happens.

Posted: **Mon Dec 29, 2014 10:31 am**

Sounds great, thanks!

Posted: **Tue Dec 30, 2014 4:25 am**

Hello,

Here daily information

Web Interface is very slow, I can't load Admin Page.
Dashboard is empty

Servers (both) are up since 18h18

[root@nagioslog ~]# uptime
10:10:07 up 18:18, 1 user, load average: 0.18, 0.11, 0.03

CPU are not high

[root@nagioslog ~]# top

top - 10:15:33 up 18:23, 1 user, load average: 0.06, 0.08, 0.02
Tasks: 131 total, 1 running, 130 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.1%us, 0.0%sy, 0.1%ni, 99.8%id, 0.0%wa, 0.0%hi, 0.1%si, 0.0%st
Mem: 4056704k total, 3637012k used, 419692k free, 135932k buffers
Swap: 262136k total, 42628k used, 219508k free, 855620k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1188 root 20 0 174m 2520 2180 S 0.7 0.1 0:36.95 vmtoolsd
1530 nagios 39 19 3763m 434m 6316 S 0.7 11.0 10:11.73 java
1382 nagios 20 0 14.5g 1.8g 17m S 0.3 47.0 111:02.98 java
1 root 20 0 19232 1256 1108 S 0.0 0.0 0:01.27 init
2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kthreadd
3 root RT 0 0 0 0 S 0.0 0.0 0:00.09 migration/0
4 root 20 0 0 0 0 S 0.0 0.0 0:00.56 ksoftirqd/0

And result of command

Code: Select all

ps -ef

[root@nagioslog ~]# ps -ef
UID PID PPID C STIME TTY TIME CMD
root 1 0 0 Dec29 ? 00:00:01 /sbin/init
root 2 0 0 Dec29 ? 00:00:00 [kthreadd]
root 3 2 0 Dec29 ? 00:00:00 [migration/0]
root 4 2 0 Dec29 ? 00:00:00 [ksoftirqd/0]
root 5 2 0 Dec29 ? 00:00:00 [migration/0]
root 6 2 0 Dec29 ? 00:00:00 [watchdog/0]
root 7 2 0 Dec29 ? 00:00:00 [migration/1]
root 8 2 0 Dec29 ? 00:00:00 [migration/1]
root 9 2 0 Dec29 ? 00:00:00 [ksoftirqd/1]
root 10 2 0 Dec29 ? 00:00:00 [watchdog/1]
root 11 2 0 Dec29 ? 00:00:00 [migration/2]
root 12 2 0 Dec29 ? 00:00:00 [migration/2]
root 13 2 0 Dec29 ? 00:00:00 [ksoftirqd/2]
root 14 2 0 Dec29 ? 00:00:00 [watchdog/2]
root 15 2 0 Dec29 ? 00:00:00 [migration/3]
root 16 2 0 Dec29 ? 00:00:00 [migration/3]
root 17 2 0 Dec29 ? 00:00:00 [ksoftirqd/3]
root 18 2 0 Dec29 ? 00:00:00 [watchdog/3]
root 19 2 0 Dec29 ? 00:00:15 [events/0]
root 20 2 0 Dec29 ? 00:00:11 [events/1]
root 21 2 0 Dec29 ? 00:00:02 [events/2]
root 22 2 0 Dec29 ? 00:00:02 [events/3]
root 23 2 0 Dec29 ? 00:00:00 [cgroup]
root 24 2 0 Dec29 ? 00:00:00 [khelper]
root 25 2 0 Dec29 ? 00:00:00 [netns]
root 26 2 0 Dec29 ? 00:00:00 [async/mgr]
root 27 2 0 Dec29 ? 00:00:00 [pm]
root 28 2 0 Dec29 ? 00:00:00 [sync_supers]
root 29 2 0 Dec29 ? 00:00:00 [bdi-default]
root 30 2 0 Dec29 ? 00:00:00 [kintegrityd/0]
root 31 2 0 Dec29 ? 00:00:00 [kintegrityd/1]
root 32 2 0 Dec29 ? 00:00:00 [kintegrityd/2]
root 33 2 0 Dec29 ? 00:00:00 [kintegrityd/3]
root 34 2 0 Dec29 ? 00:00:00 [kblockd/0]
root 35 2 0 Dec29 ? 00:00:00 [kblockd/1]
root 36 2 0 Dec29 ? 00:00:00 [kblockd/2]
root 37 2 0 Dec29 ? 00:00:00 [kblockd/3]
root 38 2 0 Dec29 ? 00:00:00 [kacpid]
root 39 2 0 Dec29 ? 00:00:00 [kacpi_notify]
root 40 2 0 Dec29 ? 00:00:00 [kacpi_hotplug]
root 41 2 0 Dec29 ? 00:00:00 [ata/0]
root 42 2 0 Dec29 ? 00:00:00 [ata/1]
root 43 2 0 Dec29 ? 00:00:00 [ata/2]
root 44 2 0 Dec29 ? 00:00:00 [ata/3]
root 45 2 0 Dec29 ? 00:00:00 [ata_aux]
root 46 2 0 Dec29 ? 00:00:00 [ksuspend_usbd]
root 47 2 0 Dec29 ? 00:00:00 [khubd]
root 48 2 0 Dec29 ? 00:00:00 [kseriod]
root 49 2 0 Dec29 ? 00:00:00 [md/0]
root 50 2 0 Dec29 ? 00:00:00 [md/1]
root 51 2 0 Dec29 ? 00:00:00 [md/2]
root 52 2 0 Dec29 ? 00:00:00 [md/3]
root 53 2 0 Dec29 ? 00:00:00 [md_misc/0]
root 54 2 0 Dec29 ? 00:00:00 [md_misc/1]
root 55 2 0 Dec29 ? 00:00:00 [md_misc/2]
root 56 2 0 Dec29 ? 00:00:00 [md_misc/3]
root 57 2 0 Dec29 ? 00:00:00 [khungtaskd]
root 58 2 0 Dec29 ? 00:00:00 [kswapd0]
root 59 2 0 Dec29 ? 00:00:00 [ksmd]
root 60 2 0 Dec29 ? 00:00:00 [khugepaged]
root 61 2 0 Dec29 ? 00:00:00 [aio/0]
root 62 2 0 Dec29 ? 00:00:00 [aio/1]
root 63 2 0 Dec29 ? 00:00:00 [aio/2]
root 64 2 0 Dec29 ? 00:00:00 [aio/3]
root 65 2 0 Dec29 ? 00:00:00 [crypto/0]
root 66 2 0 Dec29 ? 00:00:00 [crypto/1]
root 67 2 0 Dec29 ? 00:00:00 [crypto/2]
root 68 2 0 Dec29 ? 00:00:00 [crypto/3]
root 73 2 0 Dec29 ? 00:00:00 [kthrotld/0]
root 74 2 0 Dec29 ? 00:00:00 [kthrotld/1]
root 75 2 0 Dec29 ? 00:00:00 [kthrotld/2]
root 76 2 0 Dec29 ? 00:00:00 [kthrotld/3]
root 77 2 0 Dec29 ? 00:00:00 [pciehpd]
root 79 2 0 Dec29 ? 00:00:00 [kpsmoused]
root 80 2 0 Dec29 ? 00:00:00 [usbhid_resumer]
root 286 2 0 Dec29 ? 00:00:00 [mpt_poll_0]
root 287 2 0 Dec29 ? 00:00:00 [mpt/0]
root 288 2 0 Dec29 ? 00:00:00 [scsi_eh_0]
root 294 2 0 Dec29 ? 00:00:00 [scsi_eh_1]
root 295 2 0 Dec29 ? 00:00:00 [scsi_eh_2]
root 384 2 0 Dec29 ? 00:00:10 [jbd2/sda1-8]
root 385 2 0 Dec29 ? 00:00:00 [ext4-dio-unwrit]
root 386 2 0 Dec29 ? 00:00:00 [ext4-dio-unwrit]
root 387 2 0 Dec29 ? 00:00:00 [ext4-dio-unwrit]
root 388 2 0 Dec29 ? 00:00:00 [ext4-dio-unwrit]
root 461 1 0 Dec29 ? 00:00:00 /sbin/udevd -d
root 576 2 0 Dec29 ? 00:00:04 [flush-8:0]
root 616 2 0 Dec29 ? 00:00:00 [vmmemctl]
root 800 2 0 Dec29 ? 00:00:00 [kstriped]
root 848 2 0 Dec29 ? 00:00:01 [kauditd]
root 1188 1 0 Dec29 ? 00:00:36 /usr/sbin/vmtoolsd
root 1282 1 0 Dec29 ? 00:00:03 auditd
root 1298 1 0 Dec29 ? 00:00:01 /sbin/rsyslogd -i /var/run/syslogd.pid -c 5
rpc 1318 1 0 Dec29 ? 00:00:00 rpcbind
root 1352 1 0 Dec29 ? 00:00:00 /usr/sbin/sshd
ntp 1360 1 0 Dec29 ? 00:00:00 ntpd -u ntp:ntp -p /var/run/ntpd.pid -g
nagios 1382 1 10 Dec29 ? 01:51:04 /usr/bin/java -Xms2g -Xmx2g -Xss256k -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatin
nrpe 1390 1 0 Dec29 ? 00:00:00 /usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -d
root 1495 1 0 Dec29 ? 00:00:00 /usr/libexec/postfix/master
postfix 1497 1495 0 Dec29 ? 00:00:00 qmgr -l -t fifo -u
root 1501 1 0 Dec29 ? 00:00:01 /usr/sbin/httpd
root 1509 1 0 Dec29 ? 00:00:00 crond
apache 1518 1501 0 Dec29 ? 00:00:02 /usr/sbin/httpd
apache 1519 1501 0 Dec29 ? 00:00:02 /usr/sbin/httpd
apache 1521 1501 0 Dec29 ? 00:00:02 /usr/sbin/httpd
apache 1523 1501 0 Dec29 ? 00:00:02 /usr/sbin/httpd
apache 1524 1501 0 Dec29 ? 00:00:02 /usr/sbin/httpd
apache 1525 1501 0 Dec29 ? 00:00:02 /usr/sbin/httpd
apache 1526 1501 0 Dec29 ? 00:00:02 /usr/sbin/httpd
apache 1527 1501 0 Dec29 ? 00:00:03 /usr/sbin/httpd
root 1528 1 0 Dec29 ? 00:00:00 runuser -s /bin/sh -c exec /usr/local/nagioslogserver/logstash/bin/logstash agent -f /usr/local/nagioslogserver/logstash/etc/conf.d -l /var/log/logstash/logstash
nagios 1530 1528 0 Dec29 ? 00:10:11 /usr/bin/java -Djava.io.tmpdir=/usr/local/nagioslogserver/tmp -Xmx500m -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -Djava.awt.headless=true -XX:CMSInitiatingOccupan
root 1584 1 0 Dec29 tty1 00:00:00 /sbin/mingetty /dev/tty1
root 1586 1 0 Dec29 tty2 00:00:00 /sbin/mingetty /dev/tty2
root 1588 1 0 Dec29 tty3 00:00:00 /sbin/mingetty /dev/tty3
root 1590 1 0 Dec29 tty4 00:00:00 /sbin/mingetty /dev/tty4
root 1592 1 0 Dec29 tty5 00:00:00 /sbin/mingetty /dev/tty5
root 1594 1 0 Dec29 tty6 00:00:00 /sbin/mingetty /dev/tty6
root 1598 461 0 Dec29 ? 00:00:00 /sbin/udevd -d
root 1599 461 0 Dec29 ? 00:00:00 /sbin/udevd -d
apache 2403 1501 0 09:15 ? 00:00:00 /usr/sbin/httpd
root 5284 1352 0 10:04 ? 00:00:00 sshd: root@pts/0
root 5297 5284 0 10:04 pts/0 00:00:00 -bash
postfix 5828 1495 0 10:13 ? 00:00:00 pickup -l -t fifo -u
root 5927 1509 0 10:15 ? 00:00:00 CROND
root 5928 1509 0 10:15 ? 00:00:00 CROND
nagios 5929 5927 0 10:15 ? 00:00:00 /bin/sh -c /usr/bin/php -q /var/www/html/nagioslogserver/www/index.php jobs > /usr/local/nagioslogserver/var/jobs.log 2>&1
nagios 5930 5929 0 10:15 ? 00:00:00 /usr/bin/php -q /var/www/html/nagioslogserver/www/index.php jobs
nagios 5931 5928 0 10:15 ? 00:00:00 /bin/sh -c /usr/bin/php -q /var/www/html/nagioslogserver/www/index.php poller > /usr/local/nagioslogserver/var/poller.log 2>&1
nagios 5932 5931 0 10:15 ? 00:00:00 /usr/bin/php -q /var/www/html/nagioslogserver/www/index.php poller
root 5980 5297 0 10:15 pts/0 00:00:00 ps -ef

I hop this can help

Posted: **Tue Dec 30, 2014 10:19 am**

I thought you were seeing 500+ processes and CPU of 100%?

It is possible you may need to expand the cluster a bit to spread the disk load over several machines. Additionally, I would recommend upping the RAM as well as increasing the HEAP size

You can increase this by setting ES_HEAP_SIZE in /etc/sysconfig/elasticsearch. The general recommendation is to give it half of the available RAM. It looks like you have about 16GB total, so:

Code: Select all

ES_HEAP_SIZE=8g
MAX_LOCKED_MEMORY=unlimited

That last line keeps the ES heap from being swapped if possible (system limits may need to be adjusted to allow this).

Some more info on ES memory tuning: http://www.elasticsearch.org/guide/en/e ... izing.html

Posted: **Wed Dec 31, 2014 3:54 am**

scottwilkerson wrote: I thought you were seeing 500+ processes and CPU of 100%?

Yes, I had this problem last week... I don't know why it has not happened this week.
This is a rather good news...

I will apply the change to RAM and ES_HEAP_SIZE and add 3rd server to cluster.

I will inform you on January 2 of the result

Posted: **Fri Jan 02, 2015 10:40 am**

Thanks!

Posted: **Mon Jan 05, 2015 2:46 am**

Happy new year 2015 !

I add new server to cluster, now I have three nagioslog server, I increase RAM & I edit /etc/sysconfig/elasticsearch with your information.
After 5 days, same problem except that web interface isn't slow, juste dashboard which is empty.

In Administration / Instance Status : Elasticsearch & Logstash are in error, but with command ssh :

[root@nagioslog ~]# service elasticsearch status
elasticsearch (pid 1408) is running...
[root@nagioslog ~]# service logstash status
Logstash Daemon (pid 1957) is running...

Administratation / Cluster Status I get (before reboot cluster, at 8h25 2015-01-05):

And five minutes after reboot, I get this (at 8h30 2015-01-05):

My Cluster Health is Green

Posted: **Mon Jan 05, 2015 5:41 pm**

When you say the dashboard is empty, do you mean there are no results? Or that the interface itself does not display anything?

Also, please run the following on each of the Logserver machines as root and post the output:

Code: Select all

curl -XGET 'http://localhost:9200/_count?pretty'

Posted: **Mon Jan 05, 2015 6:01 pm**

In addition to tmcdonalds request, can you provide the last 100 lines of the logstash log?

Code: Select all

tail -n 100 /var/log/logstash/logstash.log

Posted: **Tue Jan 06, 2015 3:22 am**

tmcdonald wrote:When you say the dashboard is empty, do you mean there are no results? Or that the interface itself does not display anything?

The interface itself does not display anything, & in the home page Log Server Overview Receiving logs from 1 hosts...

I will execute commands when the cluster was again the problem, which should not be long.

EDIT :

Screenshot for Dashboard empty :

Command's result :

[root@nagioslog1 ~]# curl -XGET 'http://localhost:9200/_count?pretty'
{
"count" : 128879638,
"_shards" : {
"total" : 131,
"successful" : 130,
"failed" : 0
}
}

[root@nagioslog2 ~]# curl -XGET 'http://localhost:9200/_count?pretty'
{
"count" : 128879638,
"_shards" : {
"total" : 131,
"successful" : 130,
"failed" : 0
}
}

[root@nagioslog3 ~]# curl -XGET 'http://localhost:9200/_count?pretty'
{
"count" : 128879638,
"_shards" : {
"total" : 131,
"successful" : 130,
"failed" : 0
}
}

[root@nagioslog1 ~]# tail -n 100 /var/log/logstash/logstash.log
{:timestamp=>"2015-01-06T16:34:15.015000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:16.016000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:17.017000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:18.018000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:19.019000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:20.020000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:21.021000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:22.022000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:23.023000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:24.024000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:25.026000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:26.027000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:27.028000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:28.029000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:29.030000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:30.031000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:31.032000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:32.033000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:33.034000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:34.035000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:35.036000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:36.037000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:37.038000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:38.039000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:39.040000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:40.041000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:41.042000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:42.043000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:43.044000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:44.045000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:45.047000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:46.048000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:47.049000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:48.050000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:49.051000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:50.052000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:51.053000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:52.054000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:53.055000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:54.056000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:55.057000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:56.058000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:57.059000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:58.060000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:34:59.061000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:00.063000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:01.064000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:02.065000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:03.066000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:04.067000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:05.068000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:06.069000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:07.070000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:08.071000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:09.072000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:10.074000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:11.075000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:12.076000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:13.077000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:14.078000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:15.079000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:16.080000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:17.081000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:18.082000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:19.083000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:20.084000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:21.085000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:22.086000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:23.087000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:24.088000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:25.089000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:26.090000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:27.091000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:28.092000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:29.093000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:30.095000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:31.096000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:32.097000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:33.098000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:34.099000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:35.100000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:36.101000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:37.102000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:38.103000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:39.104000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:40.105000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:41.106000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:42.107000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:43.108000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:44.109000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:45.110000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:46.111000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:47.112000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:48.113000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:49.114000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:50.115000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:51.116000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:52.117000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:53.119000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}
{:timestamp=>"2015-01-06T16:35:54.120000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"eventlog\", host=>\"0.0.0.0\", mode=>\"server\">\n Error: closed stream", :level=>:error}

Same log for each server in the cluster.

Nagios Support Forum

Problem index log every day

Re: Problem index log every day

Re: Problem index log every day

Re: Problem index log every day

Re: Problem index log every day

Re: Problem index log every day

Re: Problem index log every day

Re: Problem index log every day

Re: Problem index log every day

Re: Problem index log every day

Re: Problem index log every day