Support for Nagios products and services
https://support.nagios.com/forum/
Exact same. These two servers were built same time and I did a full yum update before I installed NLS. I ran the command to be sure, and yes, exact same output.mcapra wrote:Cool, send it on over if/when it crashes. Can I also see the java -version output on the other node that is functional?
Code: Select all
Sep 21, 2016 3:31:21 PM org.elasticsearch.plugins.PluginsService <init>
INFO: [f5c0ae96-4dd5-4f16-bd5a-a73b245f7694] loaded [], sites []
Sep 21, 2016 3:31:33 PM org.elasticsearch.plugins.PluginsService <init>
INFO: [f5c0ae96-4dd5-4f16-bd5a-a73b245f7694] loaded [], sites []
Sep 21, 2016 3:31:33 PM org.elasticsearch.plugins.PluginsService <init>
INFO: [f5c0ae96-4dd5-4f16-bd5a-a73b245f7694] loaded [], sites []
Sep 21, 2016 3:31:33 PM org.elasticsearch.plugins.PluginsService <init>
INFO: [f5c0ae96-4dd5-4f16-bd5a-a73b245f7694] loaded [], sites []
Sep 21, 2016 3:31:33 PM org.elasticsearch.plugins.PluginsService <init>
INFO: [f5c0ae96-4dd5-4f16-bd5a-a73b245f7694] loaded [], sites []
Sep 21, 2016 3:41:22 PM org.elasticsearch.transport.netty.MessageChannelHandler messageReceived
WARNING: [f5c0ae96-4dd5-4f16-bd5a-a73b245f7694] Message not fully read (response) for [442] handler org.elasticsearch.action.TransportActionNodeProxy$1@17c1dd3, error [true], resetting
Sep 21, 2016 3:41:22 PM org.elasticsearch.transport.netty.MessageChannelHandler messageReceived
WARNING: [f5c0ae96-4dd5-4f16-bd5a-a73b245f7694] Message not fully read (response) for [478] handler org.elasticsearch.action.TransportActionNodeProxy$1@32737a46, error [true], resetting
Sep 21, 2016 3:41:23 PM org.elasticsearch.transport.netty.MessageChannelHandler messageReceived
WARNING: [f5c0ae96-4dd5-4f16-bd5a-a73b245f7694] Message not fully read (response) for [427] handler org.elasticsearch.action.TransportActionNodeProxy$1@5a329f3b, error [true], resetting
Sep 21, 2016 3:41:23 PM org.elasticsearch.transport.netty.MessageChannelHandler messageReceived
WARNING: [f5c0ae96-4dd5-4f16-bd5a-a73b245f7694] Message not fully read (response) for [443] handler org.elasticsearch.action.TransportActionNodeProxy$1@2473a70, error [true], resetting
Sep 21, 2016 3:41:23 PM org.elasticsearch.transport.netty.MessageChannelHandler messageReceived
WARNING: [f5c0ae96-4dd5-4f16-bd5a-a73b245f7694] Message not fully read (response) for [480] handler org.elasticsearch.action.TransportActionNodeProxy$1@eb168bf, error [true], resetting
Sep 21, 2016 3:41:24 PM org.elasticsearch.transport.netty.MessageChannelHandler messageReceived
WARNING: [f5c0ae96-4dd5-4f16-bd5a-a73b245f7694] Message not fully read (response) for [429] handler org.elasticsearch.action.TransportActionNodeProxy$1@5c76bf7b, error [true], resetting
Sep 21, 2016 3:41:24 PM org.elasticsearch.transport.netty.MessageChannelHandler messageReceived
WARNING: [f5c0ae96-4dd5-4f16-bd5a-a73b245f7694] Message not fully read (response) for [445] handler org.elasticsearch.action.TransportActionNodeProxy$1@2de6d497, error [true], resetting
Code: Select all
curl 'localhost:9200/_nodes/jvm?pretty'Code: Select all
{
"cluster_name" : "1c6d88c6-e9ca-480d-8c85-8bd97ef41adf",
"nodes" : {
"XpBH4IG2S4aMC8qR-gl3Dg" : {
"name" : "91c49ceb-0d7c-4e27-a80a-b9036f0d3375",
"transport_address" : "inet[/172.25.14.21:9300]",
"host" : "hdfs-chi-nag01.hdfs.itciss.com",
"ip" : "172.25.14.21",
"version" : "1.6.0",
"build" : "cdd3ac4",
"http_address" : "inet[localhost/127.0.0.1:9200]",
"attributes" : {
"max_local_storage_nodes" : "1"
},
"jvm" : {
"pid" : 6763,
"version" : "1.7.0_111",
"vm_name" : "OpenJDK 64-Bit Server VM",
"vm_version" : "24.111-b01",
"vm_vendor" : "Oracle Corporation",
"start_time_in_millis" : 1474470457548,
"mem" : {
"heap_init_in_bytes" : 8362393600,
"heap_max_in_bytes" : 8328577024,
"non_heap_init_in_bytes" : 24313856,
"non_heap_max_in_bytes" : 224395264,
"direct_max_in_bytes" : 8328577024
},
"gc_collectors" : [ "ParNew", "ConcurrentMarkSweep" ],
"memory_pools" : [ "Code Cache", "Par Eden Space", "Par Survivor Space", "CMS Old Gen", "CMS Perm Gen" ]
}
},
"JJlVSkeLSjCWkQJ8t9TT4g" : {
"name" : "f5c0ae96-4dd5-4f16-bd5a-a73b245f7694",
"transport_address" : "inet[/172.25.12.28:9300]",
"host" : "hdfs-365-nag01.hdfs.itciss.com",
"ip" : "172.25.12.28",
"version" : "1.6.0",
"build" : "cdd3ac4",
"http_address" : "inet[localhost/127.0.0.1:9200]",
"attributes" : {
"max_local_storage_nodes" : "1"
},
"jvm" : {
"pid" : 28233,
"version" : "1.7.0_111",
"vm_name" : "OpenJDK 64-Bit Server VM",
"vm_version" : "24.111-b01",
"vm_vendor" : "Oracle Corporation",
"start_time_in_millis" : 1474470455064,
"mem" : {
"heap_init_in_bytes" : 8360296448,
"heap_max_in_bytes" : 8326479872,
"non_heap_init_in_bytes" : 24313856,
"non_heap_max_in_bytes" : 224395264,
"direct_max_in_bytes" : 8326479872
},
"gc_collectors" : [ "ParNew", "ConcurrentMarkSweep" ],
"memory_pools" : [ "Code Cache", "Par Eden Space", "Par Survivor Space", "CMS Old Gen", "CMS Perm Gen" ]
}
}
}
}
Code: Select all
{
"cluster_name" : "1c6d88c6-e9ca-480d-8c85-8bd97ef41adf",
"nodes" : {
"XpBH4IG2S4aMC8qR-gl3Dg" : {
"name" : "91c49ceb-0d7c-4e27-a80a-b9036f0d3375",
"transport_address" : "inet[/172.25.14.21:9300]",
"host" : "hdfs-chi-nag01.hdfs.itciss.com",
"ip" : "172.25.14.21",
"version" : "1.6.0",
"build" : "cdd3ac4",
"http_address" : "inet[localhost/127.0.0.1:9200]",
"attributes" : {
"max_local_storage_nodes" : "1"
},
"jvm" : {
"pid" : 6763,
"version" : "1.7.0_111",
"vm_name" : "OpenJDK 64-Bit Server VM",
"vm_version" : "24.111-b01",
"vm_vendor" : "Oracle Corporation",
"start_time_in_millis" : 1474470457548,
"mem" : {
"heap_init_in_bytes" : 8362393600,
"heap_max_in_bytes" : 8328577024,
"non_heap_init_in_bytes" : 24313856,
"non_heap_max_in_bytes" : 224395264,
"direct_max_in_bytes" : 8328577024
},
"gc_collectors" : [ "ParNew", "ConcurrentMarkSweep" ],
"memory_pools" : [ "Code Cache", "Par Eden Space", "Par Survivor Space", "CMS Old Gen", "CMS Perm Gen" ]
}
},
"JJlVSkeLSjCWkQJ8t9TT4g" : {
"name" : "f5c0ae96-4dd5-4f16-bd5a-a73b245f7694",
"transport_address" : "inet[/172.25.12.28:9300]",
"host" : "hdfs-365-nag01.hdfs.itciss.com",
"ip" : "172.25.12.28",
"version" : "1.6.0",
"build" : "cdd3ac4",
"http_address" : "inet[localhost/127.0.0.1:9200]",
"attributes" : {
"max_local_storage_nodes" : "1"
},
"jvm" : {
"pid" : 28233,
"version" : "1.7.0_111",
"vm_name" : "OpenJDK 64-Bit Server VM",
"vm_version" : "24.111-b01",
"vm_vendor" : "Oracle Corporation",
"start_time_in_millis" : 1474470455064,
"mem" : {
"heap_init_in_bytes" : 8360296448,
"heap_max_in_bytes" : 8326479872,
"non_heap_init_in_bytes" : 24313856,
"non_heap_max_in_bytes" : 224395264,
"direct_max_in_bytes" : 8326479872
},
"gc_collectors" : [ "ParNew", "ConcurrentMarkSweep" ],
"memory_pools" : [ "Code Cache", "Par Eden Space", "Par Survivor Space", "CMS Old Gen", "CMS Perm Gen" ]
}
}
}
}
Code: Select all
curl -XGET 'http://localhost:9200/_cluster/health/*?level=shards'
curl -XGET http://localhost:9200/_cat/shards?vCode: Select all
[root@hdfs-chi-nag01 ~]# curl -XGET 'http://localhost:9200/_cluster/health/*?level=shards'
{"cluster_name":"1c6d88c6-e9ca-480d-8c85-8bd97ef41adf","status":"green","timed_out":false,"number_of_nodes":2,"number_of_data_nodes":2,"active_primary_shards":81,"active_shards":162,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"number_of_pending_tasks":0,"number_of_in_flight_fetch":0,"indices":{"logstash-2016.09.17":{"status":"green","number_of_shards":5,"number_of_replicas":1,"active_primary_shards":5,"active_shards":10,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"shards":{"0":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"1":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"2":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"3":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"4":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0}}},"logstash-2016.09.08":{"status":"green","number_of_shards":5,"number_of_replicas":1,"active_primary_shards":5,"active_shards":10,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"shards":{"0":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"1":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"2":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"3":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"4":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0}}},"logstash-2016.09.09":{"status":"green","number_of_shards":5,"number_of_replicas":1,"active_primary_shards":5,"active_shards":10,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"shards":{"0":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"1":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"2":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"3":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"4":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0}}},"logstash-2016.09.18":{"status":"green","number_of_shards":5,"number_of_replicas":1,"active_primary_shards":5,"active_shards":10,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"shards":{"0":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"1":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"2":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"3":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"4":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0}}},"logstash-2016.09.19":{"status":"green","number_of_shards":5,"number_of_replicas":1,"active_primary_shards":5,"active_shards":10,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"shards":{"0":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"1":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"2":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"3":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"4":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0}}},"logstash-2016.09.13":{"status":"green","number_of_shards":5,"number_of_replicas":1,"active_primary_shards":5,"active_shards":10,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"shards":{"0":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"1":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"2":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"3":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"4":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0}}},"nagioslogserver":{"status":"green","number_of_shards":1,"number_of_replicas":1,"active_primary_shards":1,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"shards":{"0":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0}}},"logstash-2016.09.14":{"status":"green","number_of_shards":5,"number_of_replicas":1,"active_primary_shards":5,"active_shards":10,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"shards":{"0":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"1":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"2":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"3":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"4":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0}}},"logstash-2016.09.15":{"status":"green","number_of_shards":5,"number_of_replicas":1,"active_primary_shards":5,"active_shards":10,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"shards":{"0":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"1":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"2":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"3":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"4":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0}}},"logstash-2016.09.16":{"status":"green","number_of_shards":5,"number_of_replicas":1,"active_primary_shards":5,"active_shards":10,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"shards":{"0":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"1":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"2":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"3":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"4":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0}}},"logstash-2016.09.10":{"status":"green","number_of_shards":5,"number_of_replicas":1,"active_primary_shards":5,"active_shards":10,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"shards":{"0":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"1":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"2":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"3":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"4":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0}}},"logstash-2016.09.11":{"status":"green","number_of_shards":5,"number_of_replicas":1,"active_primary_shards":5,"active_shards":10,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"shards":{"0":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"1":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"2":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"3":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"4":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0}}},"logstash-2016.09.12":{"status":"green","number_of_shards":5,"number_of_replicas":1,"active_primary_shards":5,"active_shards":10,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"shards":{"0":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"1":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"2":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"3":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"4":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0}}},"nagioslogserver_log":{"status":"green","number_of_shards":5,"number_of_replicas":1,"active_primary_shards":5,"active_shards":10,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"shards":{"0":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"1":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"2":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"3":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"4":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0}}},"logstash-2016.09.21":{"status":"green","number_of_shards":5,"number_of_replicas":1,"active_primary_shards":5,"active_shards":10,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"shards":{"0":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"1":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"2":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"3":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"4":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0}}},"logstash-2016.09.20":{"status":"green","number_of_shards":5,"number_of_replicas":1,"active_primary_shards":5,"active_shards":10,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"shards":{"0":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"1":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"2":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"3":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"4":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0}}},"kibana-int":{"status":"green","number_of_shards":5,"number_of_replicas":1,"active_primary_shards":5,"active_shards":10,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"shards":{"0":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"1":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"2":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"3":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"4":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0}}}}}[root@hdfs-chi-nag01 ~]# curl -XGET http://localhost:9200/_cat/shards?v
index shard prirep state docs store ip node
logstash-2016.09.17 2 p STARTED 5876 867.4kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.17 2 r STARTED 5876 867.4kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.17 0 p STARTED 5765 890.3kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.17 0 r STARTED 5765 890.3kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.17 3 p STARTED 5865 857.4kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.17 3 r STARTED 5865 857.4kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.17 1 p STARTED 5715 880kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.17 1 r STARTED 5715 880kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.17 4 p STARTED 5923 911.8kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.17 4 r STARTED 5923 911.8kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.08 4 p STARTED 1061 219kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.08 4 r STARTED 1061 219kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.08 0 p STARTED 1063 217kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.08 0 r STARTED 1063 217kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.08 3 p STARTED 1084 225kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.08 3 r STARTED 1084 225kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.08 1 p STARTED 1054 223.7kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.08 1 r STARTED 1054 223.7kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.08 2 p STARTED 1108 229kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.08 2 r STARTED 1108 229kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.09 4 p STARTED 4287 720.3kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.09 4 r STARTED 4287 720.3kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.09 0 p STARTED 4280 728.3kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.09 0 r STARTED 4280 728.3kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.09 3 p STARTED 4175 699.2kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.09 3 r STARTED 4175 699.2kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.09 1 p STARTED 4239 719.8kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.09 1 r STARTED 4239 719.8kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.09 2 p STARTED 4387 745.3kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.09 2 r STARTED 4387 745.3kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.18 4 p STARTED 5809 892.9kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.18 4 r STARTED 5809 892.9kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.18 0 p STARTED 5814 853.2kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.18 0 r STARTED 5814 853.2kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.18 3 p STARTED 5882 907.4kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.18 3 r STARTED 5882 907.4kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.18 1 p STARTED 5751 891.5kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.18 1 r STARTED 5751 891.5kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.18 2 p STARTED 5793 889.9kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.18 2 r STARTED 5793 889.9kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.19 2 p STARTED 5872 896.4kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.19 2 r STARTED 5872 896.4kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.19 0 p STARTED 5827 897kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.19 0 r STARTED 5827 897kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.19 3 p STARTED 5896 908.6kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.19 3 r STARTED 5896 908.6kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.19 1 p STARTED 5901 910.4kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.19 1 r STARTED 5901 910.4kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.19 4 p STARTED 5632 853kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.19 4 r STARTED 5632 853kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.13 4 p STARTED 5816 856.3kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.13 4 r STARTED 5816 856.3kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.13 0 p STARTED 5776 873.7kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.13 0 r STARTED 5776 873.7kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.13 3 p STARTED 5852 889.4kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.13 3 r STARTED 5852 889.4kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.13 1 p STARTED 5792 891kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.13 1 r STARTED 5792 891kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.13 2 p STARTED 5845 874.6kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.13 2 r STARTED 5845 874.6kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
nagioslogserver 0 p STARTED 29 71.9kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
nagioslogserver 0 r STARTED 29 80kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.14 4 p STARTED 5869 909.3kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.14 4 r STARTED 5869 909.3kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.14 0 p STARTED 5816 891.9kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.14 0 r STARTED 5816 891.9kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.14 3 p STARTED 5913 908.6kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.14 3 r STARTED 5913 908.6kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.14 1 p STARTED 5784 897.7kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.14 1 r STARTED 5784 897.7kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.14 2 p STARTED 5849 907.1kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.14 2 r STARTED 5849 907.1kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.15 2 p STARTED 5867 902.4kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.15 2 r STARTED 5867 902.4kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.15 0 p STARTED 6004 905.3kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.15 0 r STARTED 6004 905.3kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.15 3 p STARTED 5648 830.4kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.15 3 r STARTED 5648 830.4kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.15 1 p STARTED 5845 860kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.15 1 r STARTED 5845 860kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.15 4 p STARTED 5855 888.1kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.15 4 r STARTED 5855 888.1kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.16 4 p STARTED 5784 892.2kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.16 4 r STARTED 5784 892.2kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.16 0 p STARTED 5863 862kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.16 0 r STARTED 5863 862kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.16 3 p STARTED 5855 865kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.16 3 r STARTED 5855 865kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.16 1 p STARTED 5775 842.8kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.16 1 r STARTED 5775 842.8kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.16 2 p STARTED 5827 895.2kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.16 2 r STARTED 5827 895.2kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.10 2 p STARTED 5910 879.4kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.10 2 r STARTED 5910 879.4kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.10 0 p STARTED 5792 848.7kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.10 0 r STARTED 5792 848.7kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.10 3 p STARTED 5913 861.1kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.10 3 r STARTED 5913 861.1kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.10 1 p STARTED 5824 889.3kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.10 1 r STARTED 5824 889.3kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.10 4 p STARTED 5799 871.9kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.10 4 r STARTED 5799 871.9kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.11 4 p STARTED 5855 857.7kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.11 4 r STARTED 5855 857.7kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.11 0 p STARTED 5814 851.6kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.11 0 r STARTED 5814 851.6kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.11 3 p STARTED 5818 845.5kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.11 3 r STARTED 5818 845.5kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.11 1 p STARTED 5858 902kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.11 1 r STARTED 5858 902kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.11 2 p STARTED 5927 918.1kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.11 2 r STARTED 5927 918.1kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.12 4 p STARTED 5826 854.8kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.12 4 r STARTED 5826 854.8kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.12 0 p STARTED 5875 864.6kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.12 0 r STARTED 5875 864.6kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.12 3 p STARTED 5795 899.1kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.12 3 r STARTED 5795 899.1kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.12 1 p STARTED 5840 906.9kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.12 1 r STARTED 5840 906.9kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.12 2 p STARTED 5774 863.6kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.12 2 r STARTED 5774 863.6kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
nagioslogserver_log 2 p STARTED 11262 1.8mb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
nagioslogserver_log 2 r STARTED 11262 1.9mb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
nagioslogserver_log 0 p STARTED 11318 1.9mb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
nagioslogserver_log 0 r STARTED 11318 1.9mb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
nagioslogserver_log 3 p STARTED 11423 1.9mb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
nagioslogserver_log 3 r STARTED 11423 1.9mb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
nagioslogserver_log 1 p STARTED 11284 1.9mb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
nagioslogserver_log 1 r STARTED 11284 1.9mb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
nagioslogserver_log 4 p STARTED 11350 1.9mb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
nagioslogserver_log 4 r STARTED 11350 1.9mb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.21 2 p STARTED 24191 9.2mb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.21 2 r STARTED 24191 9.2mb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.21 0 p STARTED 24093 9.1mb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.21 0 r STARTED 24093 9.2mb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.21 3 p STARTED 23979 9.1mb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.21 3 r STARTED 23979 9.2mb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.21 1 p STARTED 23984 9.2mb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.21 1 r STARTED 23984 9.2mb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.21 4 p STARTED 23883 9.2mb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.21 4 r STARTED 23883 9.2mb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.20 4 p STARTED 5892 988.7kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.20 4 r STARTED 5892 988.7kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.20 0 p STARTED 5871 883.5kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.20 0 r STARTED 5871 883.5kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.20 3 p STARTED 5775 963.6kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.20 3 r STARTED 5775 949.7kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.20 1 p STARTED 5806 846kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.20 1 r STARTED 5806 846kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.20 2 p STARTED 5834 874.3kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.20 2 r STARTED 5834 874.3kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
kibana-int 4 p STARTED 0 144b 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
kibana-int 4 r STARTED 0 144b 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
kibana-int 0 p STARTED 2 27.6kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
kibana-int 0 r STARTED 2 27.6kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
kibana-int 3 p STARTED 3 21.3kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
kibana-int 3 r STARTED 3 21.3kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
kibana-int 1 p STARTED 0 144b 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
kibana-int 1 r STARTED 0 144b 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
kibana-int 2 p STARTED 1 11.8kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
kibana-int 2 r STARTED 1 11.8kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
Code: Select all
[root@hdfs-chi-nag01 ~]# curl -XGET http://localhost:9200/_cat/shards?v
index shard prirep state docs store ip node
nagioslogserver_log 4 p STARTED 11351 973.2kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
nagioslogserver_log 4 r STARTED 11351 991.8kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
nagioslogserver_log 0 p STARTED 11318 998kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
nagioslogserver_log 0 r STARTED 11318 991.2kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
nagioslogserver_log 3 p STARTED 11425 1008.4kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
nagioslogserver_log 3 r STARTED 11425 997.2kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
nagioslogserver_log 1 p STARTED 11286 976.8kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
nagioslogserver_log 1 r STARTED 11286 989.7kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
nagioslogserver_log 2 p STARTED 11263 973.8kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
nagioslogserver_log 2 r STARTED 11263 988.6kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.21 2 r STARTED 8 31.4kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.21 2 p STARTED 9 31.5kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.21 0 r STARTED 12 46.5kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.21 0 p STARTED 14 46.6kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.21 3 r STARTED 4 28.4kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.21 3 p STARTED 6 28.5kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.21 1 r STARTED 2 8.3kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.21 1 p STARTED 2 8.3kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.21 4 r STARTED 2 14.1kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
logstash-2016.09.21 4 p STARTED 2 14.2kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
nagioslogserver 0 p STARTED 29 76.6kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
nagioslogserver 0 r STARTED 29 128kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
kibana-int 2 p STARTED 1 11.8kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
kibana-int 2 r STARTED 1 11.8kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
kibana-int 0 p STARTED 2 27.6kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
kibana-int 0 r STARTED 2 27.6kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
kibana-int 3 p STARTED 3 21.3kb 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
kibana-int 3 r STARTED 3 21.3kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
kibana-int 1 p STARTED 0 144b 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
kibana-int 1 r STARTED 0 144b 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
kibana-int 4 p STARTED 0 144b 172.25.14.21 91c49ceb-0d7c-4e27-a80a-b9036f0d3375
kibana-int 4 r STARTED 0 144b 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
[root@hdfs-chi-nag01 ~]# curl -XGET 'http://localhost:9200/_cluster/health/*?level=shards'
{"cluster_name":"1c6d88c6-e9ca-480d-8c85-8bd97ef41adf","status":"green","timed_out":false,"number_of_nodes":2,"number_of_data_nodes":2,"active_primary_shards":16,"active_shards":32,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"number_of_pending_tasks":0,"number_of_in_flight_fetch":0,"indices":{"nagioslogserver_log":{"status":"green","number_of_shards":5,"number_of_replicas":1,"active_primary_shards":5,"active_shards":10,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"shards":{"0":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"1":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"2":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"3":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"4":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0}}},"logstash-2016.09.21":{"status":"green","number_of_shards":5,"number_of_replicas":1,"active_primary_shards":5,"active_shards":10,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"shards":{"0":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"1":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"2":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"3":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"4":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0}}},"nagioslogserver":{"status":"green","number_of_shards":1,"number_of_replicas":1,"active_primary_shards":1,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"shards":{"0":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0}}},"kibana-int":{"status":"green","number_of_shards":5,"number_of_replicas":1,"active_primary_shards":5,"active_shards":10,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"shards":{"0":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"1":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"2":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"3":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0},"4":{"status":"green","primary_active":true,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0}}}}}Code: Select all
cat /usr/local/nagioslogserver/logstash/etc/conf.d/*.conf
cat /usr/local/nagioslogserver/elasticsearch/config/*.yml
Wait, what? Did I explain somethign poorly in my last post or did you get that from looking at my pasted stuff?mcapra wrote:So we're allocating shards to the "Bad Node" just fine, but unable to write data to them.
Code: Select all
[root@hdfs-365-nag01 tmp]# cat /usr/local/nagioslogserver/logstash/etc/conf.d/*.conf
#
# Logstash Configuration File
# Dynamically created by Nagios Log Server
#
# DO NOT EDIT THIS FILE. IT WILL BE OVERWRITTEN.
#
# Created Wed, 21 Sep 2016 07:59:41 -0700
#
#
# Global inputs
#
input {
syslog {
type => 'syslog'
port => 5544
}
tcp {
type => 'eventlog'
port => 3515
codec => json {
charset => 'CP1252'
}
}
tcp {
type => 'import_raw'
tags => 'import_raw'
port => 2056
}
tcp {
type => 'import_json'
tags => 'import_json'
port => 2057
codec => json
}
}
#
# Local inputs
#
#
# Logstash Configuration File
# Dynamically created by Nagios Log Server
#
# DO NOT EDIT THIS FILE. IT WILL BE OVERWRITTEN.
#
# Created Wed, 21 Sep 2016 07:59:41 -0700
#
#
# Global filters
#
filter {
if [program] == 'apache_access' {
grok {
match => [ 'message', '%{COMBINEDAPACHELOG}']
}
date {
match => [ 'timestamp', 'dd/MMM/yyyy:HH:mm:ss Z', 'MMM dd HH:mm:ss', 'ISO8601' ]
}
mutate {
replace => [ 'type', 'apache_access' ]
convert => [ 'bytes', 'integer' ]
convert => [ 'response', 'integer' ]
}
}
if [program] == 'apache_error' {
grok {
match => [ 'message', '\[(?<timestamp>%{DAY:day} %{MONTH:month} %{MONTHDAY} %{TIME} %{YEAR})\] \[%{WORD:class}\] \[%{WORD:originator} %{IP:clientip}\] %{GREEDYDATA:errmsg}']
}
mutate {
replace => [ 'type', 'apache_error' ]
}
}
dns {
reverse => [ "host" ]
action => [ "replace" ]
add_tag => [ "dns" ]
}
if ([message] == "default send string") {
drop{}
}
}
#
# Local filters
#
#
# Logstash Configuration File
# Dynamically created by Nagios Log Server
#
# DO NOT EDIT THIS FILE. IT WILL BE OVERWRITTEN.
#
# Created Wed, 21 Sep 2016 07:59:42 -0700
#
#
# Required output for Nagios Log Server
#
output {
elasticsearch {
cluster => '1c6d88c6-e9ca-480d-8c85-8bd97ef41adf'
host => 'localhost'
document_type => '%{type}'
node_name => 'f5c0ae96-4dd5-4f16-bd5a-a73b245f7694'
protocol => 'transport'
workers => 4
}
}
#
# Global outputs
#
#
# Local outputs
#
[root@hdfs-365-nag01 tmp]# cat /usr/local/nagioslogserver/elasticsearch/config/*.yml
##################### Elasticsearch Configuration Example #####################
# This file contains an overview of various configuration settings,
# targeted at operations staff. Application developers should
# consult the guide at <http://elasticsearch.org/guide>.
#
# The installation procedure is covered at
# <http://elasticsearch.org/guide/en/elasticsearch/reference/current/setup.html>.
#
# Elasticsearch comes with reasonable defaults for most settings,
# so you can try it out without bothering with configuration.
#
# Most of the time, these defaults are just fine for running a production
# cluster. If you're fine-tuning your cluster, or wondering about the
# effect of certain configuration option, please _do ask_ on the
# mailing list or IRC channel [http://elasticsearch.org/community].
# Any element in the configuration can be replaced with environment variables
# by placing them in ${...} notation. For example:
#
# node.rack: ${RACK_ENV_VAR}
# For information on supported formats and syntax for the config file, see
# <http://elasticsearch.org/guide/en/elasticsearch/reference/current/setup-configuration.html>
################################### Cluster ###################################
# Cluster name identifies your cluster for auto-discovery. If you're running
# multiple clusters on the same network, make sure you're using unique names.
#
cluster.name: nagios_elasticsearch
#################################### Node #####################################
# Node names are generated dynamically on startup, so you're relieved
# from configuring them manually. You can tie this node to a specific name:
#
# node.name: "Franz Kafka"
# Every node can be configured to allow or deny being eligible as the master,
# and to allow or deny to store the data.
#
# Allow this node to be eligible as a master node (enabled by default):
#
# node.master: true
#
# Allow this node to store data (enabled by default):
#
# node.data: true
# You can exploit these settings to design advanced cluster topologies.
#
# 1. You want this node to never become a master node, only to hold data.
# This will be the "workhorse" of your cluster.
#
# node.master: false
# node.data: true
#
# 2. You want this node to only serve as a master: to not store any data and
# to have free resources. This will be the "coordinator" of your cluster.
#
# node.master: true
# node.data: false
#
# 3. You want this node to be neither master nor data node, but
# to act as a "search load balancer" (fetching data from nodes,
# aggregating results, etc.)
#
# node.master: false
# node.data: false
# Use the Cluster Health API [http://localhost:9200/_cluster/health], the
# Node Info API [http://localhost:9200/_nodes] or GUI tools
# such as <http://www.elasticsearch.org/overview/marvel/>,
# <http://github.com/karmi/elasticsearch-paramedic>,
# <http://github.com/lukas-vlcek/bigdesk> and
# <http://mobz.github.com/elasticsearch-head> to inspect the cluster state.
# A node can have generic attributes associated with it, which can later be used
# for customized shard allocation filtering, or allocation awareness. An attribute
# is a simple key value pair, similar to node.key: value, here is an example:
#
# node.rack: rack314
# By default, multiple nodes are allowed to start from the same installation location
# to disable it, set the following:
node.max_local_storage_nodes: 1
#################################### Index ####################################
# You can set a number of options (such as shard/replica options, mapping
# or analyzer definitions, translog settings, ...) for indices globally,
# in this file.
#
# Note, that it makes more sense to configure index settings specifically for
# a certain index, either when creating it or by using the index templates API.
#
# See <http://elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules.html> and
# <http://elasticsearch.org/guide/en/elasticsearch/reference/current/indices-create-index.html>
# for more information.
# Set the number of shards (splits) of an index (5 by default):
#
# index.number_of_shards: 5
# Set the number of replicas (additional copies) of an index (1 by default):
#
# index.number_of_replicas: 1
# Note, that for development on a local machine, with small indices, it usually
# makes sense to "disable" the distributed features:
#
# index.number_of_shards: 1
# index.number_of_replicas: 0
# These settings directly affect the performance of index and search operations
# in your cluster. Assuming you have enough machines to hold shards and
# replicas, the rule of thumb is:
#
# 1. Having more *shards* enhances the _indexing_ performance and allows to
# _distribute_ a big index across machines.
# 2. Having more *replicas* enhances the _search_ performance and improves the
# cluster _availability_.
#
# The "number_of_shards" is a one-time setting for an index.
#
# The "number_of_replicas" can be increased or decreased anytime,
# by using the Index Update Settings API.
#
# Elasticsearch takes care about load balancing, relocating, gathering the
# results from nodes, etc. Experiment with different settings to fine-tune
# your setup.
# Use the Index Status API (<http://localhost:9200/A/_status>) to inspect
# the index status.
#################################### Paths ####################################
# Path to directory containing configuration (this file and logging.yml):
#
# path.conf: /path/to/conf
# Path to directory where to store index data allocated for this node.
#
# path.data: /path/to/data
#
# Can optionally include more than one location, causing data to be striped across
# the locations (a la RAID 0) on a file level, favouring locations with most free
# space on creation. For example:
#
# path.data: /path/to/data1,/path/to/data2
# Path to temporary files:
#
# path.work: /path/to/work
# Path to log files:
#
# path.logs: /path/to/logs
# Path to where plugins are installed:
#
# path.plugins: /path/to/plugins
#################################### Plugin ###################################
# If a plugin listed here is not installed for current node, the node will not start.
#
# plugin.mandatory: mapper-attachments,lang-groovy
################################### Memory ####################################
# Elasticsearch performs poorly when JVM starts swapping: you should ensure that
# it _never_ swaps.
#
# Set this property to true to lock the memory:
#
bootstrap.mlockall: true
# Make sure that the ES_MIN_MEM and ES_MAX_MEM environment variables are set
# to the same value, and that the machine has enough memory to allocate
# for Elasticsearch, leaving enough memory for the operating system itself.
#
# You should also make sure that the Elasticsearch process is allowed to lock
# the memory, eg. by using `ulimit -l unlimited`.
############################## Network And HTTP ###############################
# Elasticsearch, by default, binds itself to the 0.0.0.0 address, and listens
# on port [9200-9300] for HTTP traffic and on port [9300-9400] for node-to-node
# communication. (the range means that if the port is busy, it will automatically
# try the next port).
# Set the bind address specifically (IPv4 or IPv6):
#
# network.bind_host: 192.168.0.1
# Set the address other nodes will use to communicate with this node. If not
# set, it is automatically derived. It must point to an actual IP address.
#
# network.publish_host: 192.168.0.1
# Set both 'bind_host' and 'publish_host':
#
# network.host: 192.168.0.1
# Set a custom port for the node to node communication (9300 by default):
#
# transport.tcp.port: 9300
# Enable compression for all communication between nodes (disabled by default):
#
transport.tcp.compress: true
# Set a custom port to listen for HTTP traffic:
#
# http.port: 9200
# Set a custom allowed content length:
#
# http.max_content_length: 100mb
# Disable HTTP completely:
#
# http.enabled: false
# Set the HTTP host to listen to
#
http.host: "localhost"
################################### Gateway ###################################
# The gateway allows for persisting the cluster state between full cluster
# restarts. Every change to the state (such as adding an index) will be stored
# in the gateway, and when the cluster starts up for the first time,
# it will read its state from the gateway.
# There are several types of gateway implementations. For more information, see
# <http://elasticsearch.org/guide/en/elasticsearch/reference/current/modules-gateway.html>.
# The default gateway type is the "local" gateway (recommended):
#
# gateway.type: local
# Settings below control how and when to start the initial recovery process on
# a full cluster restart (to reuse as much local data as possible when using shared
# gateway).
# Allow recovery process after N nodes in a cluster are up:
#
# gateway.recover_after_nodes: 1
# Set the timeout to initiate the recovery process, once the N nodes
# from previous setting are up (accepts time value):
#
# gateway.recover_after_time: 5m
# Set how many nodes are expected in this cluster. Once these N nodes
# are up (and recover_after_nodes is met), begin recovery process immediately
# (without waiting for recover_after_time to expire):
#
# gateway.expected_nodes: 2
############################# Recovery Throttling #############################
# These settings allow to control the process of shards allocation between
# nodes during initial recovery, replica allocation, rebalancing,
# or when adding and removing nodes.
# Set the number of concurrent recoveries happening on a node:
#
# 1. During the initial recovery
#
# cluster.routing.allocation.node_initial_primaries_recoveries: 4
#
# 2. During adding/removing nodes, rebalancing, etc
#
# cluster.routing.allocation.node_concurrent_recoveries: 2
# Set to throttle throughput when recovering (eg. 100mb, by default 20mb):
#
# indices.recovery.max_bytes_per_sec: 20mb
# Set to limit the number of open concurrent streams when
# recovering a shard from a peer:
#
# indices.recovery.concurrent_streams: 5
################################## Discovery ##################################
# Discovery infrastructure ensures nodes can be found within a cluster
# and master node is elected. Multicast discovery is the default.
# Set to ensure a node sees N other master eligible nodes to be considered
# operational within the cluster. Its recommended to set it to a higher value
# than 1 when running more than 2 nodes in the cluster.
#
# discovery.zen.minimum_master_nodes: 1
# Set the time to wait for ping responses from other nodes when discovering.
# Set this option to a higher value on a slow or congested network
# to minimize discovery failures:
#
# discovery.zen.ping.timeout: 3s
# For more information, see
# <http://elasticsearch.org/guide/en/elasticsearch/reference/current/modules-discovery-zen.html>
# Unicast discovery allows to explicitly control which nodes will be used
# to discover the cluster. It can be used when multicast is not present,
# or to restrict the cluster communication-wise.
#
# 1. Disable multicast discovery (enabled by default):
#
discovery.zen.ping.multicast.enabled: false
#
# 2. Configure an initial list of master nodes in the cluster
# to perform discovery when new nodes (master or data) are started:
#
discovery.zen.ping.unicast.hosts: ["localhost"]
# EC2 discovery allows to use AWS EC2 API in order to perform discovery.
#
# You have to install the cloud-aws plugin for enabling the EC2 discovery.
#
# For more information, see
# <http://elasticsearch.org/guide/en/elasticsearch/reference/current/modules-discovery-ec2.html>
#
# See <http://elasticsearch.org/tutorials/elasticsearch-on-ec2/>
# for a step-by-step tutorial.
# GCE discovery allows to use Google Compute Engine API in order to perform discovery.
#
# You have to install the cloud-gce plugin for enabling the GCE discovery.
#
# For more information, see <https://github.com/elasticsearch/elasticsearch-cloud-gce>.
# Azure discovery allows to use Azure API in order to perform discovery.
#
# You have to install the cloud-azure plugin for enabling the Azure discovery.
#
# For more information, see <https://github.com/elasticsearch/elasticsearch-cloud-azure>.
################################## Slow Log ##################################
# Shard level query and fetch threshold logging.
#index.search.slowlog.threshold.query.warn: 10s
#index.search.slowlog.threshold.query.info: 5s
#index.search.slowlog.threshold.query.debug: 2s
#index.search.slowlog.threshold.query.trace: 500ms
#index.search.slowlog.threshold.fetch.warn: 1s
#index.search.slowlog.threshold.fetch.info: 800ms
#index.search.slowlog.threshold.fetch.debug: 500ms
#index.search.slowlog.threshold.fetch.trace: 200ms
#index.indexing.slowlog.threshold.index.warn: 10s
#index.indexing.slowlog.threshold.index.info: 5s
#index.indexing.slowlog.threshold.index.debug: 2s
#index.indexing.slowlog.threshold.index.trace: 500ms
################################## GC Logging ################################
#monitor.jvm.gc.young.warn: 1000ms
#monitor.jvm.gc.young.info: 700ms
#monitor.jvm.gc.young.debug: 400ms
#monitor.jvm.gc.old.warn: 10s
#monitor.jvm.gc.old.info: 5s
#monitor.jvm.gc.old.debug: 2s
# you can override this using by setting a system property, for example -Des.logger.level=DEBUG
es.logger.level: INFO
rootLogger: ${es.logger.level}, console, file
logger:
# log action execution errors for easier debugging
action: DEBUG
# reduce the logging for aws, too much is logged under the default INFO
com.amazonaws: WARN
# gateway
#gateway: DEBUG
#index.gateway: DEBUG
# peer shard recovery
#indices.recovery: DEBUG
# discovery
#discovery: TRACE
index.search.slowlog: TRACE, index_search_slow_log_file
index.indexing.slowlog: TRACE, index_indexing_slow_log_file
additivity:
index.search.slowlog: false
index.indexing.slowlog: false
appender:
console:
type: console
layout:
type: consolePattern
conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"
file:
type: file
file: ${path.logs}/${cluster.name}.log
# datePattern: "'.'yyyy-MM-dd"
layout:
type: pattern
conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"
index_search_slow_log_file:
type: dailyRollingFile
file: ${path.logs}/${cluster.name}_index_search_slowlog.log
datePattern: "'.'yyyy-MM-dd"
layout:
type: pattern
conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"
index_indexing_slow_log_file:
type: dailyRollingFile
file: ${path.logs}/${cluster.name}_index_indexing_slowlog.log
datePattern: "'.'yyyy-MM-dd"
layout:
type: pattern
conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"
The output of curl -XGET http://localhost:9200/_cat/shards?v shows shards being assigned to 172.25.12.28 (hdfs-365-nag01.hdfs.itciss.com):BanditBBS wrote: Wait, what? Did I explain somethign poorly in my last post or did you get that from looking at my pasted stuff?
Code: Select all
logstash-2016.09.21 2 p STARTED 9 31.5kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
logstash-2016.09.21 0 p STARTED 14 46.6kb 172.25.12.28 f5c0ae96-4dd5-4f16-bd5a-a73b245f7694
Code: Select all
[root@hdfs-365-nag01 ~]# ll /var/log/elasticsearch/
total 64
-rw-r--r--. 1 nagios users 0 Sep 8 15:48 043f21b0-1d53-4881-8f3e-9c58be028b8d_index_indexing_slowlog.log
-rw-r--r--. 1 nagios users 0 Sep 8 15:48 043f21b0-1d53-4881-8f3e-9c58be028b8d_index_search_slowlog.log
-rw-r--r--. 1 nagios users 0 Sep 10 06:07 043f21b0-1d53-4881-8f3e-9c58be028b8d.log
-rw-r--r--. 1 nagios users 691 Sep 10 06:07 043f21b0-1d53-4881-8f3e-9c58be028b8d.log-20160910.gz
-rw-r--r--. 1 nagios users 0 Sep 8 15:50 1c6d88c6-e9ca-480d-8c85-8bd97ef41adf_index_indexing_slowlog.log
-rw-r--r--. 1 nagios users 0 Sep 8 15:50 1c6d88c6-e9ca-480d-8c85-8bd97ef41adf_index_search_slowlog.log
-rw-r--r--. 1 nagios users 26834 Sep 21 18:00 1c6d88c6-e9ca-480d-8c85-8bd97ef41adf.log
-rw-r--r--. 1 nagios users 2412 Sep 15 07:44 1c6d88c6-e9ca-480d-8c85-8bd97ef41adf.log-20160915.gz
-rw-r--r--. 1 nagios users 2251 Sep 16 06:33 1c6d88c6-e9ca-480d-8c85-8bd97ef41adf.log-20160916.gz
-rw-r--r--. 1 nagios users 2347 Sep 17 06:31 1c6d88c6-e9ca-480d-8c85-8bd97ef41adf.log-20160917.gz
-rw-r--r--. 1 nagios users 3388 Sep 18 06:12 1c6d88c6-e9ca-480d-8c85-8bd97ef41adf.log-20160918.gz
-rw-r--r--. 1 nagios users 2503 Sep 19 06:29 1c6d88c6-e9ca-480d-8c85-8bd97ef41adf.log-20160919.gz
-rw-r--r--. 1 nagios users 3737 Sep 20 06:18 1c6d88c6-e9ca-480d-8c85-8bd97ef41adf.log-20160920.gz
-rw-r--r--. 1 nagios users 2323 Sep 21 06:50 1c6d88c6-e9ca-480d-8c85-8bd97ef41adf.log-20160921.gz