NLS not accepting messages

This support forum board is for support questions relating to Nagios Log Server, our solution for managing and monitoring critical log data.
Locked
Monoman
Posts: 14
Joined: Fri Feb 27, 2015 8:37 am

Re: NLS not accepting messages

Post by Monoman »

I had the VM off so I powered it up, a waited a min or two for things to settle, and then ran the commands you requested.

Code: Select all

[root@testnls3 ~]# service logstash restart
Restarting Logstash Daemon:                                [  OK  ]
                                                           [  OK  ]
This follows a few seconds after

Code: Select all

[root@testnls3 ~]# Exception in thread ">output" org.elasticsearch.client.transport.NoNodeAvailableException: No node available
        at org.elasticsearch.client.transport.TransportClientNodesService.execute(org/elasticsearch/client/transport/TransportClientNodesService.java:219)
        at org.elasticsearch.client.transport.support.InternalTransportIndicesAdminClient.execute(org/elasticsearch/client/transport/support/InternalTransportIndicesAdminClient.java:85)
        at org.elasticsearch.client.support.AbstractIndicesAdminClient.getTemplates(org/elasticsearch/client/support/AbstractIndicesAdminClient.java:544)
        at org.elasticsearch.action.admin.indices.template.get.GetIndexTemplatesRequestBuilder.doExecute(org/elasticsearch/action/admin/indices/template/get/GetIndexTemplatesRequestBuilder.java:41)
        at org.elasticsearch.action.ActionRequestBuilder.execute(org/elasticsearch/action/ActionRequestBuilder.java:85)
        at org.elasticsearch.action.ActionRequestBuilder.execute(org/elasticsearch/action/ActionRequestBuilder.java:59)
        at org.elasticsearch.action.ActionRequestBuilder.get(org/elasticsearch/action/ActionRequestBuilder.java:67)
        at java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:606)
        at RUBY.template_exists?(/usr/local/nagioslogserver/logstash/lib/logstash/outputs/elasticsearch/protocol.rb:231)
        at RUBY.template_install(/usr/local/nagioslogserver/logstash/lib/logstash/outputs/elasticsearch/protocol.rb:21)
        at RUBY.register(/usr/local/nagioslogserver/logstash/lib/logstash/outputs/elasticsearch.rb:259)
        at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1613)
        at RUBY.outputworker(/usr/local/nagioslogserver/logstash/lib/logstash/pipeline.rb:220)
        at RUBY.start_outputs(/usr/local/nagioslogserver/logstash/lib/logstash/pipeline.rb:152)
        at java.lang.Thread.run(java/lang/Thread.java:745)

Code: Select all

[root@testnls3 ~]# netstat -na |grep LISTEN
tcp        0      0 0.0.0.0:22                  0.0.0.0:*                   LISTEN
tcp        0      0 127.0.0.1:25                0.0.0.0:*                   LISTEN
tcp        0      0 :::2056                     :::*                        LISTEN
tcp        0      0 :::5544                     :::*                        LISTEN
tcp        0      0 :::2057                     :::*                        LISTEN
tcp        0      0 ::ffff:127.0.0.1:9200       :::*                        LISTEN
tcp        0      0 :::80                       :::*                        LISTEN
tcp        0      0 :::9300                     :::*                        LISTEN
tcp        0      0 :::22                       :::*                        LISTEN
tcp        0      0 :::3515                     :::*                        LISTEN
unix  2      [ ACC ]     STREAM     LISTENING     6509   @/com/ubuntu/upstart
[root@testnls3 ~]#

Code: Select all

[root@testnls3 ~]# cat /usr/local/nagioslogserver/logstash/etc/conf.d/000_inputs.conf
#
# Logstash Configuration File
# Dynamically created by Nagios Log Server
#
# DO NOT EDIT THIS FILE. IT WILL BE OVERWRITTEN.
#
# Created Wed, 04 Mar 2015 19:35:40 -0500
#

#
# Global inputs
#

input {
    syslog {
        type => 'syslog'
        port => 5544
    }
    tcp {
        type => 'eventlog'
        port => 3515
        codec => json {
            charset => 'CP1252'
        }
    }
    tcp {
        type => 'import_raw'
        tags => 'import_raw'
        port => 2056
    }
    tcp {
        type => 'import_json'
        tags => 'import_json'
        port => 2057
        codec => json
    }
}

#
# Local inputs
#


[root@testnls3 ~]#

Code: Select all

[root@testnls3 ~]# cat /etc/sysconfig/iptables
# Generated by iptables-save v1.4.7 on Thu Feb 12 18:24:24 2015
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -p tcp -m state --state NEW -m tcp --dport 2057 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 2056 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 5544 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 3515 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 9300:9400 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 443 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 80 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 5000 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 514 -j ACCEPT
COMMIT
# Completed on Thu Feb 12 18:24:24 2015

[root@testnls3 ~]#

Code: Select all

[root@testnls3 ~]# tail /var/log/logstash/logstash.log
log4j, [2015-03-06T12:51:05.649]  WARN: org.elasticsearch.client.transport: [a0d0b751-e485-4ab8-b63f-c6a913b96a60] node [#transport#-1][testnls3][inet[localhost/127.0.0.1:9300]] not part of the cluster Cluster [bbe627a5-936a-46e9-a076-2fdc7bf43850], ignoring...
log4j, [2015-03-06T12:51:10.650]  WARN: org.elasticsearch.client.transport: [a0d0b751-e485-4ab8-b63f-c6a913b96a60] node [#transport#-1][testnls3][inet[localhost/127.0.0.1:9300]] not part of the cluster Cluster [bbe627a5-936a-46e9-a076-2fdc7bf43850], ignoring...
log4j, [2015-03-06T12:51:15.652]  WARN: org.elasticsearch.client.transport: [a0d0b751-e485-4ab8-b63f-c6a913b96a60] node [#transport#-1][testnls3][inet[localhost/127.0.0.1:9300]] not part of the cluster Cluster [bbe627a5-936a-46e9-a076-2fdc7bf43850], ignoring...
log4j, [2015-03-06T12:51:20.654]  WARN: org.elasticsearch.client.transport: [a0d0b751-e485-4ab8-b63f-c6a913b96a60] node [#transport#-1][testnls3][inet[localhost/127.0.0.1:9300]] not part of the cluster Cluster [bbe627a5-936a-46e9-a076-2fdc7bf43850], ignoring...
log4j, [2015-03-06T12:51:25.655]  WARN: org.elasticsearch.client.transport: [a0d0b751-e485-4ab8-b63f-c6a913b96a60] node [#transport#-1][testnls3][inet[localhost/127.0.0.1:9300]] not part of the cluster Cluster [bbe627a5-936a-46e9-a076-2fdc7bf43850], ignoring...
log4j, [2015-03-06T12:51:30.657]  WARN: org.elasticsearch.client.transport: [a0d0b751-e485-4ab8-b63f-c6a913b96a60] node [#transport#-1][testnls3][inet[localhost/127.0.0.1:9300]] not part of the cluster Cluster [bbe627a5-936a-46e9-a076-2fdc7bf43850], ignoring...
log4j, [2015-03-06T12:51:35.659]  WARN: org.elasticsearch.client.transport: [a0d0b751-e485-4ab8-b63f-c6a913b96a60] node [#transport#-1][testnls3][inet[localhost/127.0.0.1:9300]] not part of the cluster Cluster [bbe627a5-936a-46e9-a076-2fdc7bf43850], ignoring...
log4j, [2015-03-06T12:51:40.663]  WARN: org.elasticsearch.client.transport: [a0d0b751-e485-4ab8-b63f-c6a913b96a60] node [#transport#-1][testnls3][inet[localhost/127.0.0.1:9300]] not part of the cluster Cluster [bbe627a5-936a-46e9-a076-2fdc7bf43850], ignoring...
log4j, [2015-03-06T12:51:45.672]  WARN: org.elasticsearch.client.transport: [a0d0b751-e485-4ab8-b63f-c6a913b96a60] node [#transport#-1][testnls3][inet[localhost/127.0.0.1:9300]] not part of the cluster Cluster [bbe627a5-936a-46e9-a076-2fdc7bf43850], ignoring...
log4j, [2015-03-06T12:51:50.689]  WARN: org.elasticsearch.client.transport: [a0d0b751-e485-4ab8-b63f-c6a913b96a60] node [#transport#-1][testnls3][inet[localhost/127.0.0.1:9300]] not part of the cluster Cluster [bbe627a5-936a-46e9-a076-2fdc7bf43850], ignoring...
[root@testnls3 ~]#
Top
jolson
Attack Rabbit
Posts: 2560
Joined: Thu Feb 12, 2015 12:40 pm

Re: NLS not accepting messages

Post by jolson »

In your /usr/local/nagioslogserver/logstash/etc/conf.d/000_inputs.conf file, I do not see the proper ports (5000 and 514) as listening. Please log into the NLS GUI and navigate to Administration > Global Configuration. At this page, add a new input and write the following into it:

Code: Select all

tcp {
    type => 'test'
    port => 5000
}
After adding this port, please 'Apply Configuration' and logs should start coming through on port 5000. Please let me know if that works. Thanks!
Twits Blog
Show me a man who lives alone and has a perpetually clean kitchen, and 8 times out of 9 I'll show you a man with detestable spiritual qualities.
Top
Monoman
Posts: 14
Joined: Fri Feb 27, 2015 8:37 am

Re: NLS not accepting messages

Post by Monoman »

I believe I already added that and some others using the GUI.

edit: I added single quotes but it didn't help.

Code: Select all

# 
# Logstash Configuration File
# Dynamically created by Nagios Log Server
#
# DO NOT EDIT THIS FILE. IT WILL BE OVERWRITTEN.
#
# Created Fri, 06 Mar 2015 16:43:59 -0500
#

#
# Global inputs
#

input {
    syslog {
        type => 'syslog'
        port => 5544
    }
    tcp {
        type => 'eventlog'
        port => 3515
        codec => json {
            charset => 'CP1252'
        }
    }
    tcp {
        type => 'import_raw'
        tags => 'import_raw'
        port => 2056
    }
    tcp {
        type => 'import_json'
        tags => 'import_json'
        port => 2057
        codec => json
    }
     tcp {
                    port => 5000
                    type => syslogTCP5000
                    }
    tcp {
                    port => 514
                    type => syslogTCP514
                    }
    udp {
                    port => 514
                    type => syslogUDP514
                    }
}

#
# Local inputs
#
Top
jolson
Attack Rabbit
Posts: 2560
Joined: Thu Feb 12, 2015 12:40 pm

Re: NLS not accepting messages

Post by jolson »

After adding them in the GUI, did you apply your configuration? If not, please do so. Please do not edit that inputs file manually as Apply Configuration will re-write it.
Twits Blog
Show me a man who lives alone and has a perpetually clean kitchen, and 8 times out of 9 I'll show you a man with detestable spiritual qualities.
Top
Monoman
Posts: 14
Joined: Fri Feb 27, 2015 8:37 am

Re: NLS not accepting messages

Post by Monoman »

Yes I did add them through the GUI and clicked Apply.

I have another project taking priority for the next week or so. I may not be able to test as frequently for a week or so.
Top
jolson
Attack Rabbit
Posts: 2560
Joined: Thu Feb 12, 2015 12:40 pm

Re: NLS not accepting messages

Post by jolson »

Your configs look good to me.

Does the following command show listeners on port 5000/514:

Code: Select all

netstat -na|grep LISTEN
Let us know. Ultimately we need to ensure that Logstash is listening on the ports you want it to, otherwise the logs will never reach the system for processing. Below is an example netstat of a functioning log server:

Code: Select all

netstat -na|grep LISTEN
tcp        0      0 0.0.0.0:22                  0.0.0.0:*                   LISTEN
tcp        0      0 127.0.0.1:25                0.0.0.0:*                   LISTEN
tcp        0      0 :::2056                     :::*                        LISTEN
tcp        0      0 :::5544                     :::*                        LISTEN
tcp        0      0 :::2057                     :::*                        LISTEN
tcp        0      0 ::ffff:127.0.0.1:9200       :::*                        LISTEN
tcp        0      0 :::80                       :::*                        LISTEN
tcp        0      0 :::9300                     :::*                        LISTEN
tcp        0      0 :::22                       :::*                        LISTEN
tcp        0      0 :::3515                     :::*                        LISTEN
tcp        0      0 :::3516                     :::*                        LISTEN
I have Logstash listening on all of the above ports, and for your server to work properly, we need to get that handled. If you do not see listening on the proper ports, I ask that you run:

Code: Select all

service logstash restart
netstat -na|grep LISTEN
And check once more. Let me know if that helps.

EDIT: Also, review the following page to learn about setting the 'type' appropriately for your data: http://logstash.net/docs/1.4.2/inputs/file#type
Twits Blog
Show me a man who lives alone and has a perpetually clean kitchen, and 8 times out of 9 I'll show you a man with detestable spiritual qualities.
Top
Monoman
Posts: 14
Joined: Fri Feb 27, 2015 8:37 am

Re: NLS not accepting messages

Post by Monoman »

I apologize for the delayed response but we have not had the time to get back to this evaluation. Unfortunately it looks like it will be some time before we can get back to evaluating NLS. Thank you for taking the time to assist.
Top
jolson
Attack Rabbit
Posts: 2560
Joined: Thu Feb 12, 2015 12:40 pm

Re: NLS not accepting messages

Post by jolson »

No problem - thank you for the follow up. I will lock this thread - please feel free to open a new one if needed. Thanks!
Twits Blog
Show me a man who lives alone and has a perpetually clean kitchen, and 8 times out of 9 I'll show you a man with detestable spiritual qualities.
Top
Locked

Return to “Nagios Log Server”