Logstash not starting due config error

[dclsystems]

Port 5544 is not active, logs not showing.
Logstash and Global config -> Verify button is throwing error:

{:timestamp=>"2018-11-29T18:23:14.704000-0700", :message=>"The given configuration is invalid. Reason: Couldn't find any input plugin named 'tcp'. Are you sure this is correct? Trying to load the tcp input plugin resulted in this error: cannot link Java class org.logstash.tcp.Decoder needs Java 8 (java.lang.UnsupportedClassVersionError: org/logstash/tcp/Decoder : Unsupported major.minor version 52.0)", :level=>:fatal}

How to interpret that?

Thanks
DCL

[mcapra]

Can you share a copy of the exact input/filter/output rules you're attempting to apply?

Basically, that message means that a particular Logstash plugin (in this case logstash-input-tcp) is out-of-sync with the installed Java/JRuby runtime. If the Java runtime was modified recently (via package installs and the like), that's probably the cause.

Can you share the outputs of the following commands executed from *each* of your Nagios Log Server nodes (if more than 1):

Code: Select all

java -version
rpm -qa | grep java
/usr/local/nagioslogserver/logstash/bin/logstash --version
/usr/local/nagioslogserver/logstash/bin/logstash-plugin list --verbose | grep tcp

Sample outputs from a 2.0.6 OVA installation:

Code: Select all

[root@localhost ~]# java -version
openjdk version "1.8.0_191"
OpenJDK Runtime Environment (build 1.8.0_191-b12)
OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)

[root@localhost ~]# rpm -qa | grep java
tzdata-java-2018f-2.el7.noarch
python-javapackages-3.4.1-11.el7.noarch
java-1.8.0-openjdk-headless-1.8.0.191.b12-0.el7_5.x86_64
java-1.8.0-openjdk-1.8.0.191.b12-0.el7_5.x86_64
javapackages-tools-3.4.1-11.el7.noarch

[root@localhost ~]# /usr/local/nagioslogserver/logstash/bin/logstash --version
logstash 2.4.1

[root@localhost ~]# /usr/local/nagioslogserver/logstash/bin/logstash-plugin list --verbose | grep tcp
logstash-input-tcp (4.2.4)
logstash-output-tcp (2.0.4)

Additionally, a couple questions:

• How was Nagios Log Server originally installed on your node(s)? (OVA, online source, offline, etc)
• Was the Java runtime on this machine modified recently?
• Were any Logstash plugins manually updated?
• Was Nagios Log Server updated recently?

[dclsystems]

Commands output from my server:
[root@logs ~]# java -version
java version "1.7.0_191"
OpenJDK Runtime Environment (rhel-2.6.15.4.el7_5-x86_64 u191-b01)
OpenJDK 64-Bit Server VM (build 24.191-b01, mixed mode)
[root@logs ~]# rpm -qa | grep java
javapackages-tools-3.4.1-11.el7.noarch
java-1.8.0-openjdk-1.8.0.191.b12-0.el7_5.x86_64
java-1.7.0-openjdk-1.7.0.191-2.6.15.4.el7_5.x86_64
java-1.8.0-openjdk-headless-1.8.0.191.b12-0.el7_5.x86_64
java-1.7.0-openjdk-headless-1.7.0.191-2.6.15.4.el7_5.x86_64
python-javapackages-3.4.1-11.el7.noarch
tzdata-java-2018f-2.el7.noarch
[root@logs ~]# /usr/local/nagioslogserver/logstash/bin/logstash --version
logstash 2.4.1
[root@logs ~]# /usr/local/nagioslogserver/logstash/bin/logstash-plugin list --verbose | grep tcp
logstash-input-tcp (4.2.4)
logstash-output-tcp (2.0.4)

[dclsystems]

Answers:
How was Nagios Log Server originally installed on your node(s)? (OVA, online source, offline, etc)
NagiosLog is installed on physical machine and Centos 7
Was the Java runtime on this machine modified recently?
No, but we perform regular upgrades following https://assets.nagios.com/downloads/nag ... Server.pdf
Were any Logstash plugins manually updated?
No.
Was Nagios Log Server updated recently ?
Yes, last week.

Thanks
DCL

[cdienger]

Install java 1.7.0:

yum -y install java-1.7.0-openjdk
yum -y install java-1.7.0-openjdk-headless

Then point /etc/alternatives/java at /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.191-2.6.15.4.el7_5.x86_64/jre/bin/java:

ln -sfn /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.191-2.6.15.4.el7_5.x86_64/jre/bin/java /etc/alternatives/java

verify it points to the 1.7 version with:

java -version

Restart elasticsearch and logstash:

service elasticsearch restart
service logstash restart

[dclsystems]

Changing version to 1.7 caused java tcp class issue.
Following solved my problem:
rm /etc/alternatives/java
ln -sfn /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-1.el7_6.x86_64/jre/bin/java /etc/alternatives/java
java -version
[root@logs jvm]# java -version
openjdk version "1.8.0_191"
OpenJDK Runtime Environment (build 1.8.0_191-b12)
OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)

[root@logs java-1.8.0-openjdk-1.8.0.191.b12-1.el7_6.x86_64]# ldd /bin/java
linux-vdso.so.1 => (0x00007ffc017fa000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fe712253000)
libz.so.1 => /lib64/libz.so.1 (0x00007fe71203d000)
libjli.so => not found
libdl.so.2 => /lib64/libdl.so.2 (0x00007fe711e39000)
libc.so.6 => /lib64/libc.so.6 (0x00007fe711a6c000)
/lib64/ld-linux-x86-64.so.2 (0x00007fe71246f000)

cd /lib64/
ls -l libjli.so
lrwxrwxrwx 1 root root 88 Sep 25 09:29 libjli.so -> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.181-3.b13.el7_5.x86_64/jre/lib/amd64/jli/libjli.so
rm libjli.so
ln -s /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-1.el7_6.x86_64/jre/lib/amd64/jli/libjli.so /lib64/libjli.so

Thanks

[cdienger]

Thanks for sharing the solution!