Nagios Support Forum

Support for Nagios products and services
https://support.nagios.com/forum/

Incorrect severity & facility

https://support.nagios.com/forum/viewtopic.php?t=42206

Page 1 of 1

Incorrect severity & facility

Posted: Tue Jan 31, 2017 5:10 am
by rjjimenez
Hi.

We are getting the Juniper VPN logs but the severity and facility fields are not properly handling them in the dashboard.

For example one of the logs is:

13:00:40.512212 IP (tos 0x0, ttl 64, id 27272, offset 0, flags [none], proto UDP (17), length 1018)
192.168.253.236.514 > 192.168.253.34.5544: [udp sum ok] SYSLOG, length: 990
Facility daemon (3), Severity info (6)
Msg: 1 2017-01-31T10:43:22.607+01:00 SRX-240 kmd 1492 KMD_VPN_UP_ALARM_USER [[email protected] vpn-name="INSTANCE-KN_0002_0015_0000" remote-address="57.66.70.251" local-address="192.168.253.236" gateway-name="VPN-to-KN" group-name="KN" tunnel-id="2" interface-name="" internal-ip="Not-Available" name="46.24.17.9" peer-name="57.66.70.251" client-name="Not-Applicable" vrrp-group-id="0" traffic-selector-name="" traffic-selector-cfg-local-id="ipv4_subnet(any:0,[0..7\]=192.168.10.0/24)" traffic-selector-cfg-remote-id="ipv4(any:0,[0..3\]=10.29.67.108)"] VPN INSTANCE-KN_0002_0015_0000 from 57.66.70.251 is up. Local-ip: 192.168.253.236, gateway name: VPN-to-KN, vpn name: KN, tunnel-id: 2, local tunnel-if: , remote tunnel-ip: Not-Available, Local IKE-ID: 46.24.17.9, Remote IKE-ID: 57.66.70.251, XAUTH username: Not-Applicable, VR id: 0, Traffic-selector: , Traffic-selector local ID: ipv4_subnet(any:0,[0..7]=192.168.10.0/24), Traffic-selector remote ID: ipv4(any:0,[0..3]=10.29.67.108)

In the dashboard the following is received:

{
"_index": "logstash-2017.01.31",
"_type": "syslog",
"_id": "AVnz7os4zazFj2I0A50s",
"_score": null,
"_source": {
"message": "<30>1 2017-01-31T08:35:00.690+01:00 SRX-240 kmd 1492 KMD_VPN_UP_ALARM_USER [[email protected] vpn-name=\"INSTANCE-KN_0002_0015_0000\" remote-address=\"57.66.70.251\" local-address=\"192.168.253.236\" gateway-name=\"VPN-to-KN\" group-name=\"KN\" tunnel-id=\"2\" interface-name=\"\" internal-ip=\"Not-Available\" name=\"46.24.17.9\" peer-name=\"57.66.70.251\" client-name=\"Not-Applicable\" vrrp-group-id=\"0\" traffic-selector-name=\"\" traffic-selector-cfg-local-id=\"ipv4_subnet(any:0,[0..7\\]=192.168.10.0/24)\" traffic-selector-cfg-remote-id=\"ipv4(any:0,[0..3\\]=10.29.67.108)\"] VPN INSTANCE-KN_0002_0015_0000 from 57.66.70.251 is up. Local-ip: 192.168.253.236, gateway name: VPN-to-KN, vpn name: KN, tunnel-id: 2, local tunnel-if: , remote tunnel-ip: Not-Available, Local IKE-ID: 46.24.17.9, Remote IKE-ID: 57.66.70.251, XAUTH username: Not-Applicable, VR id: 0, Traffic-selector: , Traffic-selector local ID: ipv4_subnet(any:0,[0..7]=192.168.10.0/24), Traffic-selector remote ID: ipv4(any:0,[0..3]=10.29.67.108)",
"@version": "1",
"@timestamp": "2017-01-31T09:50:12.927Z",
"type": "syslog",
"host": "192.168.253.236",
"tags": [
"_grokparsefailure_sysloginput"
],
"priority": 0,
"severity": 0,
"facility": 0,
"facility_label": "kernel",
"severity_label": "Emergency"
},
"highlight": {
"message": [
"<30>1 2017-01-31T08:35:00.690+01:00 SRX-240 kmd 1492 @start-highlight@KMD_VPN_UP_ALARM_USER@end-highlight@ [[email protected] vpn-name=\"@start-highlight@INSTANCE@end-highlight@-@start-highlight@KN_0002_0015_0000@end-highlight@\" remote-address=\"57.66.70.251\" local-address=\"192.168.253.236\" gateway-name=\"VPN-to-KN\" group-name=\"KN\" tunnel-id=\"2\" interface-name=\"\" internal-ip=\"Not-Available\" name=\"46.24.17.9\" peer-name=\"57.66.70.251\" client-name=\"Not-Applicable\" vrrp-group-id=\"0\" traffic-selector-name=\"\" traffic-selector-cfg-local-id=\"ipv4_subnet(any:0,[0..7\\]=192.168.10.0/24)\" traffic-selector-cfg-remote-id=\"ipv4(any:0,[0..3\\]=10.29.67.108)\"] VPN @start-highlight@INSTANCE@end-highlight@-@start-highlight@KN_0002_0015_0000@end-highlight@ from 57.66.70.251 is up. Local-ip: 192.168.253.236, gateway name: VPN-to-KN, vpn name: KN, tunnel-id: 2, local tunnel-if: , remote tunnel-ip: Not-Available, Local IKE-ID: 46.24.17.9, Remote IKE-ID: 57.66.70.251, XAUTH username: Not-Applicable, VR id: 0, Traffic-selector: , Traffic-selector local ID: ipv4_subnet(any:0,[0..7]=192.168.10.0/24), Traffic-selector remote ID: ipv4(any:0,[0..3]=10.29.67.108)"
]
},
"sort": [
1485856212927,
1485856212927
]
}

how can I solve that?

best regards

Re: Incorrect severity & facility

Posted: Tue Jan 31, 2017 10:53 am
by mcapra
Basically, Logstash has an internal set of rules it uses the parse syslog messages (on the back-end, the %{SYSLOGBASE} grok pattern). Your Juniper's log format being sent to Logstash don't appear to conform to this format starting with the timestamp 2017-01-31T08:35:00.690+01:00. That's an ISO8601 timestamp, which doesn't cooperate well with Logstash's default syslog handling.

You'll likely need to give the Juniper device it's own input rule, probably using tcp rather than syslog. Then you'll need to write a custom filter rule to deal with the ISO8601 timestamp and other parts of the message you deem relevant. Here's a quick and dirty input/filter set I drew up:

Input:

Code: Select all

tcp {
    type => 'juniper_vpn'
    port => 2090
}
udp {
    type => 'juniper_vpn'
    port => 2090
}
Filter (rename the fields appropriately):

Code: Select all

if [type] == 'juniper_vpn' {
	grok {
		match => [ 'message', '<%{DATA:pri}>%{INT:field_1} %{TIMESTAMP_ISO8601} %{DATA:device} %{DATA:field_2} %{DATA:field_3} %{GREEDYDATA:extra_message}']
	}
}

Re: Incorrect severity & facility

Posted: Thu Feb 09, 2017 5:22 am
by rjjimenez
Hello.

I have applied the following filter:

Code: Select all

if [type] == 'juniper_vpn' {
   grok {
      match => [ 'message', '<%{DATA:pri}>%{INT:field_1} %{TIMESTAMP_ISO8601} %{DATA:device} %{DATA:kmd} %{DATA:msg_id} %{DATA:msg_type} %{GREEDYDATA:extra_message}']
   }
}
But if the severity & facility fields are incorrectly painted

I attach the tcpdump result:

Code: Select all


tcpdump -nnvvS host 192.168.253.236

11:15:18.089113 IP (tos 0x0, ttl 64, id 44193, offset 0, flags [none], proto UDP (17), length 664)
    192.168.253.236.514 > 192.168.253.34.2090: [udp sum ok] SYSLOG, length: 636
        Facility daemon (3), Severity info (6)
        Msg: 1 2017-02-09T11:13:41.010+01:00 SRX-240 kmd 1514 KMD_PM_SA_ESTABLISHED [[email protected] local-address="192.168.253.236" remote-address="185.20.116.200" local-initiator="ipv4_subnet(any:0,[0..7\]=0.0.0.0/0)" remote-responder="ipv4_subnet(any:0,[0..7\]=0.0.0.0/0)" argument1="outbound" index1="4058195851" index2="0" mode="Tunnel" type="dynamic" traffic-selector-name=""] Local gateway: 192.168.253.236, Remote gateway: 185.20.116.200, Local ID: ipv4_subnet(any:0,[0..7]=0.0.0.0/0), Remote ID: ipv4_subnet(any:0,[0..7]=0.0.0.0/0), Direction: outbound, SPI: 0xf1e3278b, AUX-SPI: 0, Mode: Tunnel, Type: dynamic, Traffic-selector:
        0x0000:  3c33 303e 3120 3230 3137 2d30 322d 3039
        0x0010:  5431 313a 3133 3a34 312e 3031 302b 3031
        0x0020:  3a30 3020 5352 582d 3234 3020 6b6d 6420
        0x0030:  3135 3134 204b 4d44 5f50 4d5f 5341 5f45
        0x0040:  5354 4142 4c49 5348 4544 205b 6a75 6e6f
        0x0050:  7340 3236 3336 2e31 2e31 2e31 2e32 2e33
        0x0060:  3920 6c6f 6361 6c2d 6164 6472 6573 733d
        0x0070:  2231 3932 2e31 3638 2e32 3533 2e32 3336
        0x0080:  2220 7265 6d6f 7465 2d61 6464 7265 7373
        0x0090:  3d22 3138 352e 3230 2e31 3136 2e32 3030
        0x00a0:  2220 6c6f 6361 6c2d 696e 6974 6961 746f
        0x00b0:  723d 2269 7076 345f 7375 626e 6574 2861
        0x00c0:  6e79 3a30 2c5b 302e 2e37 5c5d 3d30 2e30
        0x00d0:  2e30 2e30 2f30 2922 2072 656d 6f74 652d
        0x00e0:  7265 7370 6f6e 6465 723d 2269 7076 345f
        0x00f0:  7375 626e 6574 2861 6e79 3a30 2c5b 302e
        0x0100:  2e37 5c5d 3d30 2e30 2e30 2e30 2f30 2922
        0x0110:  2061 7267 756d 656e 7431 3d22 6f75 7462
        0x0120:  6f75 6e64 2220 696e 6465 7831 3d22 3430
        0x0130:  3538 3139 3538 3531 2220 696e 6465 7832
        0x0140:  3d22 3022 206d 6f64 653d 2254 756e 6e65
        0x0150:  6c22 2074 7970 653d 2264 796e 616d 6963
        0x0160:  2220 7472 6166 6669 632d 7365 6c65 6374
        0x0170:  6f72 2d6e 616d 653d 2222 5d20 4c6f 6361
        0x0180:  6c20 6761 7465 7761 793a 2031 3932 2e31
        0x0190:  3638 2e32 3533 2e32 3336 2c20 5265 6d6f
        0x01a0:  7465 2067 6174 6577 6179 3a20 3138 352e
        0x01b0:  3230 2e31 3136 2e32 3030 2c20 4c6f 6361
        0x01c0:  6c20 4944 3a20 6970 7634 5f73 7562 6e65
        0x01d0:  7428 616e 793a 302c 5b30 2e2e 375d 3d30
        0x01e0:  2e30 2e30 2e30 2f30 292c 2052 656d 6f74
        0x01f0:  6520 4944 3a20 6970 7634 5f73 7562 6e65
        0x0200:  7428 616e 793a 302c 5b30 2e2e 375d 3d30
        0x0210:  2e30 2e30 2e30 2f30 292c 2044 6972 6563
        0x0220:  7469 6f6e 3a20 6f75 7462 6f75 6e64 2c20
        0x0230:  5350 493a 2030 7866 3165 3332 3738 622c
        0x0240:  2041 5558 2d53 5049 3a20 302c 204d 6f64
        0x0250:  653a 2054 756e 6e65 6c2c 2054 7970 653a
        0x0260:  2064 796e 616d 6963 2c20 5472 6166 6669
        0x0270:  632d 7365 6c65 6374 6f72 3a20
I attach the RAW nagiosLS result:

Code: Select all


{
  "_index": "logstash-2017.02.09",
  "_type": "juniper_vpn",
  "_id": "AVoiWxpU_ps74PUUpfvy",
  "_score": null,
  "_source": {
    "message": "<30>1 2017-02-09T11:09:40.995+01:00 SRX-240 kmd 1514 KMD_PM_SA_ESTABLISHED [[email protected] local-address=\"192.168.253.236\" remote-address=\"185.20.116.200\" local-initiator=\"ipv4_subnet(any:0,[0..7\\]=0.0.0.0/0)\" remote-responder=\"ipv4_subnet(any:0,[0..7\\]=0.0.0.0/0)\" argument1=\"inbound\" index1=\"727543056\" index2=\"0\" mode=\"Tunnel\" type=\"dynamic\" traffic-selector-name=\"\"] Local gateway: 192.168.253.236, Remote gateway: 185.20.116.200, Local ID: ipv4_subnet(any:0,[0..7]=0.0.0.0/0), Remote ID: ipv4_subnet(any:0,[0..7]=0.0.0.0/0), Direction: inbound, SPI: 0x2b5d6d10, AUX-SPI: 0, Mode: Tunnel, Type: dynamic, Traffic-selector: ",
    "@version": "1",
    "@timestamp": "2017-02-09T10:11:19.659Z",
    "type": "juniper_vpn",
    "host": "192.168.253.236",
    "pri": "30",
    "field_1": "1",
    "device": "SRX-240",
    "kmd": "kmd",
    "msg_id": "1514",
    "msg_type": "KMD_PM_SA_ESTABLISHED",
    "extra_message": "[[email protected] local-address=\"192.168.253.236\" remote-address=\"185.20.116.200\" local-initiator=\"ipv4_subnet(any:0,[0..7\\]=0.0.0.0/0)\" remote-responder=\"ipv4_subnet(any:0,[0..7\\]=0.0.0.0/0)\" argument1=\"inbound\" index1=\"727543056\" index2=\"0\" mode=\"Tunnel\" type=\"dynamic\" traffic-selector-name=\"\"] Local gateway: 192.168.253.236, Remote gateway: 185.20.116.200, Local ID: ipv4_subnet(any:0,[0..7]=0.0.0.0/0), Remote ID: ipv4_subnet(any:0,[0..7]=0.0.0.0/0), Direction: inbound, SPI: 0x2b5d6d10, AUX-SPI: 0, Mode: Tunnel, Type: dynamic, Traffic-selector: "
  },
  "sort": [
    1486635079659,
    1486635079659
  ]
}
Can you help me with this topic?

Thanks

Re: Incorrect severity & facility

Posted: Thu Feb 09, 2017 10:40 am
by rkennedy
Are you referring to this part?

Code: Select all

        Facility daemon (3), Severity info (6)
If not, could you clarify?

NLS is simply parsing what it is shown, and in this case it ends up being the following -

Code: Select all

<30>1 2017-02-09T11:09:40.995+01:00 SRX-240 kmd 1514 KMD_PM_SA_ESTABLISHED [[email protected] local-address=\"192.168.253.236\" remote-address=\"185.20.116.200\" local-initiator=\"ipv4_subnet(any:0,[0..7\\]=0.0.0.0/0)\" remote-responder=\"ipv4_subnet(any:0,[0..7\\]=0.0.0.0/0)\" argument1=\"inbound\" index1=\"727543056\" index2=\"0\" mode=\"Tunnel\" type=\"dynamic\" traffic-selector-name=\"\"] Local gateway: 192.168.253.236, Remote gateway: 185.20.116.200, Local ID: ipv4_subnet(any:0,[0..7]=0.0.0.0/0), Remote ID: ipv4_subnet(any:0,[0..7]=0.0.0.0/0), Direction: inbound, SPI: 0x2b5d6d10, AUX-SPI: 0, Mode: Tunnel, Type: dynamic, Traffic-selector:
I don't believe NLS accounts for anything sent in the headers, just the raw message passed which is passed in the tcp / udp payload. Your log message would need to contain the facility / severity level in it somewhere, for NLS to create a variable out of it.

Re: Incorrect severity & facility

Posted: Wed Mar 01, 2017 1:59 pm
by tmcdonald
Just checking in since we have not heard from you in a while. Did @rkennedy's post clear things up?
All times are UTC-05:00
Page 1 of 1

Powered by phpBB® Forum Software © phpBB Limited