Timezone problem

[lesnikov]

We have problem with timezone which gets shown in dashboard with 2 hour delay.
Right now only problems are unix, and network devices (windows with nxlog works fine)
Even localhost (nagios log server its self) has the same problem.
Did I miss any settings regarding timezone?

example from localhost (NLS) with 2 hour delay:
event from dashboard with (timestamp of dashboard shows correct time 2024-08-27T14:18:39.000+02:00)

Code: Select all

{
  "_index": "logstash-2024.08.27",
  "_type": "syslog",
  "_id": "AZGTWEINceUxifmuEGoe",
  "_score": null,
  "_source": {
    "message": "kauditd_printk_skb: 573 callbacks suppressed\n",
    "@version": "1",
    "@timestamp": "2024-08-27T12:18:39.000Z",
    "type": "syslog",
    "host": "127.0.0.1",
    "priority": 4,
    "timestamp": "Aug 27 12:18:39",
    "logsource": "nagiosls",
    "program": "kernel",
    "severity": 4,
    "facility": 0,
    "facility_label": "kernel",
    "severity_label": "Warning"
  },
  "highlight": {
    "message": [
      "kauditd_printk_skb: @start-highlight@573@end-highlight@ @start-highlight@callbacks@end-highlight@ @start-highlight@suppressed@end-highlight@\n"
    ]
  },
  "sort": [
    1724761119000,
    1724761119000
  ]
}

log created in nagios log server /var/log/syslog

Code: Select all

root@nagiosls:/var/log# cat syslog |grep "kauditd_printk_skb: 573 callbacks suppressed"
2024-08-27T12:18:39.877252+02:00 nagiosls kernel: kauditd_printk_skb: 573 callbacks suppressed

system info:

Code: Select all

full=2024R1.1
major=2024
minor=1.1
releasedate=2024-07-16
release=3100
VERSION=3100

Code: Select all

Description:    Ubuntu 24.04 LTS
Linux version 6.8.0-41-generic (buildd@lcy02-amd64-100) (x86_64-linux-gnu-gcc-13 (Ubuntu 13.2.0-23ubuntu4) 13.2.0, GNU ld (GNU Binutils for Ubuntu) 2.42) #41-Ubuntu SMP PREEMPT_DYNAMIC Fri Aug  2 20:41:06 UTC 2024

Code: Select all

root@nagiosls:/var/log# timedatectl
               Local time: Tue 2024-08-27 14:43:55 CEST
           Universal time: Tue 2024-08-27 12:43:55 UTC
                 RTC time: Tue 2024-08-27 12:43:55
                Time zone: Europe/Ljubljana (CEST, +0200)
System clock synchronized: yes
              NTP service: n/a
          RTC in local TZ: no

global settings in gui:

Code: Select all

Cluster Timezone 
(UTC+01:00) Ljubljana

thank you.

[lesnikov]

More testing:
Some devices switches (cisco), freeBSD,.. work in real time

One type of devices unix NAS (was in delay 2 hour like problem in first post)
i changed from RFC 3164 to RFC 5424, now logs are in real time but as expected are not in correct structure.

[lesnikov]

Forgot to mention in first post that this is fresh install.

So trying to figure what the problem is i tried changing system time to UTC with same problem.
Then reverted back to Europe/Ljubljana and everything started to work as it should.

test from localhost (NLS)

Code: Select all

2024-08-28T08:36:16.000+02:00	127.0.0.1	syslog	Failed password for invalid user test from 172.17.5.2 port 53882 ssh2

Code: Select all

{
  "_index": "logstash-2024.08.28",
  "_type": "syslog",
  "_id": "AZGXswOepPiaat_Aon1_",
  "_score": null,
  "_source": {
    "message": "Failed password for invalid user test from 172.17.5.2 port 53882 ssh2\n",
    "@version": "1",
    "@timestamp": "2024-08-28T06:36:16.000Z",
    "type": "syslog",
    "host": "127.0.0.1",
    "priority": 38,
    "timestamp": "Aug 28 08:36:16",
    "logsource": "nagiosls",
    "program": "sshd",
    "pid": "7347",
    "severity": 6,
    "facility": 4,
    "facility_label": "security/authorization",
    "severity_label": "Informational"
  },
  "highlight": {
    "message": [
      "Failed password for invalid user @start-highlight@test@end-highlight@ from 172.17.5.2 port 53882 ssh2\n"
    ]
  },
  "sort": [
    1724826976000,
    1724826976000
  ]
}

Code: Select all

root@nagiosls:~# timedatectl
               Local time: Wed 2024-08-28 08:36:59 CEST
           Universal time: Wed 2024-08-28 06:36:59 UTC
                 RTC time: Wed 2024-08-28 06:36:59
                Time zone: Europe/Ljubljana (CEST, +0200)
System clock synchronized: yes
              NTP service: n/a
          RTC in local TZ: no

[jmichaelson]

We actually have an open issue tracking a time zone change problem on Ubuntu 24 that we're working on that may be the cause of your problem.