Sorting Logs by Event Time in NXLog: Understanding the Challenge

[deny7gomez]

When importing logs using NXLog, converting date fields into a usable timestamp is crucial for proper sorting and analysis. In your current setup, you’re using a filter to match your log’s mp-date field and store it in EventTime. The conversion appears correct because EventTime accurately reflects the original log time in ISO 8601 format, preserving the time zone. mywisely
However, ordering issues arise because sorting by EventTime in some systems may not interpret the field as a true datetime type. Even though the format looks correct, it could still be treated as a string in certain contexts, which would lead to unexpected orderings, as seen in your example. Lexicographic sorting of ISO timestamps can produce strange results if milliseconds or time zones are inconsistent.
To fix this, ensure that the field is stored and recognized as a proper date object. In systems like Elasticsearch or Logstash, this requires explicitly mapping the field as a date type. Additionally, always check that all log entries are consistently converted to UTC to avoid time zone discrepancies. Once properly typed, sorting by EventTime should yield a correct chronological order, making analysis more reliable and intuitive.

[jmichaelson]

What version of Nagios Log Server are you looking at for this assessment? 2024R2.x and newer uses OpenSearch for its data storage and the time stamp is a much more rigidly defined field type. It may be more of an issue in 2024R1.x and older. Either way let me know which.

[Angelina1]

The issue arises because even though EventTime appears correctly formatted, some systems may treat it as a string rather than a true datetime. To ensure proper chronological sorting:

Convert and normalize timestamps – Make sure all log entries are consistently converted to UTC.

Use proper datetime types – In systems like Elasticsearch or Logstash, explicitly map EventTime as a date field rather than leaving it as a string.

Verify consistency – Check that all entries include the same precision (milliseconds) and time zone formatting.

Once the field is properly typed and normalized, sorting by EventTime should reflect the correct chronological order.

[surgerygrilled]

When sorting logs by EventTime in NXLog, make sure the field is treated as a datetime rather than a string. Use a conversion function or parse_date() to store timestamps in proper ISO 8601 format and ensure all entries are normalized to UTC. Lexicographic (string) sorting can cause misordering, especially with different time zones or millisecond values. Once EventTime is recognized as a true date field, sorting will correctly follow chronological order.