Import archived logs

This support forum board is for support questions relating to Nagios Log Server, our solution for managing and monitoring critical log data.
User avatar
cdienger
Support Tech
Posts: 5045
Joined: Tue Feb 07, 2017 11:26 am

Re: Import archived logs

Post by cdienger »

Yes, filters would be needed. The following Input:

Code: Select all

tcp {
    type => 'import_raw_multiline'
    tags => 'import_raw'
    port => 2056
codec => multiline {
      pattern => "^CompanyName"
      negate => "true"
      what => "previous"
}
}
and Filter:

Code: Select all

if [type] == 'import_raw_multiline' {

mutate {
add_field => { "YYYY-MM-DD" => ""}
}

grok {
      match => [ "message", "(?<YYYY-MM-DD>20[1-2][0-9]-[0-1][0-9]-[0-3][0-9])" ]
}
}
With panel settings:
panel.png
Creates a bar graph like:
panel2.png
Does this help accomplish what you're looking for?
You do not have the required permissions to view the files attached to this post.
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
lukedevon
Posts: 143
Joined: Sat Mar 24, 2018 9:15 am

Re: Import archived logs

Post by lukedevon »

Hi,

Thank you so much for the help. it was really a great help to understand how to integrate that type of text files with Nagios ls.

Under the fields list, I can see YYYY-MM-DD displays as a filed. So we can easily use it as a pointer to draw other required graphs. Is there a way to add all other lines as fields to the "fields list" and are we able to add only selected lines as fields?

in this example;

CompanyName: XYZ Pvt Ltd
Date & Time: 2018-09-28 00:03:47.312
Some value: xxx
Some text: abcd
So on, it has multiple lines
END:

I want to add "Some value" and "some text" as fields in the fields list.

Then it is so flexible to draw graphs and we can have multiple types of data to display. Can you please help me with this too? Sorry to trouble you so much, but I don't have any other option :)

Regards
Luke.
User avatar
cdienger
Support Tech
Posts: 5045
Joined: Tue Feb 07, 2017 11:26 am

Re: Import archived logs

Post by cdienger »

There can only be a single field for labels so you'll need to create a new graph. I would also encourage using dashboards to group related graphs and tables into a single view. You can also use and save filters to create dashboards so relevant data shows up right away. https://assets.nagios.com/downloads/nag ... Server.pdf covers using dashboards.
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
lukedevon
Posts: 143
Joined: Sat Mar 24, 2018 9:15 am

Re: Import archived logs

Post by lukedevon »

Hi

May I how can I remove the light green color first column from the graph?

In the graph you have drawn, panel2.png. First column - Light green color; That may be the total of all the occurrences.

Actually that column quite misleading when presenting the data against the DATE. The bar chart supposed to display the count of each DATE.

Thank you
Luke
User avatar
cdienger
Support Tech
Posts: 5045
Joined: Tue Feb 07, 2017 11:26 am

Re: Import archived logs

Post by cdienger »

Looks like we can improve on something. Here is how you can remove the total:

1. save the current dashboard with the new panel
2. edit graph and add something to the Exclude Term(s) field
3. save changes to the dashboard
4. edit the graph and clear the Exclude Term(s) field
5. save changes
6. click the home link and then click back to the dashboard to refresh the screen
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
lukedevon
Posts: 143
Joined: Sat Mar 24, 2018 9:15 am

Re: Import archived logs

Post by lukedevon »

Hi

Thank you, It works.

1. Can't we import these graphs to PDF?
2. How can I generate line charts using the same data? As I see there is only PIE, BAR and TABLE available.
3. How can I generate all other types of graphs?

I am struggling with some filtering. In my block of data, I have a number field.

For example:

Mobile No : +44112233445566

I have thousands of lines like this with different Mobile Nos. in the imported block of data. Now I need to find, is there any repetition of these numbers, I need to get a count of it, and I want to get these type duplications occurred in which date.

Regards
Luke
User avatar
cdienger
Support Tech
Posts: 5045
Joined: Tue Feb 07, 2017 11:26 am

Re: Import archived logs

Post by cdienger »

1. No.
2. Not an option.
3. Other methods of displaying data use different panels. Most of them are covered in https://www.elastic.co/guide/en/kibana/3.0/panels.html.
4. You could create a filter to extract the numbers and then when you do a search for the number you'll see all events that contain it as well as when they occurred. The filter would look like:

mobile_no:44112233445566
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
Locked