Nagios Log Server Administration Guide

This support forum board is for support questions relating to Nagios Log Server, our solution for managing and monitoring critical log data.
Locked
srg1970nj
Posts: 23
Joined: Wed Apr 08, 2015 12:10 pm

Nagios Log Server Administration Guide

Post by srg1970nj »

Is there a Nagios Log Server Administration PDF/guide available? The only useful document I found so far is the following:

http://assets.nagios.com/downloads/nagi ... Server.pdf

Unfortunately that only tells me how to add new files. I need to know the following:

How to view what files are being monitored on a given server? (Going to Home, Top Sources and Types I can see the servers being monitored but you cannot see what files are being monitored.)

How do we apply filters to the log files being monitored to ensure required events are generated? (I can see all the syslog data coming in from the servers I added so far. In the test file I am monitoring I can also see all the information entered in the log file. I just do not see where filters can be applied to actually generate events.)

Where are the filters actually created? I have hundreds if not thousands of messages I need to search for in our production log files. I need to know where I can create these filters so they can be applied to the required log files.
jolson
Attack Rabbit
Posts: 2560
Joined: Thu Feb 12, 2015 12:40 pm

Re: Nagios Log Server Administration Guide

Post by jolson »

The administration guide can be found here: http://assets.nagios.com/downloads/nagi ... nistrator/
How to view what files are being monitored on a given server?
To be clear, you would like to see what files are being monitored on your remote machines - the ones sending logs to NLS? Unfortunately there's not an easy way to do this.
How do we apply filters to the log files being monitored to ensure required events are generated?
I do not think I understand what you mean by 'filters' here. Would you like to generate alerts based on events that occur? If so, that can be done on the 'Alerting' page of NLS.

If you mean getting more granular with your logs, you can set up a simple filter by clicking the little hourglass next to a given field:
2015-04-23 09_14_01-Dashboard • Nagios Log Server.png
Where are the filters actually created? I have hundreds if not thousands of messages I need to search for in our production log files. I need to know where I can create these filters so they can be applied to the required log files.
I need to know more about what you mean by 'filters' to answer this question properly.

Thanks!
You do not have the required permissions to view the files attached to this post.
Twits Blog
Show me a man who lives alone and has a perpetually clean kitchen, and 8 times out of 9 I'll show you a man with detestable spiritual qualities.
srg1970nj
Posts: 23
Joined: Wed Apr 08, 2015 12:10 pm

Re: Nagios Log Server Administration Guide

Post by srg1970nj »

Jolson thank you very much for getting back to me. I will give you a clear example of exactly what I am referring too. We are currently using ITRS to monitor a log file. With ITRS we have the ability to alert on messages/strings found in a file and to also ignore messages. As an example we scan for |ERR( but there are some messages that contain |ERR( that we also need to ignore. In ITRS this is called an ignore key.

So in one example we are monitoring the following log file:

/var/log/nyfix/abim1/abim1.log

For that log file we are scanning for the following and generating alerts. (Where severity equals Ignore we do not generate an alert):

Key: BLPSOV42
Severity: Ignore

Key: SYNC REQUEST FAILED: [0] FAILED TO COMPLETE REQUEST AFTER RETRYING
Severity: Ignore

Key: MARKETPLACE ROUTER REJECTING MESSAGE CONTAINING TARGETSUBID .* CSCAMIN42
Severity: Ignore

Key: ERROR: PROCESSING SESSION_CONNECT_FAILURE
Severity: Ignore
Active Time: 1700 Friday through 0300 Sat State: NOT ACTIVE

Key: |ERR(0,0) DISCONNECT FAILED FOR SESSION_ID
Severity: Ignore

Key: |CRIT(0,0) DK
Severity: Ignore

Key: ERR(0,0) TIMESTEN DATA STORE SRECOVERY70_3 APPEARS TO HAVE PROBLEMS
Severity: Ignore

Key: ERR(0,0) THERE APPEARS TO BE REPLICATION PROBLEMS WITH THE REMOTE SUBSCRIBER
Severity: Ignore

Key: ERR(0,0) NO REPLICATION ACTIVITY WITH SUBSCRIBER IN
Severity: Ignore

Key: FAILED TO EXECUTE ADMIN COMMAND NSB.STATE.CONNECT-BUS-ID FOR BUS_ID: MPPRD-30822
Severity: Ignore

Key: FAILED TO EXECUTE ADMIN COMMAND NSB.STATE.SOD FOR BUS_ID: MPPRD-23915, SESSION_ID: TACCRD_SPLIT
Severity: Ignore

Key: ERR(0,0) SEQUENCERESERVEREQUEST FAILED WITH ERROR: CREATE_NEW FLAG IS SPECIFIED AND SERVER FAILED TO CREATE THE SEQUENCE
Severity: Ignore

Key: ERR(0,0) SEQUENCERESERVEREQUEST FAILED WITH ERROR: SPECIFIED SEQUENCE DOES NOT EXIST
Severity: Ignore

Key: ERR(0,0) SEQUENCE RESERVE REQUEST FAILED WITH ERROR CODE: -7
Severity: Ignore

Key: ERR(0,0) SEQUENCE RESERVE REQUEST FAILED WITH ERROR CODE: -6
Severity: Ignore

Key: ERR(0,0) UNABLE TO RESERVE SPECIFIED SEQUENCE: FSRSECID
Severity: Ignore

Key: ERR(0,0) UNABLE TO RESERVE SPECIFIED SEQUENCE: FSRRPTSEQ
Severity: Ignore

Key: ERR(0,0) UNABLE TO RESERVE SPECIFIED SEQUENCE: FSREXECID
Severity: Ignore

Key: DATABASE REQUEST FAILED, RETRYING: ORA-03135: CONNECTION LOST CONTACT PROCESS ID:
Severity: Ignore

Key: CONFIGREQUEST: ERROR EXECUTING SQL: 3135: ORA-03135: CONNECTION LOST CONTACT PROCESS ID
Severity: Ignore

Key: ERR(0,0) NO SERVER FOUND WITH DSN=
Severity: Ignore

Key: |ERR(


Key: |CRIT(


Key: ERROR: session info for * is unavailable (regex)


Key: *** GLIBC


Key: |CRIT|


Key: CAN NOT EXECUTE COMMAND


Key: CAN NOT SATISFY SESSION CONFIGURATION REQUEST


Key: CONSTRUCTING BUSAPI WITH BUSID


Key: CRITICAL:


Key: ENDTRANSACTION FAILED


Key: ERROR: FAILED TO CONNECT TO SERVICE BUS


Key: ERROR: FAILED TO GET BUSID FOR SESSION:


Key: ERROR: FAILED TO RETRIEVE SESSION LIST


Key: FAILED ENDTRANSACTION FOR


Key: FAILED SENDER FIX NAME LOOKUP FOR


Key: FAILED TO EXECUTE ADMIN COMMAND


Key: FAILED TO GET CONFIG MANAGER FOR BUS_ID:


Key: FAILED TO GET CONFIGURATION FOR ALL SESSIONS


Key: FAILED TO GET RECOVERY SERVICE BUS ID FOR SESSION


Key: FAILED TO INITIALIZE WITH EXCEPTION:


Key: FAILED TO OBTAIN NSB CONNECTION OBJECT FOR SESSION_ID:


Key: FAILED TO OBTAIN RECOVERY SERVICE ID FOR SESSION_ID:


Key: FAILED TO RETRIEVE CONFIGURATION FOR REMOTE_ID


Key: FAILED TO RETRIEVE CONFIGURATION FOR SESSION_ID


Key: FAILED TO SEND CLOSESTOREREQUEST


Key: FAILED TO START NSB API WITH EXCEPTION:


Key: FAILED TO TRUNCATE STORE


Key: NO FIX SESSION NAME ASSOCIATED WITH


Key: NON-RETRIABLE FAILURE TO PROCESS


Key: NULLPOINTEREXCEPTION


Key: RECEIVED ACK TIMEOUT FOR CLOSESTOREREQUEST REQUEST FOR BUS_ID:


Key: RECEIVED NULL RESPONSE FOR CLOSESTOREREQUEST REQUEST FOR BUS_ID:


Key: RECEIVED RESPONSE TIMEOUT FOR CLOSESTOREREQUEST REQUEST FOR BUS_ID:


Key: RECEIVED SERVICE ERROR FOR CLOSESTOREREQUEST REQUEST FOR


Key: UNEXPECTED ADMIN COMMAND


Key: ignoring (regex)


Key: MARKETPLACE ROUTER REJECTING MESSAGE CONTAINING TARGETSUBID


Key: ERROR: PROCESSING SESSION_CONNECT_FAILURE


I was told by Nagios support we could replicate this configuration with Nagios Log Server. I need to know where I can create this filter/configuration to replicate the ITRS monitoring currently implemented.
jolson
Attack Rabbit
Posts: 2560
Joined: Thu Feb 12, 2015 12:40 pm

Re: Nagios Log Server Administration Guide

Post by jolson »

Nagios Log Server handles this functionality a little bit differently - but what you're asking for is 100% doable. Let me give you an example.

First, we'll navigate to 'Dashboard' and look through my log files until we find one that I want to alert on. For the sake of this example, let's say it's this one:
2015-04-23 10_00_17-Dashboard • Nagios Log Server.png
Now, we need to identify characteristics of this log that make it unique - some 'filters'. Filters can be easily applied by clicking the little 'hourglass' under 'Action.'

In this case, I want to alert if facility = 10, and facility label = security/authorization, and program = sshd, and message contains = 'Failed password for *'. I will click the following:
2015-04-23 10_04_35-Dashboard • Nagios Log Server.png
This will generate filters up top:
2015-04-23 10_07_21-Dashboard • Nagios Log Server.png
Now, I can add a message string, and see our final results:
2015-04-23 10_08_18-Dashboard • Nagios Log Server.png
Perfect, now all failed logins will show up in this query. Please save this as a query and make an alert based on it:
2015-04-23 10_10_11-Dashboard • Nagios Log Server.png
2015-04-23 10_10_47-Alerting • Nagios Log Server.png
Hopefully that all made sense. Please let me know if you have further questions.
You do not have the required permissions to view the files attached to this post.
Twits Blog
Show me a man who lives alone and has a perpetually clean kitchen, and 8 times out of 9 I'll show you a man with detestable spiritual qualities.
Locked