SLA style configuration for nagiosXI?

[ucemike]

We're trying to move away from our current SLA system to something a bit more reliable. We've used nagios as a system monitor application for years now and now we're wondering if we can do the following.

Each pop we have 2 "agents" that monitor/log jitter/latency to all the other pop agents.

What we need to do is have a system that monitors at the pop locations as above and stores/forwards the data to a sql db/host. If the connection to the DB is lost is must keep the data and send when the connection reopens.

I've had some discussion already and the suggestion was that we'd need to have a nagiosXI host on each agent that then passively sends data to a central/main nagiosXI host.

My primary concerns are can we insert the data into a DB (sql,postgres/etc) and will the data on the agents always get to the central/main nagiosXI host.

What tools/addons do I need to look at to get nagios to use a db for records? Our current setup uses flat files (guessing a basic install). Are there any tutorials for how to use passive data to a central server?

Thank you for any suggestions/links you might have.

[ssax]

You could use passive monitoring with NRDS and timestamps to store/forward the records using a cachedir.

You can follow the guide below (starting on slide 31), it shows you how to set up what you are trying to do.

http://www.slideshare.net/nagiosinc/nag ... ux-systems

Basic NRDS info: http://assets.nagios.com/downloads/nagi ... h_NRDS.pdf

Let us know if you have any questions.

[ucemike]

You could use passive monitoring with NRDS and timestamps to store/forward the records using a cachedir.

You can follow the guide below (starting on slide 31), it shows you how to set up what you are trying to do.

http://www.slideshare.net/nagiosinc/nag ... ux-systems

Basic NRDS info: http://assets.nagios.com/downloads/nagi ... h_NRDS.pdf

Let us know if you have any questions.

I actually found the passive monitoring pdf and have been experimenting with it (NRDS/passive). So far seems good though I need to see how it deals with losing connectivity to the central host (which looks like the slide you linked might cover).

Any tips on where to start looking for the "db" hooks to insert data into a DB?

[ssax]

I need to know more on what process you are trying to accomplish, is it?

Code: Select all

1. Connection goes down > Agents caching data > Connection comes up > Agent sends results to DB
                                                                    > Agent sends results to Nagios

OR

2. Connection goes down > Agents caching data > Connection comes up > Agent sends results to Nagios > Nagios sends results to DB

Option 2 is not recommended because you don't want to have that load on the Nagios server, we've seen multiple customers run into issues doing this.

Both options would require a custom script in order to put the data into a remote DB, we do offer custom development if that is something you are interested in, if so you can contact [email protected] to get more information on it.


Am I understanding what you are trying to achieve or are you trying to do something else?

[ucemike]

I need to know more on what process you are trying to accomplish, is it?

Code: Select all

1. Connection goes down > Agents caching data > Connection comes up > Agent sends results to DB
                                                                    > Agent sends results to Nagios

OR

2. Connection goes down > Agents caching data > Connection comes up > Agent sends results to Nagios > Nagios sends results to DB

Option 2 is not recommended because you don't want to have that load on the Nagios server, we've seen multiple customers run into issues doing this.

Both options would require a custom script in order to put the data into a remote DB, we do offer custom development if that is something you are interested in, if so you can contact [email protected] to get more information on it.

Option 1 is acceptable. The data does not need to go through nagios to get to the DB for any reason we might have. As long as the data is present in the DB (most important for the SLA) and nagios for the alerts.

[ssax]

If the connection drops the agent is going to keep trying to send as normal until the connection is established, then it will send it.

You would need to modify your /usr/local/nrdp/clients/send_nrdp.sh to call another script that will:

• 1. Try to establish DB connection and fails gracefully if it can't connect.
  2. If the connection is established, send data into the DB.

You'll want to make sure that you handle the database inserting properly in case send_nrdp.sh can't connect to nagios but can connect to the database, you don't want to keep trying to insert the same results.

[ucemike]

Just for my own edification, what tools would one use to have nagios process the data and insert into DB? I know you mentioned this causes issues if a lot of remote sites had issues connecting and then flood the central server...

[ssax]

Yes and that you're having Nagios process the inserting to the DB which adds more load to the server. It's not something we recommend (or support) because the SQL queries may take a long time or consume a lot of resources and will impact you Nagios server in a bad way which could lead to other problems that may not look like it's from your SQL queries but they are actually responsible for the problem. If I'm not being clear, we really really don't recommend you do it that way.

If you do decide to do that, it's really the same for both in terms of making it work, you will have to write a custom script in your favorite language to connect to your DB and insert the data, how you would want to do that would need to be architected (cron jobs/event handlers/however you wish to do it).