Home » Categories » Products » Nagios Log Server » Documentation » Architecture

Nagios Log Server - Jobs Subsystem Architecture

Jobs Subsystem Overview

The jobs subsystem of Nagios Log Server runs on every Log Server instance, and is responsible for running jobs that are scheduled to run.  Jobs can be scheduled to run on a specific instance (local jobs) or they can be run on any single instance (global jobs). Additionally, a job can be schedule to run just once, or it can be scheduled to run on a given frequency, e.g. daily, hourly, etc.

 

Example Jobs

  • Local
    • apply_config - Job is scheduled for EACH instance (via instance jobs) to create config snapshot, write logstash configs, restart logstash

    • change_timezone - Changes the timezone on the local instance

    • create_snapshot - creates a config snapshot

    • delete_snapshot - deletes a config snapshot

    • restore_snapshot - restores a config snapshot

    • stop_service - stops a specific service on the local instance

    • start_service - starts a specific service on the local instance

    • restart_service - restarts specific service on the local instance

  • Global
    • run_alerts - Runs every 20 seconds to send alerts

    • backup_maintenance- Runs every day to perform index maintenance and backups

    • cleanup - deletes old completed tasks from the jobs queue more than 1 day old

 

Many global jobs are able to be run from the Admin > System > Command Subsystem page:

 

 

 

Architecture Components And Execution Flow

The jobs subsystem starts every minute via a cron located at /etc/cron.d/nagioslogserver and runs as the nagios user:

 

This cron executes a loop that runs every 5 seconds to perform the following actions:

  1. Query the elasticsearch index to get a list of local jobs scheduled to be executed this instance.

    1. Execute function in command field

    2. Update the Audit Report with results _type = JOBS

  2. Query the elasticsearch index to get a list of global jobs that need to be executed.

    1. Execute function in command field

    2. Update the Audit Report with results _type = JOBS

 NOTE: Global jobs are jobs that ANY instance may process, they are NOT executed by all instances.

 

 

 

The general flow of execution of the jobs subsystem works as follows:

  1. The jobs.php controller runs as a background process and executes the commands in the process_jobs() method.  The jobs.php script is located at /var/www/html/nagioslogserver/application/controllers/jobs.php and runs under cron every minute.  The cron job is defined in /etc/cron.d/nagioslogserver

  2. The jobs.php script executes the functions listed in the process_jobs() method, and the functions will be located in the cmdsubsys_helper.php located at /var/www/html/nagioslogserver/application/helpers/cmdsubsys_helper.php

  3. The poller cron saves output of the run in /var/www/html/nagioslogserver/var/jobs.log

 

 

Troubleshooting Problems

Some potential problems with the jobs subsystem, as well as troubleshooting information are listed below:

Problem: Daily Backups are not being processes or alerts are not being run on designated interval

Potential Causes:

  • The jobs scripts may not be running.  Run the following from the command line to see if the script is running:

    • ps axuw | grep jobs
  • There may be a problem with the cron job.  Check the cron file /etc/cron.d/nagioslogserver to ensure the job is not commented out. Execute the following from the command line to look for possible errors:

    • tail /var/log/cron
  • Check the /usr/local/nagioslogserver/var/jobs.log log file for errors

  • The nagios user account could be expired - you can check this with the following from the command line:

    • chage -l nagios

 

 

Final Thoughts

For any support related questions please visit the Nagios Support Forums at:

http://support.nagios.com/forum/

2 (1)
Article Rating (1 Votes)
Rate this article
  • Icon PDFExport to PDF
  • Icon MS-WordExport to MS Word
Attachments Attachments
There are no attachments for this article.
Related Articles RSS Feed
Nagios Log Server - Poller Subsystem Architecture
Viewed 1258 times since Fri, Feb 6, 2015
Nagios Log Server - API Examples
Viewed 2727 times since Tue, Jun 13, 2017
Nagios Log Server - Full Architecture Overview
Viewed 3598 times since Thu, Jun 4, 2015
Nagios Log Server - Log Monitoring and Log Management with Nagios - NWC14
Viewed 1116 times since Thu, Feb 4, 2016
Nagios Log Server - Configuration Overview
Viewed 1698 times since Sat, Feb 7, 2015
Nagios Log Server - Data Backup and Archiving
Viewed 1455 times since Sat, Feb 7, 2015
Nagios Log Server - Architecture and Overview - NWC15
Viewed 1304 times since Mon, Feb 1, 2016
Nagios Log Server - Real Life Experience of Nagios Log Server - NWC15
Viewed 1324 times since Mon, Feb 8, 2016
Nagios Log Server - Overview And Terminology
Viewed 1333 times since Wed, Jan 28, 2015