Nagios 5.3.0 Upgrade "broke" my traffic collection?

This support forum board is for support questions relating to Nagios XI, our flagship commercial network monitoring solution.
Locked
srhart
Posts: 28
Joined: Thu Feb 18, 2016 5:08 am

Nagios 5.3.0 Upgrade "broke" my traffic collection?

Post by srhart »

Hi

We "upgraded" to 5.3.0 on Oct 6. It looks like that upgrade stopped mrtg from running which in turn prevented traffic stats being collected for my customer. The reason for the problem appears to have trivial:

I tried manually running mrtg using the cron command in /etc/cron.d/mrtg:
[root@nms cron.d]# LANG=C LC_ALL=C /usr/bin/mrtg /etc/mrtg/mrtg.cfg --lock-file /var/lock/mrtg/mrtg_l --confcache-file /var/lib/mrtg/mrtg.ok
2016-10-24 11:23:44: ERROR: Creating templock /var/lock/mrtg/mrtg_l_25614: No such file or directory at /usr/bin/mrtg line 1961.
[root@nms cron.d]# cd /var/lock
[root@nms lock]# cd mrtg
-bash: cd: mrtg: No such file or directory
[root@nms lock]# mkdir mrtg
[root@nms lock]# LANG=C LC_ALL=C /usr/bin/mrtg /etc/mrtg/mrtg.cfg --lock-file /var/lock/mrtg/mrtg_l --confcache-file /var/lib/mrtg/mrtg.ok
== mrtg now runs ok ==

It looks like there is another "upgrade" pending for Nagios XI - obviously I'm not overly keen to perform it
avandemore
Posts: 1597
Joined: Tue Sep 27, 2016 4:57 pm

Re: Nagios 5.3.0 Upgrade "broke" my traffic collection?

Post by avandemore »

Can you tell us more details? What OS, OS version, and Nagios version where you upgrading from? How was the upgrade preformed? Can we get the upgrade.log from the process?

5.3.1 is a bug fix release, not one with new features as was 5.3.0. I'm not aware of any other reports with this behavior, nor was it present on internal testing so we'll need to isolate how the issue occurred in your system.
Previous Nagios employee
srhart
Posts: 28
Joined: Thu Feb 18, 2016 5:08 am

Re: Nagios 5.3.0 Upgrade "broke" my traffic collection?

Post by srhart »

Hi

OS is CentOS Linux release 7.2.1511 (Core)
Cant remember the version we upgraded from - sorry
We used the Nagios GUI upgrade link that came up in the upgrade dialogue
We did experience a SQL error at the end of the upgrade process - NAGIOSQL LOGIN FAILED - we reloaded and all seemed to be working so we had to get on - we were trying to get a customer back working again.

Re the upgrade.log - is there a PM I can send it to - it's huge and I'm not happy posting it on here.

Thanks
avandemore
Posts: 1597
Joined: Tue Sep 27, 2016 4:57 pm

Re: Nagios 5.3.0 Upgrade "broke" my traffic collection?

Post by avandemore »

Sure, you can PM it to me. Please compress it first if overly large.
NAGIOSQL LOGIN FAILED
This was one of the fixes in 5.3.1.
Previous Nagios employee
avandemore
Posts: 1597
Joined: Tue Sep 27, 2016 4:57 pm

Re: Nagios 5.3.0 Upgrade "broke" my traffic collection?

Post by avandemore »

Please try a manual upgrade using this document: How to Manually Update Nagios XI

It directs you to upgrade with a version which has a bug fix for your issue.
Previous Nagios employee
srhart
Posts: 28
Joined: Thu Feb 18, 2016 5:08 am

Re: Nagios 5.3.0 Upgrade "broke" my traffic collection?

Post by srhart »

Due to issues with upgrades etc the customer is VERY reluctant to let me do anything to it for a while. This loss of traffic statistics a few days ago was the last straw.

Worse I just discovered that, after a reload yesterday to fix a different problem (again with Nagios XI), the traffic collection stopped AGAIN at 1pmish yesterday. I just found the fault to be that /var/lock/mrtg/ seemed to be missing again after a reboot of the server!
avandemore
Posts: 1597
Joined: Tue Sep 27, 2016 4:57 pm

Re: Nagios 5.3.0 Upgrade "broke" my traffic collection?

Post by avandemore »

5.3.1 is a bug fix only release. It does not contain new functionality as did 5.3.0. From a technical perspective, the only reason not to upgrade to 5.3.1 would be if 5.3.0 was working properly.
Previous Nagios employee
srhart
Posts: 28
Joined: Thu Feb 18, 2016 5:08 am

Re: Nagios 5.3.0 Upgrade "broke" my traffic collection?

Post by srhart »

Just been going through the upgrade to 5.3.1. Still got the "NAGIOSQL LOGIN FAILED!" in the upgrade window (so that wasn't fixed) and a dialogue saying that the server is still upgrading. Been waiting for 15 minutes or so.

Do I reboot now or what?

So now I don't know what state my server is in. Can I use it like this or is it likely to fail in a few hours/days?

Unimpressed!
srhart
Posts: 28
Joined: Thu Feb 18, 2016 5:08 am

Re: Nagios 5.3.0 Upgrade "broke" my traffic collection?

Post by srhart »

AND I just reloaded the server and the /var/lock/mrtg folder was missing again which kills the traffic collection AGAIN so after waiting until Sunday when all my server issues will be 'fixed in 5.3.1' in fact NONE of my server problems have been fixed at all. A complete waste of time.

Lucky I know how to fix the mrtg problem myself isn't it.
tmcdonald
Posts: 9117
Joined: Mon Sep 23, 2013 8:40 am

Re: Nagios 5.3.0 Upgrade "broke" my traffic collection?

Post by tmcdonald »

I've just replied to your email ticket, with reference to the issues discussed in this thread. Please see my answer there.
Former Nagios employee
Locked