Page 1 of 2
DNS cache not automatically updating
Posted: Thu Jul 23, 2020 2:29 am
by solarmon
Hi,
It seems our Nagios Network Analyzer 2.4.0 setup is having problems with updating its DNS cache.
I have seen the following related posts:
https://support.nagios.com/forum/viewto ... 32&t=59294
https://support.nagios.com/forum/viewto ... 32&t=38274
We had recently enabled "Resolve Hostnames in:" by checking "Reports and Queries" and "Graphs" and setting "Hostname Cache Time:" to 3600
However, I'm finding that some IPs are not be resolved to DNS addresses, even though the DNS resolution is working when testing with:
Code: Select all
php -r 'echo gethostbyaddr("<IP Address>")."\n";'
But in the database query entry still shows the hostname as the IP address:
Code: Select all
SELECT * FROM nagiosna_hostname_cache WHERE ip='<IP Address>';
I have manually run:
Code: Select all
su nna
/usr/bin/php -q /var/www/html/nagiosna/www/index.php cache
To try to force an update. This seems to take a while to run, but it does seem to work.
I do not want to have to manually run this command, so what can I do to get the automatic DNS cache updated?
Thank you.
Re: DNS cache not automatically updating
Posted: Thu Jul 23, 2020 5:01 pm
by tgriep
First thing to to is to upgrade the NNA server to the latest version to rule that out.
The latest version is 2.4.1.
Can you run the following as root and post the /tmp/info.txt file here?
Code: Select all
echo 'SELECT * FROM nagiosna_hostname_cache;' | mysql -t -unagiosna -pnagiosna nagiosna >/tmp/info.txt
Thanks
Re: DNS cache not automatically updating
Posted: Fri Jul 24, 2020 3:33 am
by solarmon
@tgriep
When I try to 'Check for Updates' it takes a while to respond - but it does come back with:
"You're running the latest version of Network Analyzer."
I believe this might be because it does not have access to the internet, as we only allow access by proxy. How do we configure Nagios Network Analyzer to access the internet via a proxy?
Checking the following page does indeed indicate that there is a version 2.4.1:
https://www.nagios.com/checkforupdates/ ... sion=2.4.0
I won't post the info.txt file here, for security reasons, but what I can tell you is that the 'time' for each entry is always stuck at
1595510742
I assume this is the issue and the entries are not timing out?
I find it quickest just to TRUNCATE the nagiosna_hostname_cache table rather than run the "index.php cache" command.
Thank you.
Re: DNS cache not automatically updating
Posted: Fri Jul 24, 2020 3:39 am
by solarmon
Sorry, that statement about the 'time' value was not correct. Checking the file completely, there are actually some differrent values.
Code: Select all
+------------+
| time |
+------------+
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510743 |
| 1595510743 |
| 1595510743 |
| 1595510743 |
| 1595510743 |
| 1595510743 |
| 1595510743 |
| 1595510743 |
| 1595510743 |
| 1595510743 |
| 1595510743 |
| 1595510743 |
| 1595510743 |
| 1595510743 |
| 1595510743 |
| 1595510743 |
| 1595510848 |
| 1595579954 |
| 1595579954 |
| 1595579954 |
| 1595579954 |
| 1595579954 |
| 1595579954 |
| 1595580006 |
| 1595580006 |
| 1595580014 |
| 1595580014 |
| 1595580014 |
| 1595580014 |
| 1595580014 |
| 1595580014 |
| 1595580014 |
| 1595580014 |
| 1595580014 |
| 1595580014 |
| 1595580014 |
| 1595580014 |
| 1595580014 |
| 1595580014 |
| 1595580014 |
| 1595580014 |
| 1595580014 |
| 1595580014 |
| 1595580014 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580016 |
| 1595580016 |
| 1595580016 |
| 1595580016 |
| 1595580016 |
| 1595580016 |
| 1595580016 |
| 1595580016 |
| 1595580016 |
| 1595580016 |
| 1595580016 |
| 1595580016 |
| 1595580016 |
| 1595580016 |
| 1595580016 |
| 1595580016 |
| 1595580016 |
| 1595580016 |
+------------+
Re: DNS cache not automatically updating
Posted: Fri Jul 24, 2020 4:59 am
by solarmon
So I realise/think now that the value in the 'time' coloumn is the epoch time when that entry was created. So, it is not a 'timeout' countdown value.
I can check the current epoch time on the Nagios Network Analyzer server using date +%s and I do see it incrementing as expected.
Re: DNS cache not automatically updating
Posted: Fri Jul 24, 2020 5:39 am
by solarmon
Checking the crontab, it looks OK to me - so I expect the DNS cache to be refresh on the hour, every hour since my Hostname Cache Timeout is set to 3600.
Code: Select all
# /etc/cron.d/nagiosna: crontab fragment for nagiosna
* * * * * nna /usr/bin/php -q /var/www/html/nagiosna/www/index.php cmdsubsys > /usr/local/nagiosna/var/cmdsubsys.log 2>&1
0 * * * * nna /usr/bin/php -q /var/www/html/nagiosna/www/index.php cache > /usr/local/nagiosna/var/cache.log 2>&1
However, I have noticed that
/usr/local/nagiosna/var/cache.log does not exist, so maybe it is not running, or hasn't run?
Re: DNS cache not automatically updating
Posted: Fri Jul 24, 2020 7:21 am
by solarmon
Here is an example where the current epoch time is more than 3600 from the 'time' value for the entry, but it has not been updated (I had added PTR record entry for the IP ADDRESS).
Code: Select all
# echo "SELECT * FROM nagiosna_hostname_cache WHERE ip='<IP ADDRESS>;" | mysql -t -unagiosna -pnagiosna nagiosna
+----+-------------+-------------+------------+
| id | ip | hostname | time |
+----+-------------+-------------+------------+
| 37 | <IP ADDRESS> | <IP ADDRESS> | 1595581117 |
+----+-------------+-------------+------------+
Re: DNS cache not automatically updating
Posted: Fri Jul 24, 2020 11:00 am
by tgriep
There is not a setting in the Web interface for configuring a Proxy server but here are the instructions for upgrading in an offline environment.
https://support.nagios.com/kb/article.php?id=738
I have been trying to recreate the issue but is seems to function correctly.
The empty cache.log is OK, as it only logs errors and every time it runs, it will overwrite the log file anyways.
If you run the following, does it update the entries?
Code: Select all
/usr/bin/php /var/www/html/nagiosna/www/index.php cache
You could truncate the table so the next time the cron runs, it will update all of the entries.
To do that, run this.
Code: Select all
echo "truncate nagiosna_hostname_cache;" | mysql -t -unagiosna -pnagiosna nagiosna
Re: DNS cache not automatically updating
Posted: Fri Jul 24, 2020 11:21 am
by solarmon
Thanks for the offline upgrade instructions.
I noticed that the /var/log/cron did not have any entries for the 'cache' cron entry, but it did have entries for the 'cmdsubsys' crontab entry.
I swapped the two entries in /etc/cron.d/nagiosna to make the 'cache' one appear first and now it does appear in /var/log/cron.
It did seem to update on the first time cron was run. Then I changed a DNS PTR recorded and waited for the next hourly cron run. But the entry did not update in NNA, even though the DNS entry did update - I checked using "getent hosts <IP Address>" and "resolveip <IP address>". I will give it another hour and check again.
As stated in my previous posts - it does seem to work when I run the php cache command manually, and when I truncate the table.
The issues seems to be that it does not seem to run properly when it is activated by cron. It is strange that I had to swap the entries before it appeared in the cron logs, but this might suggest some issue related to cron. Also, the cron log timestamp seem to be about 6 hours behind, even though the system and hardware clock time is correct. Again, there are a few things about cron that makes me suspect it is related to it.
Re: DNS cache not automatically updating
Posted: Fri Jul 24, 2020 11:40 am
by tgriep
Because cron was not running the job, it would not update the entries, that would be the cause for sure.
What OS and release is your server running?
Can you update the cron daemon to the latest version to see if that kicks off all of the jobs?