DNS cache not automatically updating

This support forum board is for support questions relating to Nagios Network Analyzer, our network traffic and bandwidth analysis solution.
solarmon
Posts: 81
Joined: Fri Sep 13, 2019 3:57 am

DNS cache not automatically updating

Post by solarmon »

Hi,

It seems our Nagios Network Analyzer 2.4.0 setup is having problems with updating its DNS cache.

I have seen the following related posts:

https://support.nagios.com/forum/viewto ... 32&t=59294
https://support.nagios.com/forum/viewto ... 32&t=38274

We had recently enabled "Resolve Hostnames in:" by checking "Reports and Queries" and "Graphs" and setting "Hostname Cache Time:" to 3600

However, I'm finding that some IPs are not be resolved to DNS addresses, even though the DNS resolution is working when testing with:

Code: Select all

php -r 'echo gethostbyaddr("<IP Address>")."\n";'
But in the database query entry still shows the hostname as the IP address:

Code: Select all

SELECT * FROM nagiosna_hostname_cache WHERE ip='<IP Address>';
I have manually run:

Code: Select all

su nna
/usr/bin/php -q /var/www/html/nagiosna/www/index.php cache
To try to force an update. This seems to take a while to run, but it does seem to work.

I do not want to have to manually run this command, so what can I do to get the automatic DNS cache updated?

Thank you.
User avatar
tgriep
Madmin
Posts: 9177
Joined: Thu Oct 30, 2014 9:02 am

Re: DNS cache not automatically updating

Post by tgriep »

First thing to to is to upgrade the NNA server to the latest version to rule that out.
The latest version is 2.4.1.

Can you run the following as root and post the /tmp/info.txt file here?

Code: Select all

echo 'SELECT * FROM nagiosna_hostname_cache;' | mysql -t -unagiosna -pnagiosna nagiosna >/tmp/info.txt
Thanks
Be sure to check out our Knowledgebase for helpful articles and solutions!
solarmon
Posts: 81
Joined: Fri Sep 13, 2019 3:57 am

Re: DNS cache not automatically updating

Post by solarmon »

@tgriep

When I try to 'Check for Updates' it takes a while to respond - but it does come back with:

"You're running the latest version of Network Analyzer."

I believe this might be because it does not have access to the internet, as we only allow access by proxy. How do we configure Nagios Network Analyzer to access the internet via a proxy?

Checking the following page does indeed indicate that there is a version 2.4.1:

https://www.nagios.com/checkforupdates/ ... sion=2.4.0

I won't post the info.txt file here, for security reasons, but what I can tell you is that the 'time' for each entry is always stuck at 1595510742

I assume this is the issue and the entries are not timing out?

I find it quickest just to TRUNCATE the nagiosna_hostname_cache table rather than run the "index.php cache" command.

Thank you.
solarmon
Posts: 81
Joined: Fri Sep 13, 2019 3:57 am

Re: DNS cache not automatically updating

Post by solarmon »

Sorry, that statement about the 'time' value was not correct. Checking the file completely, there are actually some differrent values.

Code: Select all

+------------+
| time       |
+------------+
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510742 |
| 1595510743 |
| 1595510743 |
| 1595510743 |
| 1595510743 |
| 1595510743 |
| 1595510743 |
| 1595510743 |
| 1595510743 |
| 1595510743 |
| 1595510743 |
| 1595510743 |
| 1595510743 |
| 1595510743 |
| 1595510743 |
| 1595510743 |
| 1595510743 |
| 1595510848 |
| 1595579954 |
| 1595579954 |
| 1595579954 |
| 1595579954 |
| 1595579954 |
| 1595579954 |
| 1595580006 |
| 1595580006 |
| 1595580014 |
| 1595580014 |
| 1595580014 |
| 1595580014 |
| 1595580014 |
| 1595580014 |
| 1595580014 |
| 1595580014 |
| 1595580014 |
| 1595580014 |
| 1595580014 |
| 1595580014 |
| 1595580014 |
| 1595580014 |
| 1595580014 |
| 1595580014 |
| 1595580014 |
| 1595580014 |
| 1595580014 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580015 |
| 1595580016 |
| 1595580016 |
| 1595580016 |
| 1595580016 |
| 1595580016 |
| 1595580016 |
| 1595580016 |
| 1595580016 |
| 1595580016 |
| 1595580016 |
| 1595580016 |
| 1595580016 |
| 1595580016 |
| 1595580016 |
| 1595580016 |
| 1595580016 |
| 1595580016 |
| 1595580016 |
+------------+
solarmon
Posts: 81
Joined: Fri Sep 13, 2019 3:57 am

Re: DNS cache not automatically updating

Post by solarmon »

So I realise/think now that the value in the 'time' coloumn is the epoch time when that entry was created. So, it is not a 'timeout' countdown value.

I can check the current epoch time on the Nagios Network Analyzer server using date +%s and I do see it incrementing as expected.
solarmon
Posts: 81
Joined: Fri Sep 13, 2019 3:57 am

Re: DNS cache not automatically updating

Post by solarmon »

Checking the crontab, it looks OK to me - so I expect the DNS cache to be refresh on the hour, every hour since my Hostname Cache Timeout is set to 3600.

Code: Select all

# /etc/cron.d/nagiosna: crontab fragment for nagiosna

* * * * * nna /usr/bin/php -q /var/www/html/nagiosna/www/index.php cmdsubsys > /usr/local/nagiosna/var/cmdsubsys.log 2>&1
0 * * * * nna /usr/bin/php -q /var/www/html/nagiosna/www/index.php cache > /usr/local/nagiosna/var/cache.log 2>&1
However, I have noticed that /usr/local/nagiosna/var/cache.log does not exist, so maybe it is not running, or hasn't run?
solarmon
Posts: 81
Joined: Fri Sep 13, 2019 3:57 am

Re: DNS cache not automatically updating

Post by solarmon »

Here is an example where the current epoch time is more than 3600 from the 'time' value for the entry, but it has not been updated (I had added PTR record entry for the IP ADDRESS).

Code: Select all

# echo "SELECT * FROM nagiosna_hostname_cache WHERE ip='<IP ADDRESS>;" | mysql -t -unagiosna -pnagiosna nagiosna
+----+-------------+-------------+------------+
| id | ip          | hostname    | time       |
+----+-------------+-------------+------------+
| 37 | <IP ADDRESS> | <IP ADDRESS> | 1595581117 |
+----+-------------+-------------+------------+

Code: Select all

# date +%s
1595593266
User avatar
tgriep
Madmin
Posts: 9177
Joined: Thu Oct 30, 2014 9:02 am

Re: DNS cache not automatically updating

Post by tgriep »

There is not a setting in the Web interface for configuring a Proxy server but here are the instructions for upgrading in an offline environment.
https://support.nagios.com/kb/article.php?id=738

I have been trying to recreate the issue but is seems to function correctly.
The empty cache.log is OK, as it only logs errors and every time it runs, it will overwrite the log file anyways.

If you run the following, does it update the entries?

Code: Select all

/usr/bin/php  /var/www/html/nagiosna/www/index.php cache
You could truncate the table so the next time the cron runs, it will update all of the entries.

To do that, run this.

Code: Select all

echo "truncate nagiosna_hostname_cache;" | mysql -t -unagiosna -pnagiosna nagiosna
Be sure to check out our Knowledgebase for helpful articles and solutions!
solarmon
Posts: 81
Joined: Fri Sep 13, 2019 3:57 am

Re: DNS cache not automatically updating

Post by solarmon »

Thanks for the offline upgrade instructions.

I noticed that the /var/log/cron did not have any entries for the 'cache' cron entry, but it did have entries for the 'cmdsubsys' crontab entry.

I swapped the two entries in /etc/cron.d/nagiosna to make the 'cache' one appear first and now it does appear in /var/log/cron.

It did seem to update on the first time cron was run. Then I changed a DNS PTR recorded and waited for the next hourly cron run. But the entry did not update in NNA, even though the DNS entry did update - I checked using "getent hosts <IP Address>" and "resolveip <IP address>". I will give it another hour and check again.

As stated in my previous posts - it does seem to work when I run the php cache command manually, and when I truncate the table.

The issues seems to be that it does not seem to run properly when it is activated by cron. It is strange that I had to swap the entries before it appeared in the cron logs, but this might suggest some issue related to cron. Also, the cron log timestamp seem to be about 6 hours behind, even though the system and hardware clock time is correct. Again, there are a few things about cron that makes me suspect it is related to it.
User avatar
tgriep
Madmin
Posts: 9177
Joined: Thu Oct 30, 2014 9:02 am

Re: DNS cache not automatically updating

Post by tgriep »

Because cron was not running the job, it would not update the entries, that would be the cause for sure.
What OS and release is your server running?
Can you update the cron daemon to the latest version to see if that kicks off all of the jobs?
Be sure to check out our Knowledgebase for helpful articles and solutions!
Locked