MRTG reports low bandwidth and high pings at times

[TomS]

CentOS
2014R2.6
VM

The issue: MRTG shows very low bandwidth on 192.168.xx.1 Adtran routers (Under 1MB) but intermittently we get very high (500 -1000ms) ping times with severe slow downs at the branch endpoints.
We have Netvanta 1238/1224 PoE and Adtran 3448 Routers with very basic configurations. We have ATT MPLS and Metro-E at 3MB and 10MB respectively at branches in a hub and spoke back to Headquarters 100MB ATT Cisco router

All devices point to the Adtran router (192.168.xx.1) as the gateway which routes all traffic to the ATT routers (192.168.xx.2) (I know it is an unnecessary hop but we do not have access to the ATT routers and may have wanted backup connectivity at some point in the past). Both Routers are connected to the primary switch via single 100MB eth port 0/1 (IOW:, these routers are not directly connected to each other in line)

Some branches have two switches daisy chained on GigE ports. There are 10-40 devices at each branch sending all internet and intranet traffic back to HQ,, Domain Controllers are at each branch.

I have only 1 VLAN set on the switches and all are full 100mb connections (except GigE of Course) Any ideas of where to start looking for possible bandwidth throttling or cap on traffic flow Here is a sample switch config:

!
!
hostname "BRANCHNAME"
enable password md5 encrypted xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
!
ip subnet-zero
ip classless
ip routing
!
event-history on
logging forwarding on
no logging console
logging forwarding priority-level info
logging forwarding receiver-ip 192.168.XX.XX
no logging email
logging email priority-level info
!
username "XXXX" password "PPPP"
!
banner motd ^
Welcome to the Jungle ^
!
banner login ^
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
STOP! This is a private computer system on a private computer network.
ALL access is logged and monitored.
Unauthorized users are not allowed and any attempt to enter the network or
this system without explicit permission will result in civil and criminal
liabilities.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
^
!
!
!
!
!
!
!
!
!
!
!
vlan 1
name "Default"
!
interface eth 0/1
no shutdown
!
interface eth 0/2
no shutdown
!
interface eth 0/3
no shutdown
!
interface eth 0/4
no shutdown
!
interface eth 0/5
no shutdown
!
interface eth 0/6
no shutdown
!
interface eth 0/7
no shutdown
!
interface eth 0/8
no shutdown
!
interface eth 0/9
no shutdown
!
interface eth 0/10
no shutdown
!
interface eth 0/11
no shutdown
!
interface eth 0/12
no shutdown
!
interface eth 0/13
no shutdown
!
interface eth 0/14
no shutdown
!
interface eth 0/15
no shutdown
!
interface eth 0/16
no shutdown
!
interface eth 0/17
no shutdown
!
interface eth 0/18
no shutdown
!
interface eth 0/19
no shutdown
!
interface eth 0/20
no shutdown
!
interface eth 0/21
no shutdown
!
interface eth 0/22
no shutdown
!
interface eth 0/23
no shutdown
!
interface eth 0/24
no shutdown
!
!
interface vlan 1
ip address 192.168.XX.1 255.255.255.0
no shutdown
!
!
interface t1 1/1
description XX to YY T1
tdm-group 1 timeslots 1-24 speed 64
no shutdown
!
interface t1 1/2
line-length 0
shutdown
!
interface bri 1/1
shutdown
!
interface ppp 1
ip address 192.168.205.2 255.255.255.255
no shutdown
cross-connect 1 t1 1/1 1 ppp 1
!
!
!
!
!
ip route 0.0.0.0 0.0.0.0 ppp 1
!
no ip tftp server
no ip http server
ip http secure-server
no ip snmp agent
no ip ftp agent
!
!
!
!
!
!
!
!
line con 0
login
password md5 encrypted xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
line-timeout 3
!
line telnet 0 4
login
password md5 encrypted yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
line-timeout 3
!
!
!
!
end

Thanks for any help,
Tom

[tmcdonald]

This is a question you will need to present to your network team. This forum is for issues related to the Nagios products themselves, and not for general troubleshooting of the things being monitored.

[TomS]

I have asked myself the question and I determined that it appears to be either MRTG or a sampling issue with snmp on ADTRAN routers.

Are you aware of any issue with ADTRAN router snmp or MRTG accuracy? If you can rule those two out then I'll go back to ATT ask them for their graphs again.

Something is not accurate within MRTG; additionally the HOST and PING graphs are reporting distinctly different ping times on the same router or switch. Am I missing something?

[Box293]

Can you please run these commands and send us the output:

Code: Select all

ls -al /etc/mrtg/conf.d/ | wc -l

ls -al /var/lib/mrtg/ | wc -l

additionally the HOST and PING graphs are reporting distinctly different ping times on the same router or switch. Am I missing something?

Can you provide some graphs that show what you are talking about.
Can you share the HOST check being used AND the SERVICE check being used.

[TomS]

[rot@localhost ~]# ls -al /etc/mrtg/conf.d/ | wc -l
36
[root@localhost ~]# ls -al /var/lib/mrtg/ | wc -l
760

43.PNG

13.PNG

[TomS]

I used the Switch and router wizard to add these monitors. The following are default values for the service and host templates used on all these configs.

xiqizzard_switch_ping_service.PNG

xiwizard_switch_host.PNG

13_Services_Config.PNG

[lmiltchev]

Something is not accurate within MRTG; additionally the HOST and PING graphs are reporting distinctly different ping times on the same router or switch. Am I missing something?

Do you have errors in the MRTG config? Run the following command and show us the output (no output is good - it means "no errors"):

Code: Select all

LANG=C LC_ALL=C /usr/bin/mrtg /etc/mrtg/mrtg.cfg

The graphs for the host and the ping servers are similar enough. They cannot be identical as these checks are performed at different times and we are talking about milliseconds here...

[TomS]

No Errors in MRTG

Maybe this is a better example of what doesn't seem to make sense

13thPing_v_Bandwidth.PNG

There is no bandwidth usage reported from 10:15pm to 8:am but the ping times are high. We are closed and I would expect little to no bandwidth at this time.

[tgriep]

The ping times could be affected by the equipment that is in between the Nagios XI server and your device that you are checking.
High bandwidth usage may or may not affect the ping times. It all depends on when the check is run at that moment.

[TomS]

To the contrary, It is the absence of bandwidth with high ping times that confounds me. We have ample bandwidth at all branches (according to Nagios graphs) but still experience severe slowdowns and high pings at times throughout the day. Without access to the ATT routers in between I am certainly flying blind with regard to the bandwidth between those nodes.

I guess we can close this discussion since there is no way to isolate the latency and the Nagios graphs are nebulous for correlate the exact times and specific latencies to bandwidth (and between hosts and service on the same device for that matter)