Nagios Support Forum

Support for Nagios products and services
https://support.nagios.com/forum/

troubleshooting sources

https://support.nagios.com/forum/viewtopic.php?t=50771

Page 1 of 2

troubleshooting sources

Posted: Mon Oct 22, 2018 8:55 am
by Hoygen
Greetings,
it might seem an odd question but is there a troubleshooting guide for sources?
I have several sources showing "no data" as the traffic in the last 30 minutes.
But if I take a look in /usr/local/nagiosna/var/source_name/flows ,I can find files written recently.
Kindest regards

Re: troubleshooting sources

Posted: Mon Oct 22, 2018 2:15 pm
by benjaminsmith
Hi @Hoygen

The knowledgebase article below covers some reasons why data may not be showing up. Let me know if this helps resolves your issue.

Nagios Network Analyzer - Graph Says "No Data Available"
https://support.nagios.com/kb/article/n ... le-32.html

Another cause could be the time is not set correctly on the NNA server or the devices that are sending the flow data.
Make sure that the time did not change on the server.

Re: troubleshooting sources

Posted: Tue Oct 23, 2018 8:33 am
by Hoygen
I've read it.
I've read also here https://assets.nagios.com/downloads/nag ... ackend.pdf

it's strange because:
a) netstat -nl | grep 1060
udp 0 0 0.0.0.0:1060 0.0.0.0:*
shows the port as listening.

b) tcpdump dst port 1060 -vv
shows packets

c) ps aux | grep nfcapd | grep 1060
shows the process

d) iptables -nL | grep 1060
ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpt:1060
ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpt:1060
ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpt:1060
iptables is open

but the graph shows no data available.
is there anywhere else I should look or do to make it work?

Re: troubleshooting sources

Posted: Tue Oct 23, 2018 4:35 pm
by lmiltchev
Can you run the following command on the NNA server

Code: Select all

tcpdump -i any -s 65535 -w 1060.cap port 1060
let it run for a while (e.g. a few minutes), exit by running ctrl +c, and upload the 1060.cap file that was generated from running the above command?

Also, run:

Code: Select all

rrdtool dump /usr/local/nagiosna/var/<source_name>/bandwidth.rrd
grep -2 '<source_name>' /usr/local/nagiosna/var/backend.log | tail
and show the output in code wraps. Please, substitute '<source_name>' with the actual name of your source.

Re: troubleshooting sources

Posted: Wed Oct 24, 2018 8:57 am
by Hoygen
at the time I can't upload the rrdtool dump or the tcpdump.
should I look for something peculiar inside them?


grep -2 'STAB1CS01' /usr/local/nagiosna/var/backend.log | tail

Code: Select all

2018-10-24 15:40:00 INFO : Parsing data for the source id: 88
2018-10-24 15:40:00 DEBUG : Arguments: /usr/local/nagiosna/var/STAB1CS01/flows, nfcapd.201810241535, 88
2018-10-24 15:40:00 DEBUG : Running checks...
2018-10-24 15:40:00 DEBUG : Getting relevant checks for source id: 88
--
2018-10-24 15:45:00 DEBUG : Arguments: /usr/local/nagiosna/var/MPLS_pri_Castenaso/flows, nfcapd.201810241540, 25
2018-10-24 15:45:00 INFO : Parsing data for the source id: 88
2018-10-24 15:45:00 DEBUG : Arguments: /usr/local/nagiosna/var/STAB1CS01/flows, nfcapd.201810241540, 88
2018-10-24 15:45:00 DEBUG : Running checks...
2018-10-24 15:45:00 DEBUG : Running checks...

Re: troubleshooting sources

Posted: Wed Oct 24, 2018 9:10 am
by Hoygen
the rrddump shows, cleaning the output

Code: Select all

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE rrd SYSTEM "http://oss.oetiker.ch/rrdtool/rrdtool.dtd">
<!-- Round Robin Database Dump -->
<rrd>
	<version>0003</version>
	<step>300</step> <!-- Seconds -->
	<lastupdate>1540294056</lastupdate> <!-- 2018-10-23 13:27:36 CEST -->

	<ds>
		<name> flows </name>
		<type> GAUGE </type>
		<minimal_heartbeat>600</minimal_heartbeat>
		<min>0.0000000000e+00</min>
		<max>NaN</max>

		<!-- PDP Status -->
		<last_ds>U</last_ds>
		<value>NaN</value>
		<unknown_sec> 156 </unknown_sec>
	</ds>

	<ds>
		<name> bytes </name>
		<type> GAUGE </type>
		<minimal_heartbeat>600</minimal_heartbeat>
		<min>0.0000000000e+00</min>
		<max>NaN</max>

		<!-- PDP Status -->
		<last_ds>U</last_ds>
		<value>NaN</value>
		<unknown_sec> 156 </unknown_sec>
	</ds>

	<ds>
		<name> packets </name>
		<type> GAUGE </type>
		<minimal_heartbeat>600</minimal_heartbeat>
		<min>0.0000000000e+00</min>
		<max>NaN</max>

		<!-- PDP Status -->
		<last_ds>U</last_ds>
		<value>NaN</value>
		<unknown_sec> 156 </unknown_sec>
	</ds>

	<ds>
		<name> bps </name>
		<type> GAUGE </type>
		<minimal_heartbeat>600</minimal_heartbeat>
		<min>0.0000000000e+00</min>
		<max>NaN</max>

		<!-- PDP Status -->
		<last_ds>U</last_ds>
		<value>NaN</value>
		<unknown_sec> 156 </unknown_sec>
	</ds>

	<ds>
		<name> pps </name>
		<type> GAUGE </type>
		<minimal_heartbeat>600</minimal_heartbeat>
		<min>0.0000000000e+00</min>
		<max>NaN</max>

		<!-- PDP Status -->
		<last_ds>U</last_ds>
		<value>NaN</value>
		<unknown_sec> 156 </unknown_sec>
	</ds>

	<ds>
		<name> bpp </name>
		<type> GAUGE </type>
		<minimal_heartbeat>600</minimal_heartbeat>
		<min>0.0000000000e+00</min>
		<max>NaN</max>

		<!-- PDP Status -->
		<last_ds>U</last_ds>
		<value>NaN</value>
		<unknown_sec> 156 </unknown_sec>
	</ds>

	<!-- Round Robin Archives -->
	<rra>
		<cf>AVERAGE</cf>
		<pdp_per_row>1</pdp_per_row> <!-- 300 seconds -->

		<params>
		<xff>5.0000000000e-01</xff>
		</params>
		<cdp_prep>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<value>NaN</value>
			<unknown_datapoints>0</unknown_datapoints>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<value>NaN</value>
			<unknown_datapoints>0</unknown_datapoints>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<value>NaN</value>
			<unknown_datapoints>0</unknown_datapoints>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<value>NaN</value>
			<unknown_datapoints>0</unknown_datapoints>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<value>NaN</value>
			<unknown_datapoints>0</unknown_datapoints>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<value>NaN</value>
			<unknown_datapoints>0</unknown_datapoints>
			</ds>
		</cdp_prep>
		<database>
			<!-- 2018-08-23 16:35:00 CEST / 1535034900 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
			<!-- 2018-10-23 13:00:00 CEST / 1540292400 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
		</database>
	</rra>
	<rra>
		<cf>AVERAGE</cf>
		<pdp_per_row>288</pdp_per_row> <!-- 86400 seconds -->

		<params>
		<xff>5.0000000000e-01</xff>
		</params>
		<cdp_prep>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<value>NaN</value>
			<unknown_datapoints>137</unknown_datapoints>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<value>NaN</value>
			<unknown_datapoints>137</unknown_datapoints>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<value>NaN</value>
			<unknown_datapoints>137</unknown_datapoints>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<value>NaN</value>
			<unknown_datapoints>137</unknown_datapoints>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<value>NaN</value>
			<unknown_datapoints>137</unknown_datapoints>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<value>NaN</value>
			<unknown_datapoints>137</unknown_datapoints>
			</ds>
		</cdp_prep>
		<database>
			<!-- 2017-10-24 02:00:00 CEST / 1508803200 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
			<!-- 2018-10-23 02:00:00 CEST / 1540252800 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
		</database>
	</rra>
	<rra>
		<cf>HWPREDICT</cf>
		<pdp_per_row>1</pdp_per_row> <!-- 300 seconds -->

		<params>
		<hw_alpha>1.0000000000e-01</hw_alpha>
		<hw_beta>3.5000000000e-03</hw_beta>
		<dependent_rra_idx>4</dependent_rra_idx>
		</params>
		<cdp_prep>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<intercept>NaN</intercept>
			<last_intercept>NaN</last_intercept>
			<slope>NaN</slope>
			<last_slope>NaN</last_slope>
			<nan_count>1</nan_count>
			<last_nan_count>1</last_nan_count>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<intercept>NaN</intercept>
			<last_intercept>NaN</last_intercept>
			<slope>NaN</slope>
			<last_slope>NaN</last_slope>
			<nan_count>1</nan_count>
			<last_nan_count>1</last_nan_count>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<intercept>NaN</intercept>
			<last_intercept>NaN</last_intercept>
			<slope>NaN</slope>
			<last_slope>NaN</last_slope>
			<nan_count>1</nan_count>
			<last_nan_count>1</last_nan_count>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<intercept>NaN</intercept>
			<last_intercept>NaN</last_intercept>
			<slope>NaN</slope>
			<last_slope>NaN</last_slope>
			<nan_count>1</nan_count>
			<last_nan_count>1</last_nan_count>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<intercept>NaN</intercept>
			<last_intercept>NaN</last_intercept>
			<slope>NaN</slope>
			<last_slope>NaN</last_slope>
			<nan_count>1</nan_count>
			<last_nan_count>1</last_nan_count>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<intercept>NaN</intercept>
			<last_intercept>NaN</last_intercept>
			<slope>NaN</slope>
			<last_slope>NaN</last_slope>
			<nan_count>1</nan_count>
			<last_nan_count>1</last_nan_count>
			</ds>
		</cdp_prep>
		<database>
			<!-- 2018-10-18 13:30:00 CEST / 1539862200 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
			<!-- 2018-10-23 13:25:00 CEST / 1540293900 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
		</database>
	</rra>
	<rra>
		<cf>SEASONAL</cf>
		<pdp_per_row>1</pdp_per_row> <!-- 300 seconds -->

		<params>
		<seasonal_gamma>1.0000000000e-01</seasonal_gamma>
		<seasonal_smooth_idx>214</seasonal_smooth_idx>
		<dependent_rra_idx>3</dependent_rra_idx>
		</params>
		<cdp_prep>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<seasonal>NaN</seasonal>
			<last_seasonal>NaN</last_seasonal>
			<init_flag>1</init_flag>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<seasonal>NaN</seasonal>
			<last_seasonal>NaN</last_seasonal>
			<init_flag>1</init_flag>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<seasonal>NaN</seasonal>
			<last_seasonal>NaN</last_seasonal>
			<init_flag>1</init_flag>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<seasonal>NaN</seasonal>
			<last_seasonal>NaN</last_seasonal>
			<init_flag>1</init_flag>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<seasonal>NaN</seasonal>
			<last_seasonal>NaN</last_seasonal>
			<init_flag>1</init_flag>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<seasonal>NaN</seasonal>
			<last_seasonal>NaN</last_seasonal>
			<init_flag>1</init_flag>
			</ds>
		</cdp_prep>
		<database>
			<!-- 2018-10-22 13:30:00 CEST / 1540207800 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
			<!-- 2018-10-23 13:25:00 CEST / 1540293900 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
		</database>
	</rra>
	<rra>
		<cf>DEVSEASONAL</cf>
		<pdp_per_row>1</pdp_per_row> <!-- 300 seconds -->

		<params>
		<seasonal_gamma>1.0000000000e-01</seasonal_gamma>
		<seasonal_smooth_idx>214</seasonal_smooth_idx>
		<dependent_rra_idx>3</dependent_rra_idx>
		</params>
		<cdp_prep>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<seasonal>NaN</seasonal>
			<last_seasonal>NaN</last_seasonal>
			<init_flag>1</init_flag>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<seasonal>NaN</seasonal>
			<last_seasonal>NaN</last_seasonal>
			<init_flag>1</init_flag>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<seasonal>NaN</seasonal>
			<last_seasonal>NaN</last_seasonal>
			<init_flag>1</init_flag>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<seasonal>NaN</seasonal>
			<last_seasonal>NaN</last_seasonal>
			<init_flag>1</init_flag>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<seasonal>NaN</seasonal>
			<last_seasonal>NaN</last_seasonal>
			<init_flag>1</init_flag>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<seasonal>NaN</seasonal>
			<last_seasonal>NaN</last_seasonal>
			<init_flag>1</init_flag>
			</ds>
		</cdp_prep>
		<database>
			<!-- 2018-10-22 13:30:00 CEST / 1540207800 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
			<!-- 2018-10-23 13:25:00 CEST / 1540293900 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
		</database>
	</rra>
	<rra>
		<cf>DEVPREDICT</cf>
		<pdp_per_row>1</pdp_per_row> <!-- 300 seconds -->

		<params>
		<dependent_rra_idx>5</dependent_rra_idx>
		</params>
		<cdp_prep>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			</ds>
		</cdp_prep>
		<database>
			<!-- 2018-10-18 13:30:00 CEST / 1539862200 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
			<!-- 2018-10-23 13:25:00 CEST / 1540293900 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
		</database>
	</rra>
	<rra>
		<cf>FAILURES</cf>
		<pdp_per_row>1</pdp_per_row> <!-- 300 seconds -->

		<params>
		<delta_pos>2.0000000000e+00</delta_pos>
		<delta_neg>2.0000000000e+00</delta_neg>
		<window_len>9</window_len>
		<failure_threshold>7</failure_threshold>
		<dependent_rra_idx>5</dependent_rra_idx>
		</params>
		<cdp_prep>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<history>000000000</history>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<history>000000000</history>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<history>000000000</history>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<history>000000000</history>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<history>000000000</history>
			</ds>
			<ds>
			<primary_value>0.0000000000e+00</primary_value>
			<secondary_value>0.0000000000e+00</secondary_value>
			<history>000000000</history>
			</ds>
		</cdp_prep>
		<database>
			<!-- 2018-10-22 13:30:00 CEST / 1540207800 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v>
			<!-- 2018-10-23 13:25:00 CEST / 1540293900 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
		</database>
	</rra>
</rrd>

Re: troubleshooting sources

Posted: Wed Oct 24, 2018 4:32 pm
by tgriep
Can you go to this folder on the NNA server, get a few of the latest nfcapd files and upload them to the post so we can view the data that the NNA server is capturing?

Code: Select all

/usr/local/nagiosna/var/MPLS_pri_Castenaso/flows

Re: troubleshooting sources

Posted: Thu Oct 25, 2018 2:39 am
by Hoygen
tgriep wrote:Can you go to this folder on the NNA server, get a few of the latest nfcapd files and upload them to the post so we can view the data that the NNA server is capturing?

Code: Select all

/usr/local/nagiosna/var/MPLS_pri_Castenaso/flows
I don't have any issue with MPLS_pri_Castenaso on port 1050
I have issues with STAB1CS01 on port 1060 for example and many others
here are the nfcapd files from STAB1CS01 https://ufile.io/055uw
here is the file 1060.cap https://ufile.io/h14sr output of tcpdump -i any -s 65535 -w 1060.cap port 1060

Re: troubleshooting sources

Posted: Thu Oct 25, 2018 10:24 am
by tgriep
Sorry about selecting the wrong source name.

The nfcapd files that you posted are empty and do not have any data which matches what you see in the GUI.

I am having troubles decoding the tcpdump file and it may not be sending valid data which is why the nfcapd files are empty.

Typically, when you do not see any data in the cap files, it could be that the device is not sending flow data, or that is is blocked or in a wrong format.
The inability to decode the tcpdump file kind of points to the wrong format issue.

What is the make and model number of the device that is sending the flow data?
What format of flow data is it configured to send?
Can you post it's configuration settings so we can verify they are configured for the NNA server?

Thanks.

Re: troubleshooting sources

Posted: Fri Oct 26, 2018 8:30 am
by Hoygen
Cisco, model WS-C3850-48XS
I can't provide more details right now
All times are UTC-05:00
Page 1 of 2

Powered by phpBB® Forum Software © phpBB Limited