Not receiving SNMP Traps from Nagios

[dfmco]

CentOS version 7 and Nagios 5.2.7 with manual install
I ran the snmp trap setup script and configured the interface down traps as critical. I am not getting any logs in /var/log/snmptt folder except the system logs. The devices being monitored are Cisco using SNMP V3. I have a separate install with the same configuration that does work.

[gormank]

Do you see snmptrapd log entries in /var/log/messages or wherever you have it set to log?
You can see if you're getting traps with tcpdump.
tcpdump -i eth0 -s 0 port 162

If you get traps but snmptrapd doesn't log, stop iptables.

[dfmco]

CentOS 7 uses firewalld and it is not running:
[root]# systemctl status firewalld
● firewalld.service
Loaded: not-found (Reason: No such file or directory)
Active: inactive (dead)

[dfmco]

I am also getting the encrypted trap traffic:
[root]# tcpdump -i eno16777984 -s 0 port 162
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eno16777984, link-type EN10MB (Ethernet), capture size 65535 bytes
15:13:04.488133 IP 10.17.120.252.snmptrap > dfmco-rbuc-esx01.snmptrap: F=ap U=nms [!scoped PDU]fc_51_d2_53_71_3b_b9_d9_d3_92_a6_39_04_8d_b2_4e_15_c6_c9_5b_8f_e1_4b_61_4e_4a_a5_a5_aa_2e_38_6a_30_32_d4_1c_52_ca_4d_19_ee_f2_ca_91_23_13_e4_c2_47_a9_26_c6_c2_95_9a_c5_3e_61_fa_05_61_39_f6_07_c0_d2_c5_23_37_be_ec_ec_a8_c0_c8_2d_23_ec_cd_ae_a9_a7_17_3f_47_0f_a1_2c
15:13:05.148132 IP 10.17.16.4.iad2 > dfmco-rbuc-esx01.snmptrap: F=ap U=nms [!scoped PDU]9c_68_64_d8_3e_11_a8_61_99_2a_41_cb_ea_ce_59_80_18_0e_6b_6b_71_b4_f3_6d_be_8d_76_fd_26_32_65_38_b4_e3_87_6d_4e_71_5c_3c_21_7f_2f_3b_54_3b_dc_52_3f_2d_fa_de_ae_e4_0d_68_3e_76_78_35_0b_d1_20_0b_8a_e1_f6_e2_7d_27_67_36_6c_11_8b_96_5f_b5_74_af_31_d4_7d_1b_cd_14_85_2b_98_31_43_c1_ec_24_c0_ec_b0_82_ac_59_31_b3_25_ad_1d_23_59_51_ba_7a_f0_74_54_7c_de_94_8f_42_4c_1e_b4_8e_81_a8_ec_99_47_c0_fe_3d_53_76_1c_a6_f9_a5_9a_d7_c4_69_b6_a7_19_a8_c3_12
15:13:06.490142 IP 10.17.120.252.snmptrap > dfmco-rbuc-esx01.snmptrap: F=ap U=nms [!scoped PDU]4d_32_e3_c3_8e_66_eb_26_ed_b5_a5_89_43_00_7e_4c_cf_38_46_c2_22_ee_0f_c6_f9_ab_6b_26_91_39_1a_ff_a1_81_7e_b7_11_fd_b1_52_b8_b9_42_5d_71_34_bb_b8_59_24_bd_ad_2b_2b_8e_61_96_f0_23_98_bc_56_2a_20_08_e5_6c_12_54_07_54_a5_f9_2e_a9_9c_9b_4c_ae_e3_81_f2_e8_c3_04_2b_26_53
15:13:07.148168 IP 10.17.16.4.iad2 > dfmco-rbuc-esx01.snmptrap: F=ap U=nms [!scoped PDU]b3_a3_c8_7c_95_67_12_fc_57_73_5b_59_63_ed_10_37_63_bd_cc_76_65_89_72_a3_4d_c6_ae_84_d9_8c_92_3f_ce_8c_79_9b_14_75_28_a8_77_82_22_f9_8f_67_2d_6b_c5_59_84_99_33_d6_db_86_f9_50_78_bc_12_0b_fa_ae_87_e4_50_c9_ba_c5_94_3f_32_1f_ce_64_3e_b9_6e_32_21_19_2f_0e_8e_56_25_73_6f_79_f9_99_32_75_f0_0d_3c_07_00_1c_34_27_2c_51_b9_3b_8e_91_19_39_26_38_a5_3a_7e_7c_5b_e4_dc_6e_b9_28_ca_52_a4_60_4f_f7_5c_c2_43_78_4b_55_97_6d_68_74_fe_1a_65_29_1b_08_7a_69
15:13:08.492085 IP 10.17.120.252.snmptrap > dfmco-rbuc-esx01.snmptrap: F=ap U=nms [!scoped PDU]18_fc_55_dd_cd_0a_33_3f_75_ed_74_e2_ec_a1_17_9b_01_d4_bf_36_a3_f6_42_5c_dc_95_76_10_40_b6_bd_21_c7_be_4b_27_7e_55_43_3f_c1_8a_a8_8c_b8_5e_e2_c6_7d_b5_a5_92_f8_95_a0_73_3d_60_27_cb_ab_c4_42_b4_56_89_8e_c1_91_57_8b_f3_05_65_90_82_cf_ac_96_18_11_9e_6e_65_3f_ba_20_3d
15:13:09.160115 IP 10.17.16.4.iad2 > dfmco-rbuc-esx01.snmptrap: F=ap U=nms [!scoped PDU]b0_a0_5b_5e_0f_c0_b2_05_31_02_af_a8_de_82_6c_d2_6b_e3_71_c3_d0_de_41_cf_00_67_3a_a8_e6_0f_72_ba_95_6c_dc_0e_93_ab_23_0f_4c_84_b2_44_48_60_45_73_85_a6_96_f9_a2_38_0e_95_64_eb_ab_0f_58_71_44_12_d1_13_6e_1a_5c_86_a0_b0_3a_7d_1e_d7_5f_a5_a6_64_c0_19_cd_e1_22_1c_71_1a_9c_bd_f4_e3_33_4f_ed_a4_d8_2e_fe_43_7b_1b_44_62_ad_6b_5b_da_4f_a3_4b_4d_96_ab_24_f8_2b_1e_12_2b_71_97_6e_75_bf_08_d0_4e_f8_8a_2a_72_83_b2_67_35_9f_86_0d_37_11_1b_0a_09_41_76
15:13:10.494180 IP 10.17.120.252.snmptrap > dfmco-rbuc-esx01.snmptrap: F=ap U=nms [!scoped PDU]20_04_8d_3c_75_92_70_8d_08_06_40_75_20_ae_a4_2e_bf_30_2d_64_44_5c_f1_eb_49_ee_f1_5e_6f_e1_71_86_3c_e2_a5_63_fa_34_96_c7_a2_68_a7_0f_38_27_1f_d5_eb_a6_42_d6_43_5b_9e_c1_3d_1b_69_6b_49_96_63_f9_9c_4d_2c_8b_8b_cb_7c_d7_e5_de_59_ac_ff_f7_fe_0d_2b_0a_0b_22_32_d1_36_b0
15:13:11.161086 IP 10.17.16.4.iad2 > dfmco-rbuc-esx01.snmptrap: F=ap U=nms [!scoped PDU]4e_1e_43_a0_8b_64_92_12_bc_6c_56_a7_ad_b8_c8_52_0d_90_71_c8_e4_82_09_52_e5_56_b0_50_2c_31_d9_c6_69_3b_98_27_6f_55_75_43_e6_b2_37_2c_62_03_e1_a5_7a_b3_1a_3e_86_c2_be_53_89_9f_a7_47_a7_86_68_50_09_24_3a_2a_b4_ef_65_16_73_64_36_53_8c_a5_0b_94_76_a9_f2_96_c3_24_f6_b6_e8_ad_56_48_a0_b6_23_32_c4_48_d1_68_22_ea_ec_54_57_d5_e0_dc_c4_e2_1b_42_fb_94_27_3e_fa_59_dd_d9_e7_73_ab_ce_87_a4_ef_e6_30_f9_70_a2_45_41_35_7f_2e_4d_cc_34_e3_e8_90_d3_24_ec
15:13:12.758169 IP 10.17.104.4.62147 > dfmco-rbuc-esx01.snmptrap: F=ap U=nms [!scoped PDU]85_82_d1_9e_ed_40_cc_ec_f3_d6_42_10_26_39_01_76_32_c0_69_17_3a_b0_d1_18_1c_a1_49_6d_c4_2a_53_5a_86_06_3f_bc_be_4c_31_e2_21_ea_5b_12_c5_1f_75_42_bb_9d_ce_95_e1_c9_54_a0_ad_6a_24_93_5b_af_19_2b_1d_d4_00_50_05_54_7f_03_ba_a8_53_11_3b_12_04_8e_bc_84_88_46_a5_ce_c8_01_8e_34_4f_4d_e9_88_2c_ce_c2_9c_0d_3d_45_5b_20_02_fa_57_47_11_44_82_07_d0_e6_ea_15_91_79_a4_bc_26_f1_b4_66_17_d3_1d_7f_ec_05_92_20_5b_9a_69_b4_4a_23_08_e1_05_b4_92_68_47_0d_84
15:13:12.915201 IP 10.17.16.252.snmptrap > dfmco-rbuc-esx01.snmptrap: F=ap U=nms [!scoped PDU]72_1f_24_03_70_61_18_4e_87_bc_a8_a5_10_50_87_3b_6f_b2_65_23_c6_10_73_53_7b_12_aa_06_ed_84_8e_d0_36_d1_94_b7_b4_6f_37_5f_cb_80_ce_44_2c_86_97_10_9d_a1_1d_1f_6d_f3_75_33_8b_10_b6_ea_32_d0_44_4e_d8_08_1e_f0_11_ac_74_37_87_bc_e6_2c_15_03_3f_6d_a7_9c_ab_ff_02_c8_d7
15:13:14.754195 IP 10.17.104.4.62147 > dfmco-rbuc-esx01.snmptrap: F=ap U=nms [!scoped PDU]4d_a1_da_fc_a0_ce_52_75_89_81_92_be_9b_ca_24_e0_d4_cd_37_22_d2_64_11_82_b1_42_83_39_a4_54_9d_ad_4d_11_35_e6_8a_4f_d9_05_b7_05_6f_ca_2d_da_c0_d8_bc_82_b1_49_b7_f8_b3_3f_39_99_99_aa_cc_69_61_84_86_f9_b8_58_04_b1_a3_c0_22_82_cc_74_76_21_90_4b_7e_70_a6_bc_4a_44_1e_46_4a_e1_87_5d_d3_e4_d8_54_ad_ed_63_f4_29_a7_82_84_25_c6_ae_04_2b_12_c5_44_44_43_58_03_b9_fc_48_1d_62_ca_c2_99_5c_3b_3a_b7_5f_01_19_fb_3c_55_f1_d9_6d_cc_b1_fb_55_11_c3_c5_87_52
15:13:14.916215 IP 10.17.16.252.snmptrap > dfmco-rbuc-esx01.snmptrap: F=ap U=nms [!scoped PDU]4c_7a_39_d1_dc_6c_ce_ea_3c_05_8a_79_e0_01_a5_a5_e0_14_0a_06_31_6e_22_fb_af_c9_64_d9_2f_92_66_26_83_1d_1f_67_a6_9a_75_f5_cc_32_9b_99_59_72_b1_76_12_2f_8e_d8_d8_60_c7_30_3f_57_fb_b7_2a_9a_eb_cb_e6_ad_ca_45_3c_6a_a7_77_cd_d6_7f_10_68_9e_ef_a0_ca_bd_b8_bf_a8_6c_e8
15:13:16.759434 IP 10.17.104.4.62147 > dfmco-rbuc-esx01.snmptrap: F=ap U=nms [!scoped PDU]5b_c3_4b_98_88_23_93_46_86_3d_d2_fc_39_58_67_bb_be_1d_42_22_cf_90_f2_08_68_db_d3_f6_d1_24_9a_d6_7f_a0_e8_46_7f_ef_0b_11_45_41_8a_74_af_87_74_ba_e5_ec_eb_a2_fc_81_4c_00_6d_ee_31_86_c5_d9_d7_8f_f9_99_38_6b_f6_db_24_ba_66_e9_b4_0f_5e_ac_b2_19_51_d5_a2_a0_6c_fa_19_16_9d_84_ce_42_46_99_f8_a5_95_48_1d_e8_01_ad_9e_6d_f5_bc_4b_b6_e7_71_bf_9b_44_fd_28_e2_f2_2e_79_63_44_5a_a6_15_2c_6d_6f_6d_ca_bf_d1_ff_3d_e6_56_5d_b8_c6_9c_79_c5_52_0b_e5_e8_ab
15:13:16.917231 IP 10.17.16.252.snmptrap > dfmco-rbuc-esx01.snmptrap: F=ap U=nms [!scoped PDU]3d_ff_d7_92_2a_df_c1_3d_5f_64_e2_e4_92_4a_a9_5d_63_04_93_14_2c_3a_28_61_27_ca_f4_4e_67_60_6f_07_8d_36_83_02_ce_e9_a7_c3_dc_93_59_de_5f_63_bc_d0_ea_bf_e7_12_c8_f3_d5_44_56_1c_a7_84_5a_f0_d1_a5_35_2c_bd_a0_f3_63_50_72_d3_48_f2_95_62_05_e5_e6_1b_e0_69_3d_2b_74_dc
^C

[dfmco]

I think I have it sorted. I totally forgot to add the SNMP V3 user ID to the snmptrapd.conf file!

[tmcdonald]

Sorry for the silence, you caught us near the end of our day. Is this for sure working now for you?

[dfmco]

It looks like it is still broken unfortunately. I added the SNMP v3 information to the snmptt file and restarted the server but no luck. Still not seeing anything in the snmptt log folder. I am getting the traps from the routers, switches and firewalls.

[dfmco]

tcpdump shows that I am still getting traps:
listening on eno16777984, link-type EN10MB (Ethernet), capture size 65535 bytes
16:03:50.197160 IP 10.17.144.252.snmptrap > dfmco-rbuc-esx01.snmptrap: F=ap U=nms [!scoped PDU]41_21_34_2e_65_53_cf_21_6f_2b_23_7d_4d_19_03_9e_8a_cf_6e_11_b9_2b_49_f7_ee_46_69_81_b2_2c_ad_15_51_03_13_a0_f7_28_c7_e6_15_4f_ae_98_ea_ea_51_6e_84_e2_3f_8f_1b_2f_99_44_0e_c4_25_1c_aa_05_54_d4_0e_e1_1b_06_be_e0_1b_a9_9c_ec_7d_eb_c5_b8_02_7d_46_66_a9_74_e4_ba_6e
16:03:50.240057 IP 10.12.0.252.snmptrap > dfmco-rbuc-esx01.snmptrap: F=ap U=nms [!scoped PDU]8e_f8_dc_f3_03_5c_08_e1_1e_a3_d2_b8_78_ee_af_cf_57_df_dd_ec_7c_31_0d_ae_2e_80_3e_ef_52_a0_e3_64_50_50_15_ea_ee_56_65_13_e7_ac_e8_92_61_1f_59_ae_05_21_1c_31_e7_4c_37_af_2f_9d_67_b6_bf_d8_6b_0e_a4_5e_70_7c_29_e6_71_c4_da_cb_f1_1f_a7_06_d0_3d_89_59_a1_b1_a9_58_3e_c9
16:03:52.191218 IP 10.17.120.252.snmptrap > dfmco-rbuc-esx01.snmptrap: F=ap U=nms [!scoped PDU]21_f0_d7_e6_e9_42_95_55_6a_de_ec_69_45_2d_a2_41_25_87_02_b9_0b_05_3b_de_83_dd_5e_1e_3e_2c_94_13_f4_80_d9_1d_73_08_ab_4f_76_4a_7d_12_45_03_69_4d_ac_1f_3d_94_7c_5a_d9_a6_c4_33_cb_6c_8f_0d_7d_b9_a1_a7_97_48_b2_a2_f8_81_a3_8f_40_15_84_36_0b_53_89_26_f4_76_b4_64_c7_8f

There are no traps received in Nagios (even under unconfigured objects) and no logs besides the system log in the smtptt folder. Any ideas?

[rkennedy]

Can you run the following, and post the output? ls -l /var/spool/smptt/

See if there are any error logs showing up in there, as well, and post them if so.

[dfmco]

[root@dfmco-rbuc-esx01 snmptt]# ls -l /var/spool/snmptt/
total 0

Nothing in there at all.