Weird issue on CentOS 7. Connectivity not restored

[dfmco]

I have tried 4 reinstalls (clean OS and Nagios XI) on 3 different computers and each one shows the same behavior. Occasionally when a host fails and comes back in service the host remains unreachable via Nagios. Pinging from the monitored node to Nagios fails and traceroute die at the last hop before Nagios. From Nagios, ping fails but traceroutes die immediately (no first hop). If I restart networking, the connection is restored. This happened on 4 different reinstalls (formatted and installed OS fresh from 2 different sources on 2 different networks using different network hardware). Here are my questions:

Is this something that has been reported and if so, is there a fix?
If CentOS 7 is not ready for production would you recommend CentOS 6.X with a manual install or would the OVA be a better way to go? I would like the best performance possible.
If OVA, is there a way to add more CPU to the OVA without causing any issues? I also want to add a RAM drive of 200M to help with performance.

My environment is <100 nodes with >700 checks and some scripting. I want to make sure that the system can keep up. The VM server is a ML360 G8 with 64 Gig RAM and 15k RPM drives so I can throw resources at this.

[Box293]

I have tried 4 reinstalls (clean OS and Nagios XI)

Is "clean OS" just a bare minimum install of CentOS 7?
Does it have XWindows / GUI?
Is it a custom based corporate build?

From Nagios, ping fails but traceroutes die immediately (no first hop).

Is this from the XI GUI, or from the CLI of the XI server?
If it's happening from the CLI, then clearly this is a networking issue in your environment, or a bug in CentOS 7.x. We've not had any reports of the behaviours you're experiencing with XI and CentOS 7.x.

If CentOS 7 is not ready for production would you recommend CentOS 6.X with a manual install or would the OVA be a better way to go? I would like the best performance possible.
If OVA, is there a way to add more CPU to the OVA without causing any issues? I also want to add a RAM drive of 200M to help with performance.

There is no performance difference running XI on CentOS 6.x x86_64 or CentOS 7.x.
The OVA is just a fast track way to get XI up and running, there's no difference using it compared to a fresh install of CentOS 6.x.
Using a VM vs running on the hardware directly is not really an issue these days. The Hypervisor software is advanced enough that it's almost impossible to tell the difference between physical and virtual. Personally I like virtual as it makes the VM transportable and allows you to take snapshots before upgrades and easily can have more resources added.
Adding a RAM drive is a good recommendation. Here's some best practices for Nagios XI that might help:
https://support.nagios.com/kb/category.php?id=164

[dfmco]

This is a fresh minimal install of CentOS7 installed from the CentOS download site. There is no GUI.
It sounds like I need to do the move to the OVA but what issues will I run into moving from CentOS7 to 6? The documentation for a restore talks about deleting some files in the install folder but I am not sure if that is on the source machine or target. Is it safe to go from a backup/restore from 7 to 6 or would it be recommended to to a full rebuild. I really need the system to be as stable as possible.

Thanks!

[dfmco]

Also, any issues with adding processor to the OVA? I have an existing install built from OVA which is constantly in IO-Wait and I want to avoid that here.

[hsmith]

I would do a full rebuild if you do not have too much to lose.

There should be no issue adding anything to the OVA. I have never heard of this issue happening in CentOS 7. Could you perhaps post logs or screenshots, or do you need to just get this going ASAP.

[dfmco]

What kids do you need?

[hsmith]

What kids do you need?

None for awhile

I would like to see your /var/log/messages around the time you're having network issues on CentOS 7, there might be something useful in there. If not, we could check dmesg.