Are you using SSL at all for the IMAP connection? Can you verify that the date time are the same on each server, rather than an hour or two off (verify by running the date command on both machines)? Do you have the proper time zone specified? This can cause the time being off with ntp as it references the timezone.
To ensure you have the proper time zone set you will need to find your timezone in the /usr/share/zoneinfo subdirectories, and then create symlink to /etc/localtime with the proper timezone. I live in the Chicago time zone so I would something like this.
cp /etc/localtime /backups/localtime-bak #Always a good idea to make a backup
ln -sf /usr/share/zoneinfo/America/Chicago /etc/localtime
Then make sure you have the correct timezone in your /etc/sysconfig/clock file.
Ok, then the next thing to do is to telnet in from your Nagios box. You'll need to know which port you're using for IMAP. Its typically port 143, depending on your IMAP configuration it may different, and you should check to see which port it actually is.