Page 2 of 3

Re: Inbound Mail hash errors: "^M" in cleaner.log

Posted: Mon Aug 10, 2020 1:40 pm
by benjaminsmith
Hi,

I have an update on this issue, we've been able to repeat this here with Outlook and a bug report has been filed. It's a formatting issue of some kind and we should have this accounted for in the next maintenance release of Nagios XI.

Re: Inbound Mail hash errors: "^M" in cleaner.log

Posted: Mon Aug 10, 2020 1:46 pm
by gdksc
Hi, @benjaminsmith,

Sounds good. Thanks for the update.

Re: Inbound Mail hash errors: "^M" in cleaner.log

Posted: Tue Aug 11, 2020 1:07 pm
by benjaminsmith
HI @gdksc,

Your welcome!

Re: Inbound Mail hash errors: "^M" in cleaner.log

Posted: Thu Aug 13, 2020 5:00 pm
by benjaminsmith
Hi,

If you're interested in testing this out, I have a patch from development to resolve this issue. Open up the following file on the Nagios XI Server:

Code: Select all

/usr/local/nagiosxi/html/includes/components/xicore/xicore.inc.php
Then find line 587 and replace it with the following line:

Code: Select all

$msg = substr($email['body'], 0, $msg_end); with $msg = preg_replace("/[\\r\\n]+/", "\n", substr($email['body'], 0, $msg_end))
If you have time to changes this, let me know how it goes.

Thanks!
Benjamin

Re: Inbound Mail hash errors: "^M" in cleaner.log

Posted: Fri Aug 14, 2020 9:37 am
by gdksc
Hi, @benjaminsmith,

I made the change to line 587, but still no luck.

Here's diff of the change I made and tested:

Code: Select all

# diff xicore.inc.php.orig xicore.inc.php
587c587
<             $msg = substr($email['body'], 0, $msg_end);
---
>             $msg = preg_replace("/[\\r\\n]+/", "\n", substr($email['body'], 0, $msg_end));
Here's sample result from cleaner.log:

Code: Select all

Running callbacks:

Error - Could not find a host object in the hash provided
Processed 1 incoming emails
----------------------------------
Given that error message, I'm not sure I'm getting to line 587.

Seems to fail the check starting at line 562:

Code: Select all

     
                  // Verify that at least a hostname is present
                  if (empty($obj->host)) {
                      echo "Error - Could not find a host object in the hash provided\n";
                      $error = true;
                  }
Something interesting, of note...

Previously, I'd added a line for debugging, around 563, to echo the email body:

Code: Select all

echo $email['body'];
That was where I was seeing the ^M in the hash, in the cleaner.log:

Code: Select all

-----^M
^M
To reply, keep the following data in your response email.^M
^M
##YTk2M2IzNmQ5NGRjMDU3YW1GVm5vNlNmdURYMzVHU0UxWHd1R0RweVpQdUJNTXliTy9EYThPe=^M
XVJQ0R4djRkVWxmbW5TLzY0ZSt0THppL1FuK21EOGVEZVF6eG40emJEQzZ5WmpRPT0=3D##^M
^M
Error - Could not find a host object in the hash provided
Processed 1 incoming emails
However, I've now removed the "echo email body" line replaced it with the following, starting at 561:

Code: Select all

               $hash = trim(str_replace(array('> ', '>', '=', "\n", "\t", "\r"), '', substr($email['body'], $start + 2, $end - $start - 2)));
               $obj = json_decode(decrypt_data($hash));
               echo "hash: " . $hash . "\n";
               echo "\$obj = json_decode(decrypt_data($hash))\n";
               $error = json_last_error();
               echo "json_last_error: " . $error . "\n";
               var_dump($obj);
Following are sample results from cleaner.log, after adding those lines:

Code: Select all

----------------------------------
Running callbacks:
hash: OTc2ZTAzMDgzNTI5ZmEyOTY5SHNRT3dSRi9wb012bTVaZEtLUjRqZG4rSVRlem8vS3hjT1pRaVpENkZWWXRmQnI1MG9lRHhqcmIrbEs1VzZReFBFN1lja1dHL2xsemg5NENObTVRPT03D
$obj = json_decode(decrypt_data(OTc2ZTAzMDgzNTI5ZmEyOTY5SHNRT3dSRi9wb012bTVaZEtLUjRqZG4rSVRlem8vS3hjT1pRaVpENkZWWXRmQnI1MG9lRHhqcmIrbEs1VzZReFBFN1lja1dHL2xsemg5NENObTVRPT03D))
json_last_error: 0
NULL
Error - Could not find a host object in the hash provided
Processed 1 incoming emails
----------------------------------
So, it seems the hash does not have the ^M, but the decode is not finding the host object, for some reason.

Additionally -- and this goes back to something observed early on -- all works OK for a host down alert, where the only noticeable difference is the # of characters in the hash.

A snip from the cleaner.log, for a host ack which has a smaller # of chars in the hash:

Code: Select all

----------------------------------
Running callbacks:
hash: YzdjNjFmNDNhOGUzMGM1ZUI5QkZ2TjVXN3U4MlYvQXUyWjhkRzBxQnkxRFNENTBySnZ6NTBCSmJXNmM9
$obj = json_decode(decrypt_data(YzdjNjFmNDNhOGUzMGM1ZUI5QkZ2TjVXN3U4MlYvQXUyWjhkRzBxQnkxRFNENTBySnZ6NTBCSmJXNmM9))
json_last_error: 0
object(stdClass)#8 (1) {
  ["host"]=>
  string(5) "dummy"
}
...
Is there any additional debugging we could add for original line 560, for the json_decode ?

Code: Select all

 $obj = json_decode(decrypt_data($hash));
To recap, here is the current state of my xicore.inc.php vs the original:

Code: Select all

# diff xicore.inc.php.orig xicore.inc.php
560a561,565
>                 echo "hash: " . $hash . "\n";
>                 echo "\$obj = json_decode(decrypt_data($hash))\n";
>                 $error = json_last_error();
>                 echo "json_last_error: " . $error . "\n";
>                 var_dump($obj);
587c592
<             $msg = substr($email['body'], 0, $msg_end);
---
>             $msg = preg_replace("/[\\r\\n]+/", "\n", substr($email['body'], 0, $msg_end));
Thanks again for your help with this.

Re: Inbound Mail hash errors: "^M" in cleaner.log

Posted: Fri Aug 14, 2020 11:11 am
by benjaminsmith
Hi,

Thanks for testing this out and providing feedback, this is really helpful. I'm going to share this with the developer here and I'll follow up with you again.

Benjamin

Re: Inbound Mail hash errors: "^M" in cleaner.log

Posted: Tue Aug 18, 2020 3:56 pm
by jomann
Hello, I've been testing this out internally with my own Outlook account and can you please try the following?

Can you please replace the line:

Code: Select all

$msg = preg_replace("/[\\r\\n]+/", "\n", substr($email['body'], 0, $msg_end));
with the original line:

Code: Select all

$msg = substr($email['body'], 0, $msg_end);
I think what needs to happen is we need to convert all Windows line endings to Linux for the hash to properly verify. Can you try adding the following code:

Code: Select all

$email['body'] = preg_replace("/[\\r\\n]+/", "\n",$email['body']);
right before the line:

Code: Select all

$start = strpos($email['body'], '##');

Re: Inbound Mail hash errors: "^M" in cleaner.log

Posted: Wed Aug 19, 2020 7:30 am
by gdksc
Hi, @jomann.

I added the line, so that it's the only change from the original file:

Code: Select all

# diff xicore.inc.php.orig xicore.inc.php
554a555
>             $email['body'] = preg_replace("/[\\r\\n]+/", "\n",$email['body']);
For a first test, I sent a reply email with an "ack" to an alert (e.g. Disk Usage on /var), which included the hash line:

Code: Select all

-----

To reply, keep the following data in your response email.

##M2Q1NGFhYmU1MDcyNzdiNlNxY051VjhVUFhVL0xYZWhyOGlSaGJLOVE5S05JUGFWQ3dPZi91enhiYk11MkZWSzBOYTZLWVpnaHRjZy9iOG94dndVQ3JBcmdXcXBjM1dhVUdpUG9nPT0=##
Unfortunately, I still received the error in the cleaner.log:

Code: Select all

----------------------------------
Running callbacks:
Error - Could not find a host object in the hash provided
Processed 1 incoming emails
----------------------------------
For a second test, I sent a reply email with an "ack" to a different alert (e.g. Host is Up), which included the hash line:

Code: Select all

-----

To reply, keep the following data in your response email.

##Nzk1OGZkNWZlM2RmMzY3MUFoMUdLTVYrdm9XZ0pYcFJzc0FkNEtmSlpKVDE5a1ltZVNqRDRwb2lZeFU9##
And the hash was decoded, per the cleaner.log:

Code: Select all

----------------------------------
Running callbacks:
msh_end: 386
msg_lines_count: 15
Array
(
    [0] => ack
...
For some additional tests, I re-added some "echo" statements to xicore.inc.php:

Code: Select all

# diff xicore.inc.php.orig xicore.inc.php
554a555,556
>             $email['body'] = preg_replace("/[\\r\\n]+/", "\n",$email['body']);
>             echo $email['body'];
560a563,569
>
>                  echo "hash: " . $hash . "\n";
>                  echo "\$obj = json_decode(decrypt_data($hash))\n";
>                  $error = json_last_error();
>                  echo "json_last_error: " . $error . "\n";
>                  var_dump($obj);
>
Then I re-sent the two "ack" reply emails.

From looking at the cleaner.log, ^M are being stripped from the email body, as expected. I no longer see the ^M in the cleaner.log, when I echo the email body.

However, I'm still having trouble decoding this hash:

Code: Select all

hash: M2Q1NGFhYmU1MDcyNzdiNlNxY051VjhVUFhVL0xYZWhyOGlSaGJLOVE5S05JUGFWQ3dPZi91enhiYk11MkZWSzBOYTZLWVpnaHRjZy9iOG94dndVQ3JBcmdXcXBjM1dhVUdpUG9nPT03D
$obj = json_decode(decrypt_data(M2Q1NGFhYmU1MDcyNzdiNlNxY051VjhVUFhVL0xYZWhyOGlSaGJLOVE5S05JUGFWQ3dPZi91enhiYk11MkZWSzBOYTZLWVpnaHRjZy9iOG94dndVQ3JBcmdXcXBjM1dhVUdpUG9nPT03D))
json_last_error: 0
NULL
Error - Could not find a host object in the hash provided
Processed 1 incoming emails
----------------------------------
While not having trouble decoding this hash:

Code: Select all

hash: Nzk1OGZkNWZlM2RmMzY3MUFoMUdLTVYrdm9XZ0pYcFJzc0FkNEtmSlpKVDE5a1ltZVNqRDRwb2lZeFU9
$obj = json_decode(decrypt_data(Nzk1OGZkNWZlM2RmMzY3MUFoMUdLTVYrdm9XZ0pYcFJzc0FkNEtmSlpKVDE5a1ltZVNqRDRwb2lZeFU9))
json_last_error: 0
object(stdClass)#8 (1) {
  ["host"]=>
  string(14) "fbv-nagios-d01"
}
msh_end: 386
msg_lines_count: 15
Array
(
    [0] => ack
...
Let me know if any additional debugging suggestions...

Thanks again for your help.

Re: Inbound Mail hash errors: "^M" in cleaner.log

Posted: Wed Aug 19, 2020 1:22 pm
by jomann
The first hash you are testing was changed to PT03D instead of PT0= which would definitely cause problems. There shouldn't be any numbers or letters after the = in the encrypted object data hash. In the example above the one you just posted, it shows the hash having the value TPO=3D inside the hash portion, which shouldn't happen when the object is encrypted.

Can you try decoding with the = still in the hash?

You can also try adjusting the line that creates the hash with str_replace to remove the '=' as a replaceable character. This line,

Code: Select all

$hash = trim(str_replace(array('> ', '>', '=', "\n", "\t", "\r"), '', substr($email['body'], $start + 2, $end - $start - 2)));
to this line instead

Code: Select all

$hash = trim(str_replace(array('> ', '>', "\n", "\t", "\r"), '', substr($email['body'], $start + 2, $end - $start - 2)));

Re: Inbound Mail hash errors: "^M" in cleaner.log

Posted: Wed Aug 19, 2020 1:59 pm
by gdksc
Very interesting...

The "=" is ASCII "3D".

I tested changing the hash in my reply email to be simply five "=", like this:

Code: Select all

-----

To reply, keep the following data in your response email.

##=====##
And here's what I received in the cleaner.log, given I still have the "echo" statements in there:

Code: Select all

-----
To reply, keep the following data in your response email.
##=3D=3D=3D=3D=3D##
PHP Warning:  openssl_decrypt(): IV passed is only 7 bytes long, cipher expects an IV of precisely 16 bytes, padding with \0 in /usr/local/nagiosxi/html/includes/utilsl-helpers.inc.php on line 0
hash: 3D3D3D3D3D
$obj = json_decode(decrypt_data(3D3D3D3D3D))
json_last_error: 0
NULL
Error - Could not find a host object in the hash provided
Processed 1 incoming emails
----------------------------------
Something seems to be converting the "=" in the hash to "3D"...