A Little History
Over the years, we've had a fairly good run when it comes to viruses and malware. Much of that I can put down to the fact that we've always used IBM Notes as our mail system and it's less susceptible to hijacking. Of course, notes only slows down the distribution (and reduces the likelihood of specific mail calls being used). It's not an effective anti-virus solution.
Years ago, I used to run my anti-spam services on the mail server. There were two problems with this approach;
Anti-Spam was the first service we moved offsite.
For the past few years, we've been using the Symantec.Cloud anti-spam service. This was a very good service when it was a recent acquisition (MessageLabs). Back in those days, the spam used to pass through the filters of many of the major anti-spam vendors. These days, I think that it only runs through the Symatnec solution; making it far less valuable. We're finding that more and more spam is slipping through.
Our desktop scanners are Kaspersky. We've spent years on Symantec/Norton (slowed all of our PCs down) and McAfee (never actually caught anything) and Kaspersky has been pretty good overall but it didn't catch this one.
So How did it Start?
In this case, the email that made it into our systems was a variant of the Australia Post cryptolocker email that hit Australia from August last year onwards. This particular email looks very similar to real emails that Australia Post sends out. Our users had been warned about this particular problem three or four months ago but the fact is that if you keep throwing links at an organisation, eventually you're going to get lucky.
Detection
The first sign of trouble was when some of our users called the helpdesk saying that their files were encrypted. I was just standing up to go off to lunch but luckily I decided to investigate. This is why you need a responsive helpdesk - The reaction (and recognition of the problem) was time-critical. I immediately ascertained that the files were not .zip they were simply normal files renamed with .encrypted -- and there was a whole folder full of them.
I'd been following trends and reading bulletins from AusCERT, so although I didn't know the exact effects of cryptolocker, I immediately suspected it was the problem.
I quickly googled signs of it and discovered that the ransom message was the clue. I looked for one on the person's computer but couldn't find one. I couldn't see one on the network either. I was just about to start disconnecting all devices from the network (all our PCs go to the servers via a single, easily isolated switch) when a user reported an unusual message. We'd found the PC with the issue ... and it was a different PC to the one which reported the problem. We immediately disconnected it from the network and started a local scan on it.
Confirming the Problem
I was pretty sure that Cryptolocker was malware, not a virus (meaning that it could wreck files but it couldn't infect) but I needed to be sure. I called one of our suppliers who had knowledge of cryptolocker and he advised me to look for the ransom notes in all the folders. There was a html and a txt version called "HELP_TO_DECRYPT_YOUR_FILES.txt" -- though some variants of cryptolocker use different names. They hadn't been there prior to the message but now they were everywhere. If you want to read them, open the text file.... there was too much HTML in the the other file, and it's too risky.
Looking at the properties of these ransom notes, we were able to confirm that all of them were created by the same user. There was only one problematic PC and it was now disconnected.
Cleaning Up
I already knew that the cryptolocker malware uses irreversible encryption, so the choices were either "pay up" or restore.
If you're interested, paying up was about $400 AUD with a timer set to go off in a few hours that would increase the price to $1,400. They wanted their money in bitcoin.
I know people and companies who have paid up and they have had their files decrypted, so at least these people seem to have some honour. Of course, if you have a decent backup, then it's safer not to draw attention to yourself.
In our case, we have drive shadowing turned on for our main drives which results in them being copied every two hours. It also makes restoration fairly simple.
The process of recovery was still long, but mainly because I wanted to be careful.
Tips and Problems in Restoration
I'm always telling people never to restore things to the same folders. There's lots of good reasons for this which I won't go into right now. We didn't have enough space to restore all of our data at once, so we did it in chunks. Then we copied each chunk over the top of the good data (without overwriting). This meant that if a file was missing (because it had been renamed to .encrypted), it got restored but if a file was new/unaffected, it wasn't overwritten with an older version.
Part way though the restore process, we discovered that the malware had been triggered about three hours prior and that some files being restored had already been affected. Once we'd finished restoring the 10am files, we repeated the process with a 7am copy (which was definitely prior to the email). That way we made sure that all of the right files were restored.
Getting rid of the Rubbish
The last things we did were;
Del *.encrypted /s
On each affected drive letter. This removed the encrypted files. We also did a
Del HELP_TO_DECRYPT_YOUR_FILES.* /s
It certainly helps to know DOS.
As to the infected PC...,
Over the years, we've had a fairly good run when it comes to viruses and malware. Much of that I can put down to the fact that we've always used IBM Notes as our mail system and it's less susceptible to hijacking. Of course, notes only slows down the distribution (and reduces the likelihood of specific mail calls being used). It's not an effective anti-virus solution.
Years ago, I used to run my anti-spam services on the mail server. There were two problems with this approach;
- The mail had already reached our systems before the first scan occurred - even if it was just spam, you're now using your bandwidth and your storage.
- You're running secondary processes on (or between) your mail server. It needs updates, maintenance etc.
Anti-Spam was the first service we moved offsite.
For the past few years, we've been using the Symantec.Cloud anti-spam service. This was a very good service when it was a recent acquisition (MessageLabs). Back in those days, the spam used to pass through the filters of many of the major anti-spam vendors. These days, I think that it only runs through the Symatnec solution; making it far less valuable. We're finding that more and more spam is slipping through.
Our desktop scanners are Kaspersky. We've spent years on Symantec/Norton (slowed all of our PCs down) and McAfee (never actually caught anything) and Kaspersky has been pretty good overall but it didn't catch this one.
So How did it Start?
In this case, the email that made it into our systems was a variant of the Australia Post cryptolocker email that hit Australia from August last year onwards. This particular email looks very similar to real emails that Australia Post sends out. Our users had been warned about this particular problem three or four months ago but the fact is that if you keep throwing links at an organisation, eventually you're going to get lucky.
Detection
The first sign of trouble was when some of our users called the helpdesk saying that their files were encrypted. I was just standing up to go off to lunch but luckily I decided to investigate. This is why you need a responsive helpdesk - The reaction (and recognition of the problem) was time-critical. I immediately ascertained that the files were not .zip they were simply normal files renamed with .encrypted -- and there was a whole folder full of them.
I'd been following trends and reading bulletins from AusCERT, so although I didn't know the exact effects of cryptolocker, I immediately suspected it was the problem.
I quickly googled signs of it and discovered that the ransom message was the clue. I looked for one on the person's computer but couldn't find one. I couldn't see one on the network either. I was just about to start disconnecting all devices from the network (all our PCs go to the servers via a single, easily isolated switch) when a user reported an unusual message. We'd found the PC with the issue ... and it was a different PC to the one which reported the problem. We immediately disconnected it from the network and started a local scan on it.
Confirming the Problem
I was pretty sure that Cryptolocker was malware, not a virus (meaning that it could wreck files but it couldn't infect) but I needed to be sure. I called one of our suppliers who had knowledge of cryptolocker and he advised me to look for the ransom notes in all the folders. There was a html and a txt version called "HELP_TO_DECRYPT_YOUR_FILES.txt" -- though some variants of cryptolocker use different names. They hadn't been there prior to the message but now they were everywhere. If you want to read them, open the text file.... there was too much HTML in the the other file, and it's too risky.
Looking at the properties of these ransom notes, we were able to confirm that all of them were created by the same user. There was only one problematic PC and it was now disconnected.
Cleaning Up
I already knew that the cryptolocker malware uses irreversible encryption, so the choices were either "pay up" or restore.
If you're interested, paying up was about $400 AUD with a timer set to go off in a few hours that would increase the price to $1,400. They wanted their money in bitcoin.
I know people and companies who have paid up and they have had their files decrypted, so at least these people seem to have some honour. Of course, if you have a decent backup, then it's safer not to draw attention to yourself.
In our case, we have drive shadowing turned on for our main drives which results in them being copied every two hours. It also makes restoration fairly simple.
The process of recovery was still long, but mainly because I wanted to be careful.
Tips and Problems in Restoration
I'm always telling people never to restore things to the same folders. There's lots of good reasons for this which I won't go into right now. We didn't have enough space to restore all of our data at once, so we did it in chunks. Then we copied each chunk over the top of the good data (without overwriting). This meant that if a file was missing (because it had been renamed to .encrypted), it got restored but if a file was new/unaffected, it wasn't overwritten with an older version.
Part way though the restore process, we discovered that the malware had been triggered about three hours prior and that some files being restored had already been affected. Once we'd finished restoring the 10am files, we repeated the process with a 7am copy (which was definitely prior to the email). That way we made sure that all of the right files were restored.
Getting rid of the Rubbish
The last things we did were;
Del *.encrypted /s
On each affected drive letter. This removed the encrypted files. We also did a
Del HELP_TO_DECRYPT_YOUR_FILES.* /s
It certainly helps to know DOS.
As to the infected PC...,
- A complete scan using a current version of Kaspersky took nearly 24 hours and discovered nothing.
- The PC has now been wiped.
Comments