Monday, July 25, 2016

The Importance of Email Retention, Journaling and Recoverability (and why Cloud Solutions Fail)

For most of us, email is simply a means of communicating work.  It's a glorified, bi-directional to-do list with comments. Emails come in, we read, do and delete. Once the work is done, there's usually no need to find the email again. 

That's all very true except for when something goes wrong and your company gets taken to court. Suddenly then, all those deleted emails are very, very important. 

How Email Discovery could work under Litigation

So, assuming that there's a legal case, such as a lawsuit, that involves your company. You could be asked to produce all the emails within a given period (say, six months) which include certain key phrases -- or perhaps all emails from a now-terminated employee.

In the event that email cannot be produced, you could be fined or worse, you could lose the ability to defend your company in court. 

...and it doesn't stop there, your company might not even be directly involved in the court case but could be dragged in as a third party via a subpoena.

No matter how lawfully YOU run your business, not having adequate email retention is a business risk that you simply can't afford to take.

Why A Restore is Not Enough

From an IT point of view, we don't usually talk about legal things.  We're all about Backups and Restores (and recoverability) as if it's simply a matter of getting missing data back.

The theory, for example being that if a user loses their file today and they've not worked on it for a couple of weeks, then any backup from the last few weeks is sufficient.

In this case, the intention is simply to recover the lost data.

What we're finding though is that recoverability is much more than simple data restoration. What if, instead of simply recovering the data, we had to prove that no changes had occurred in it between the recovered date and the date it was deleted.  The only way to do that would be to recover all versions of this (or to have unquestioned data tracking enabled).

Email is very much a dynamic kind of "file".  For example, you might be able to recover a mail from July 7 which was deleted on July 20, via the Backup from July 15, but that doesn't mean that someone didn't reply to that message on July 16 and then delete the reply along with the original message on July 20.

Mail Journaling

There's only one sure way to demonstrate that you've effectively captured all email;

Have a copy of every single inbound, outbound and internal mail copied to mail storage which does not permit deletion - and retain that mail for the appropriate legal period (not necessarily 7 years) - even if the employee in question has left the company. 

and...

Have auditing facilities in place to protect the mail stores from administrator intervention or unauthorised access and, have a monitoring process watching the store-process to ensure that it doesn't stop.

In our case, we've been using the Veritas solution... but now we've discovered that moving to IBM Verse will prevent us from being able to journal purely internal mail.

Why the Cloud Systems are failing us

In the past, when we had our own mail servers on-site, we could direct outbound SMTP traffic to go via our external archiving partners, our inbound mail could be captured via redirected MX records and our purely internal mail could be captured via Journaling.

With the cloud services, attempting to provide a one-size-fits-all solution, these options are not necessarily available to us. In our case, with IBM Verse, we've been able to sort out inbound and outbound mail mail via the traditional means (after a bit of fiddling) but it turns out that there's no way to journal purely internal mail to an external system (so much for open systems).

We have to abandon our archive solution and go for IBM's offering -- except, of course, that we can't really abandon our old solution because we need to keep it going, possibly indefinitely... unless we migrate it elsewhere (See the Chart at the end of this post).

I've looked at Microsoft and Google and they seem to have the same problems. Their products don't seem to support external journaling any more (or they're in the process of depreciating them).

I've also noticed that since we're using cloud services, it's no longer possible to restore mail (after the trash has been emptied).  This too is a feature of the three cloud services I looked at.

One thing is certain - If you're looking to put your email in the cloud you MUST subscribe to the cloud mail retention service from the SAME vendor.... and, the choices you make today could be the choices you continue to pay for well after you've migrated to a competitor's system. 

Recommended Reading and thinking

The whole Email Retention thing pretty much kicked off in 2002 with the Sarbanes-Oxley Act in the US.  Most of the western world now has an equivalent act in place. If you're not up on that, it's good reading.

The whole Hillary Clinton thing is worth reading too - it's a bit wider than simply mail preservation but it's a good example of the rules around email in action.

There are lots of free whitepapers around on Email Retention. Just do a google search and click on some of the PDFs that come up.

Thinking more widely, we need to be prepared for the next leap in litigation; At some point, the courts are going to start asking people to produce records of instant messaging, posts and comments on collaboration platforms.

Do your staff leaving processes leave their collaborative data intact and allocated to the original owner?  How do you handle "deleted comments"?


How Long do we need to Retain Email?

This excellent chart is from Contural Inc's excellent 2007 Whitepaper: How Long Should Email be Saved?  It was sponsored by Symantec who have since moved the business to Veritas.  The chart shows that different types of emails have different retention times.


No comments: