Monday, July 25, 2016

The Importance of Email Retention, Journaling and Recoverability (and why Cloud Solutions Fail)

For most of us, email is simply a means of communicating work.  It's a glorified, bi-directional to-do list with comments. Emails come in, we read, do and delete. Once the work is done, there's usually no need to find the email again. 

That's all very true except for when something goes wrong and your company gets taken to court. Suddenly then, all those deleted emails are very, very important. 

How Email Discovery could work under Litigation

So, assuming that there's a legal case, such as a lawsuit, that involves your company. You could be asked to produce all the emails within a given period (say, six months) which include certain key phrases -- or perhaps all emails from a now-terminated employee.

In the event that email cannot be produced, you could be fined or worse, you could lose the ability to defend your company in court. 

...and it doesn't stop there, your company might not even be directly involved in the court case but could be dragged in as a third party via a subpoena.

No matter how lawfully YOU run your business, not having adequate email retention is a business risk that you simply can't afford to take.

Why A Restore is Not Enough

From an IT point of view, we don't usually talk about legal things.  We're all about Backups and Restores (and recoverability) as if it's simply a matter of getting missing data back.

The theory, for example being that if a user loses their file today and they've not worked on it for a couple of weeks, then any backup from the last few weeks is sufficient.

In this case, the intention is simply to recover the lost data.

What we're finding though is that recoverability is much more than simple data restoration. What if, instead of simply recovering the data, we had to prove that no changes had occurred in it between the recovered date and the date it was deleted.  The only way to do that would be to recover all versions of this (or to have unquestioned data tracking enabled).

Email is very much a dynamic kind of "file".  For example, you might be able to recover a mail from July 7 which was deleted on July 20, via the Backup from July 15, but that doesn't mean that someone didn't reply to that message on July 16 and then delete the reply along with the original message on July 20.

Mail Journaling

There's only one sure way to demonstrate that you've effectively captured all email;

Have a copy of every single inbound, outbound and internal mail copied to mail storage which does not permit deletion - and retain that mail for the appropriate legal period (not necessarily 7 years) - even if the employee in question has left the company. 


Have auditing facilities in place to protect the mail stores from administrator intervention or unauthorised access and, have a monitoring process watching the store-process to ensure that it doesn't stop.

In our case, we've been using the Veritas solution... but now we've discovered that moving to IBM Verse will prevent us from being able to journal purely internal mail.

Why the Cloud Systems are failing us

In the past, when we had our own mail servers on-site, we could direct outbound SMTP traffic to go via our external archiving partners, our inbound mail could be captured via redirected MX records and our purely internal mail could be captured via Journaling.

With the cloud services, attempting to provide a one-size-fits-all solution, these options are not necessarily available to us. In our case, with IBM Verse, we've been able to sort out inbound and outbound mail mail via the traditional means (after a bit of fiddling) but it turns out that there's no way to journal purely internal mail to an external system (so much for open systems).

We have to abandon our archive solution and go for IBM's offering -- except, of course, that we can't really abandon our old solution because we need to keep it going, possibly indefinitely... unless we migrate it elsewhere (See the Chart at the end of this post).

I've looked at Microsoft and Google and they seem to have the same problems. Their products don't seem to support external journaling any more (or they're in the process of depreciating them).

I've also noticed that since we're using cloud services, it's no longer possible to restore mail (after the trash has been emptied).  This too is a feature of the three cloud services I looked at.

One thing is certain - If you're looking to put your email in the cloud you MUST subscribe to the cloud mail retention service from the SAME vendor.... and, the choices you make today could be the choices you continue to pay for well after you've migrated to a competitor's system. 

Recommended Reading and thinking

The whole Email Retention thing pretty much kicked off in 2002 with the Sarbanes-Oxley Act in the US.  Most of the western world now has an equivalent act in place. If you're not up on that, it's good reading.

The whole Hillary Clinton thing is worth reading too - it's a bit wider than simply mail preservation but it's a good example of the rules around email in action.

There are lots of free whitepapers around on Email Retention. Just do a google search and click on some of the PDFs that come up.

Thinking more widely, we need to be prepared for the next leap in litigation; At some point, the courts are going to start asking people to produce records of instant messaging, posts and comments on collaboration platforms.

Do your staff leaving processes leave their collaborative data intact and allocated to the original owner?  How do you handle "deleted comments"?

How Long do we need to Retain Email?

This excellent chart is from Contural Inc's excellent 2007 Whitepaper: How Long Should Email be Saved?  It was sponsored by Symantec who have since moved the business to Veritas.  The chart shows that different types of emails have different retention times.

Thursday, July 07, 2016

Copy as Table works (in One Direction) for IBM Notes/Verse Interactions

Copy as table used to be one of my favourite IBM Notes features. We have a lot of databases full of documents, news stories etc. We also have an office which is "entirely migrated to" but not entirely using Verse. 

Some people just won't let go of the Notes interface - we're working on that problem. 

One of our databases contains news stories, the links for which we regularly send out to the rest of the organisation.

We quickly discovered that the Verse users couldn't open the links. This is because the doclinks had our Notes Server names (eg: http://internalserver.ournetwork.local) as http, instead of Notes protocol; (Notes://internalserver).

This morning I discovered a great update to Verse.  I don't know when exactly IBM did it, but I'm very grateful.

To Copy Documents as a Table

  1. Go to a Notes database (in the Notes Client) and select a bunch of documents 
  2. Use Right Mouse click, Copy as Table, 
  3. They're now on the clipboard. 
If you paste these into a new email via the Notes client, then ONLY notes users will be able to use them. 

If you paste these into a new email via the VERSE client, then clients with either Notes or Verse will be able to use them. 

Always use the Verse client... even if your people aren't on verse yet.  One day they will be ... and wouldn't it be nice if the old emails still worked. 

Copy as Table - in Action

For added fun, in case you don't want a table in your email, you can try merging the cells in verse and then re-copying the data out of the table into another part of verse (and deleting the original table).

One Final Caution

Since the links are Notes:// links, they won't work on mobile versions of Verse - at least not yet.  Not until IBM provides us with the mobile app for ICAA - IBM® Client Application Access (formerly known as IBM Notes® Browser Plug-in).

(hint, hint IBM).