Tuesday, April 04, 2017

Migrating Mail from IBM Notes and Verse to Microsoft Outlook on Office 365 - Part 2

Last time on Real World Computing, I talked about migrating mail from IBM Notes and Verse to Microsoft Office 365. Now it's time for Part 2. 

Mail Routing

We did routing in two parts. Initially we had MX records for both IBM and Microsoft with Microsoft having the higher number (which means lower priority). After the cutover date we switched the priorities so that Microsoft Office 365 had the higher priority.

One of the cool things about Microsoft’s setup is that they give you two domains, one is your own and the other is an @mycompany.onmicrosoft.com one.

When we first saw this we thought it was a “bit wanky” but as it turned out, it was very useful indeed.we quickly discovered that we couldn't send mail to our internal colleagues on outlook from notes and that all of the agents in our Domino applications were only delivering internally.

Changing these to point to the onmicrosoft addresses fixed that problem. As far internal mail, we just added a mail rule to forward all new mail from our personal Domino and verse mailboxes to onmicrosoft.

Mail Migration

We tried a few things to get our old mail migrated. We had originally hoped that we could simply have a cutover date with a small amount of overlap but the reality is that other departments use mail quite differently from IT and they rely heavily on foldering and calendaring. They use search and archive very lightly - and it's not something that can be changed in the short term, regardless of how good the technology is.

We first tried putting their Verse mail into outlook and then copying and pasting mail between the mailboxes. This worked very well but we quickly discovered that there was a 90 day limit in terms of mail that Verse makes available to outlook.

The next phase involved taking an unencrypted ACL-free copy of users mail files and using conversion applications. We tried two and they both gave us a lot of trouble.

Stellar NSF to PST Converter

The first product we tried, Stellar NSF to PST Converter, had some very restrictive licensing. It was expensive and we could only run it on one PC. The trial version worked fairly well  but it was limited to only a very small amount of conversion.

The advertising claimed that it worked with Office 2016 but the reality was that we eventually had to run it with Office 2010.

Our initial attempts to get a NSF mail file converted to the 365 cloud took 24 hours for a single user with less than 3 months worth of mail. We complained about the software and after threatening to pursue the issue of a refund, the Stellar technical team worked with us on the problem for a couple of days.

Kernel Lotus Notes to Outlook Conversion

In the meantime, we bought another migration package; Kernel Lotus Notes to Outlook Conversion. This one had much better licensing and could be run on multiple machines at once. It worked out of the box and was able to do the same mail file in an hour.

We were ready to go with Kernel when suddenly the Stellar product started working. It processed our test file in 30 minutes. In the end, we used a mix of the products.

The conversion process produced a PST file which we were able to import into outlook. It was interesting to note that the folders and the calendar entries came over well and we had relatively few complaints from user about the conversion. Most of the complaints were actually outlook usability issues.

One thing that was a little problematic was that we had used an archiving and compliance solution, MailAbility. This software moved some of the older mail to external NSF archives and simply left links in the Notes mail client. Obviously those mails weren't “real” emails and couldn't be imported. We’ll convert the archives later and convert them to a “locked down” compliance-only shared mailbox.


The final big problem was groups. We currently need to retain the groups in both systems but duplicating them makes no sense. We've looked around for ways to easily synchronise the groups from Domino to Office 365 but apart from one-time migration or some really dangerous scripting, we've found nothing that will do the job.

The other important thing to realise is that Microsoft is redefining groups. There's a new group type called an "Office 365 Group" which everyone should probably be using.  Unfortunately, at the moment, it doesn't support people outside of the organisation...

...apparently that feature is coming "real soon" though.

In the meantime, groups are easily created via outlook or via the admin console or, as is apparently the preference (which doesn't currently support external parties), via Yammer.   As for getting contacts across, there's the solution I discussed back in January.


Anonymous said...

Thanks for creating this series. We're about to do the same so it's really useful learning some of the pitfalls nobody tells you about from others.

Did you trial any of the co-existence tools such as Binary Tree's CMT or Quest's Coexistence Manager?

Gavin Bollard said...

We considered co-existence tools but we didn't really need them. We are in fact co-existing in terms of groups because we need them for standard domino security.

It's possible to co-exist just using the onmicrosoft domain and the DNS, and a little tweaking of the domino mail domain.

I should probably write that up at some point.

Pavel Bludov said...

Hi Everybody!
@Gavin Thanks for spending your time and sharing your experience.

I'll describe which tools we have tried so far, which worked better, and which not so much.

First of all, I found the link down below really helpful even though it doesn't cover all the things in details, but it gives an idea that you can make O365 and Domino On-prem co-exist.
So as the result, two systems share the same domain (there is no need to use @onmicrosoft.com). Messages route without issues from on-prem to the cloud and vice versa. The only condition is a mailbox has to be either in Domino OR in O365. And yes, both directories need to have the same data (groups, lists, users). Of course, that coexistence doesn't include Free/Busy lookup. At this point a solution like "Quest Coexistence Manager" might be handy.

Next, every company has to decide how old email / calendar will be available to users. It can be as simple as having a local PST file mapped to Outlook client.
Now, how do you get those PST files? I've tried different products. Systools NSF to PST can convert lotus mailboxes to PST files. Though in the past I experienced some data loss during the conversion of a big mail box (~5% of documents were not converted out of 35k mail box). That company has another tool that allows you to convert your Domino mailbox into Office 365 mailbox on the fly. It works and combines all the steps into just 1, though with a trial tool I couldn't verify if my calendar events got moved properly. Plus I saw some Domino Canonical Names in the messages. The bottom line, in my opinion (only) you might expect some artifacts.

The best way (BUT not the fastest or easiest) and the most accurate would be using IMSMO as a middle step. Very likely my company will have that route chosen.
The process would be:
1) Create another Domino server, create all mailbox replicas there, configure IMSMO
2) Configure multiple VMs with Outlook clients (plus multiple email profiles) to cover every user. All that to have *.ims.
Note: some limitations have to be removed - enable import/export for Outlook, set no limits to number of messages downloaded, disable truncation of messages.
3) Convert IMS to PST
4) Upload PST to Azure Cloud using AzureCopy
5) Create CSV that map each mailbox to O365 user.
6) Use MS portal to import data (multiple different options are available).
All that will allow us to make a switch within a weekend. All data looks accurate as IMSMO makes it very polished and UI ready.

Now I have a question myself, which if anyone knows the answer to, would be great!
Conditions: We have AD synced to O365. We also extended AD with Exchange 2016 schema to get those extra attributes we had been missing because of not having Exchange On-prem. All users are in O365 - some with licenses, some without. All users have those main attributes like email address, displayname etc. Users can be looked up in Skype for Business or any other O365 App EXCEPT for Outlook/Exchange Online.
Now the Question: what values in what attributes in AD are we missing to enable Lookup in Exchange Online?

Sorry for my message being a wall of text, I guess it's not a process to be explained in one message.
Pavel / Paul

Ben Langhinrichs said...

Gavin- I'm curious how large an NSF mail file you were migrating in your example. As you might know, we've recently released CoexLinks Migrate, and I'm interested in even anecdotal evidence about how fast other solutions are. Our specialization is on high fidelity rendering, obviously, but I have little idea how fast/slow other solutions are. I know on my PC, 17500 emails take about 8-9 minutes to export.

Our next step is to make sure the process of migrating to Office365 is seamless, but for now, I just want to be sure we are in the right ballpark. - Ben

Pavel Bludov said...

Does anyone know how to migrate Rooms and their reservations from Domino to O365/Exchange? There is no surprise that when I migrate mailboxes to O365, reservations stay and are only in people's calendars.
Maybe there is a way to move those into a mailbox first and then convert.