Monday, April 24, 2017

Using SharePoint with OneDrive as a File Server (for Ex-Domino Admins and Traditionalists)

Over the past few months, I've been looking at a whole range of options to do with file storage on the basis that Microsoft's OneDrive simply doesn't do what we need. The whole time of course, I've been unable to shake the feeling that Microsoft should be offering something that already covers this space. After all, file sharing is one of the major "tentpoles" in most Windows networks. 

As it turns out, SharePoint is the answer to this - and it works well if it's playing nicely with OneDrive. 

My initial investigation of SharePoint was flawed for a number of reasons. Firstly, it appears that I was looking at an "old version".  The "new" version has really only started to come out over the last few months but it's light years ahead of its predecessor.

The second reason that SharePoint was overlooked was because I really didn't have a great understanding of how (or why) it works. I was trying to compare it to IBM Domino and IBM Connections. There's a lot of similarities, particularly, with connections but they're vastly different beasts.

SharePoint Security differs from Domino Security

The security model of SharePoint is actually the opposite of Domino. In Domino, you start off with reasonably "open" access controls. For example, the server is open to "everyone in the company". From there, you restrict access to specific databases.  Within each of these databases, you further restrict access to views, forms, controls and documents; firstly via roles and then later via reader and editor fields.

SharePoint seems to be the opposite. Systems start out with a specific set of restrictions and then suddenly, even if you're halfway down the file structure tree, you might decide to share a particular folder and all sub-folders with someone who didn't previously have access. In fact, you can take it a step further an share with someone entirely outside of the company.

In SharePoint it is much, much easier to grant access to people however I'd venture to say that the security model it uses is by no means as "safe" as Domino.  

In Domino, if you wanted to suddenly revoke access for someone at a database level, you could simply remove them from the ACL.  In SharePoint, it's potentially a lot more complicated. As administrators, it's important to understand these differences because the SharePoint security model is also "mostly opposite" to the way the standard file server security works in Windows.

The Explorer Interface is Gone

If you talk to the Microsoft support team, they'll tell you how to map a drive from Windows file explorer to SharePoint. It works but in order to do it properly, you need to use Microsoft Internet Explorer ... not Edge ... nope, only the older technology will do.

I asked about the plans for edge but clearly there are no plans. It's not something that Microsoft wants to support. This was quite a shock to me after the amazing level of integration offered by IBM Connections.

I guess the important "take-away" from this is that Microsoft feels that the Windows File Explorer interface needs to "die".  Everything will be web from here on. 

Plan Ahead

In the old days, you created a file server by dumping all of your files into folders and sharing various bits out. It was a fairly forgiving process that enabled you to fiddle about with files until you got them right. SharePoint today is not quite so forgiving and moving things about will generally break links and muck up shared connections. You need to plan ahead.

Logical Ownership First

In the first instance, separate your files via logical ownership, which generally means "departments".  For example, most companies will have files in at least the following business areas;
  • Administration
  • Finance
  • Information Technology
  • Human Resources
  • Sales and Marketing
  • Strategy

Unless your company is a very small one (under about 30 employees), you'd be best off creating a SharePoint site for each of these major areas. This will make overall security a whole lot easier.

Smaller Chunks Second

The other thing to be aware of is that there seem to be some fairly serious limitations on the way that OneDrive syncs SharePoint data. In particular, there's an upper limit on both the number and the total size of the files that can be synched.

I'm not reiterating the limits in this post although I've seen them stated in several other places. There are two reasons for this;

1. They are subject to change
2. I've seen OneDrive behave poorly with much lower limits.

The file limitation would not normally be a problem except that OneDrive is currently incapable of doing a "partial synch".  It tries to synchronise the entire library and will dummy spit if it's too large.

For this reason, you need to add multiple document libraries to your SharePoint site.

Next Time....

I'll discuss this in my next post where I assume that like me, you've already uploaded a lot of files to SharePoint and now need to move them. Moving them is actually pretty easy and once it's done, they Sync problems are fixed. 

Tuesday, April 04, 2017

Migrating Mail from IBM Notes and Verse to Microsoft Outlook on Office 365 - Part 2

Last time on Real World Computing, I talked about migrating mail from IBM Notes and Verse to Microsoft Office 365. Now it's time for Part 2. 

Mail Routing

We did routing in two parts. Initially we had MX records for both IBM and Microsoft with Microsoft having the higher number (which means lower priority). After the cutover date we switched the priorities so that Microsoft Office 365 had the higher priority.

One of the cool things about Microsoft’s setup is that they give you two domains, one is your own and the other is an one.

When we first saw this we thought it was a “bit wanky” but as it turned out, it was very useful indeed.we quickly discovered that we couldn't send mail to our internal colleagues on outlook from notes and that all of the agents in our Domino applications were only delivering internally.

Changing these to point to the onmicrosoft addresses fixed that problem. As far internal mail, we just added a mail rule to forward all new mail from our personal Domino and verse mailboxes to onmicrosoft.

Mail Migration

We tried a few things to get our old mail migrated. We had originally hoped that we could simply have a cutover date with a small amount of overlap but the reality is that other departments use mail quite differently from IT and they rely heavily on foldering and calendaring. They use search and archive very lightly - and it's not something that can be changed in the short term, regardless of how good the technology is.

We first tried putting their Verse mail into outlook and then copying and pasting mail between the mailboxes. This worked very well but we quickly discovered that there was a 90 day limit in terms of mail that Verse makes available to outlook.

The next phase involved taking an unencrypted ACL-free copy of users mail files and using conversion applications. We tried two and they both gave us a lot of trouble.

Stellar NSF to PST Converter

The first product we tried, Stellar NSF to PST Converter, had some very restrictive licensing. It was expensive and we could only run it on one PC. The trial version worked fairly well  but it was limited to only a very small amount of conversion.

The advertising claimed that it worked with Office 2016 but the reality was that we eventually had to run it with Office 2010.

Our initial attempts to get a NSF mail file converted to the 365 cloud took 24 hours for a single user with less than 3 months worth of mail. We complained about the software and after threatening to pursue the issue of a refund, the Stellar technical team worked with us on the problem for a couple of days.

Kernel Lotus Notes to Outlook Conversion

In the meantime, we bought another migration package; Kernel Lotus Notes to Outlook Conversion. This one had much better licensing and could be run on multiple machines at once. It worked out of the box and was able to do the same mail file in an hour.

We were ready to go with Kernel when suddenly the Stellar product started working. It processed our test file in 30 minutes. In the end, we used a mix of the products.

The conversion process produced a PST file which we were able to import into outlook. It was interesting to note that the folders and the calendar entries came over well and we had relatively few complaints from user about the conversion. Most of the complaints were actually outlook usability issues.

One thing that was a little problematic was that we had used an archiving and compliance solution, MailAbility. This software moved some of the older mail to external NSF archives and simply left links in the Notes mail client. Obviously those mails weren't “real” emails and couldn't be imported. We’ll convert the archives later and convert them to a “locked down” compliance-only shared mailbox.


The final big problem was groups. We currently need to retain the groups in both systems but duplicating them makes no sense. We've looked around for ways to easily synchronise the groups from Domino to Office 365 but apart from one-time migration or some really dangerous scripting, we've found nothing that will do the job.

The other important thing to realise is that Microsoft is redefining groups. There's a new group type called an "Office 365 Group" which everyone should probably be using.  Unfortunately, at the moment, it doesn't support people outside of the organisation...

...apparently that feature is coming "real soon" though.

In the meantime, groups are easily created via outlook or via the admin console or, as is apparently the preference (which doesn't currently support external parties), via Yammer.   As for getting contacts across, there's the solution I discussed back in January.

Saturday, April 01, 2017

Migrating Mail from IBM Notes and Verse to Microsoft Outlook on Office 365 - Part 1

It was always just a matter of time. Eventually we were going to have to make the jump from IBM to Microsoft. It's not that IBMs software isn't good. It's very good. It's simply that IBM is the Beta to Microsoft's VHS. Technically the IBM product line is far superior but on the surface, IBMs poor UI will never match the incredible pull of Microsoft's polished Office 365 offerings.

We're just finishing a mail migration from IBM Notes/Verse to Microsoft Outlook, which we did entirely in-house and I thought it would be worthwhile going over the method we used.

The Before Status

Prior to the migration, all of our users had the IBM Notes client on their desktops. We had three production servers and two test/dev servers. All of our user mailboxes were on the IBM Cloud and we had a split with some users on Verse and some on Notes.

We were also running an extensive extranet with a myriad of centrally controlled expansive security options. Our address books contain in excess of 22,000 groups.


Since we'd decided to use Office 365 for word, excel and powerpoint, the migration was technically already underway with the purchase of Office 365 licensing. It should have been a simple matter of extending the licensing but as we already had a number of 365 licenses, we had to establish the right options.

This meant finding a Microsoft business partner and buying new licenses since there was no upgrade path from Office 365 Pro Plus, which doesn't include outlook to 365 E3 which does. 

We later decided that E5 would have been a better choice but funding wasn't available until later and again, there was no easy upgrade option. That's an upgrade we’ll probably do on the one year anniversary instead.

Active Directory 

Our original plans were to migrate to Microsoft’s fully cloud based active directory service and retire our three AD servers. We might still do this but in the meantime, a decision was made to retain our on-prem AD.

This change had a significant impact as connecting the AD to Azure resulted in duplicate users which took some time to resolve. The solution ended up being to delete all our existing AD users and let them get recreated by the sync process.

Unfortunately deleting them required powershell, a tool we ended up getting a bit too familiar with. It also meant that their OneDrive data and yammer posts and profiles got trashed.


The next step was to get our users familiar with outlook. To do that, we needed to set up routing.

In setting up our office 365 environment, we had to claim our domain. You do this in the Office 365 admin  portal which is surprisingly full featured.You have to verify your DNS via txt record and you're encouraged to put in a mx record that includes a spf flag.

We hadn't been using SPF prior to this but as soon as the SPF record took hold, it didn't matter that we had dual routing (with mail primarily going to our IBM accounts and a secondary MX record pointing to Office 365).

Our old connection immediately became “untrusted” and we had to quickly modify the SPF record to include our existing servers.

Stay tuned... in part 2, I'll cover how we managed routing, mailbox migration, and groups.