Avoid bumpy rides with Exchange Online

I’m posting from a hotel room in Hartford, CT, getting ready for a meeting with a client tomorrow.  Getting here reminded me of why you might want to choose to not try to go it alone when you set up Exchange Server for your organization.  More details on the trip later.

One of the biggest problems I see clients facing when they deploy Exchange Server is the simple fact that it’s a huge software package.  In reality, it’s more of a suite of tools than one application, and therefore requires either pre-existing expertise, or a very fast ramp-up to be able to properly install, manage and maintain the solution set.  Luckily, there are options today that were simply not available in the past, namely Hosted Exchange Services (a.k.a.. Exchange Online).

Hosted Exchange is – at its heart – renting space on a shared or dedicated Exchange Server that is managed and maintained by a 3rd party.  These are hosting providers vetted by Microsoft (or at least they should be) and skilled in creating, deploying and managing Exchange Server solutions. No hardware on-site, and no major management needed by your own tech people.  So what are the pros and cons of using a solution like this.

The major con is that you have to keep your data on another server system that doesn’t reside within your firewall and security network.  Though the reality is that anything which requires connectivity to the outside world is technically open to attempted attack, some companies may not be able to tolerate hosted data.  Either corporate or external compliance policies and laws may prohibit this, so be sure to check with your compliance staff to make sure it’s ok before you ship the email off-site.

Barring compliance issues, security may still be a concern.  Make sure you completely check out your hosting provider, make sure they are a Microsoft Partner and make sure to get references before you sign over your digital life.  It’s totally fair to put the hard questions to your hosting provider, as you’re trusting them with a tremendous amount of sensitive data.

Beyond those two potential cons, the pros are pretty deep.  Many small to mid-sized organizations may attempt to host an Exchange Server in their own network.  Due to inexperience or just plain lack of time, patches are missed, settings are mis-configured, security concerns are overlooked.  This leads to either a poorly functioning Exchange system, or worse, an attack against the organization using the messaging platform as the attack vector.  Larger organizations who have dedicated messaging staff can afford to have people who look after these details, but smaller shops can’t, and often the company infrastructure suffers because of it.  I’ve discussed a lot of potential pitfalls to running an Exchange environment – this is a well-rounded way beyond most of them.

As a matter of fact, when talking with Jon Orton – one of the TPM’s that works with Exchange Online for Microsoft – he related to me that “we’re seeing more and more people, especially in the “small Exchange deployment, not much expertise” category, signing up for Exchange Online and partner-hosted Exchange.”  By placing the infrastructure in the hands of a team of experienced Exchange engineers, you remove that burden from your IT staffers.  Don’t get me wrong, I believe anyone can indeed learn to manage an Exchange Server messaging and collaboration environment However, if you do not need to, then this is one thing you can get off your plate and off your mind – safely.

So, even though it might be very tempting to set up Exchange Server within your network, the facts that it is readily available for purchase and that server hardware is getting cheaper should not be the major factors in decision.  Hosted Exchange Services with a trusted provider are easier to work with and just as readily available, and with a little legwork ahead of time, will be a better solution for many small and mid-sized organizations.

Which leads me to my harrowing journey today.  When considering how to get to Hartford from my native New York City (Queens, to be exact),  I had a few choices.  All were pretty easy.  I could take a train to a mid-way city and hitch a ride with a co-worker going to the same meeting from there.  I could take a different train and then a quick shuttle here.  But, because it was readily available and easy to set up, I chose to take a bus directly out to Hartford.  Much as with the decision to leap before you look and set up an Exchange Infrastructure on-site, I nabbed the tickets and got on the bus without a lot of pre-existing experience with the bus route.  After getting bounced around the bus for about 3 hours, and nearly losing the sandwich I had grabbed at the terminal, I finally got off the bus from hell and collapsed into the hotel room.  Had I researched my options a little better – and sought out some experience from others – I would have realized that a little more legwork would have led to a much better experience for me.

So do a little legwork before you put in your Exchange Server.  Know what you’re getting into and know that there are great options for getting out of the situation.  Exchange Online with a trusted provider can offer a convenient, safe and workable option for those who are just not quite ready to roll their own.

Update now - no I mean it, NOW

Though I'm known for keeping a level head during these kinds of scenarios, there is an emerging vulnerability in both Exchange Server and Internet Explorer that you'll have to address immediately to keep your enterprise safe.  I wish I was kidding, but no, it's true.

The long and short of it is that specially crafted messages could be sent to an Exchange server that - even if only previewed by a user - can allow an attacker to take control of the Exchange server itself with full Exchange Admin privileges.  The details can be found at this link, but I'll summarize here.

If an attacker crafts a message in certain ways via TNEF (Transport Neutral Encapsulation Format), and that message is then either previewed or opened by an end-user (which is the point of email after all), then the attacker can gain elevated privileges on the Exchange server and wreak havoc. While that sounds like a major problem, there is good news.  The attack would require a sophisticated attacker creating just the right message and then finding a live mailbox to send it to.  Not only that, but it would need to get past the SPAM and CAPTCHA systems if you use them.  Finally, eventually the anti-virus vendors will jump in as well, scanning for messages crafted with the exploit.

The really bad news is that end-users do NOT have to open an attachment this time.  They can unwittingly infect the mail servers by previewing the message body or opening the message itself.  Since Outlook clients and now even Outlook Web Access both Auto-Preview by default, end-users can follow all best-practice safety protocols and still end up infecting the organization through no fault of their own.  Add to this that some anti-virus tools may use MAPI viewers to scan mail, which means that the system designed to shield you could accidentally infect you instead.  The same goes for backup agents that use MAPI clients to do brick-level backups.

There is one more bit of scariness to be told, though information on this last peice is more sketchy than the other exploit vectors. It would appear that the TNEF message may not need to be even previewed by an end user, but could in fact attack a server just be being received by the server itself.  Even until we know more for sure on this last point, I think the previous points - which are verified by this author and others - are bad enough that you should jump on this right now.

Microsoft has released a patch - also available via this same link, under "Security Update Deployment" and so you can protect yourself and your organization.  I strongly urge you all to do this immediately, and to have your end-users run Windows Update (Start|Windows Update on Vista, Microsoft Windows Update Online for everyone else) to get critical Internet Explorer patches as well.

Now, this is not only your Exchange server that you are protecting.  If an attacker gains privileges on your mail server, they can use it as a launching pad for attacking other servers with the same exploit or any other they'd care to try.  That means that if you don't patch your systems, you could inadvertently be responsible for attacks against other servers all over the net.

And so, I am asking my readers to protect the community at large as much as they can, by spending the time necessary to download and install the appropriate patches.  Yes, this will mean a maintenance window for a reboot.  Yes, it's annoying that we've gotten hit with yet another critical vulnerability.  But, yes, it is our responsibility to make sure that the attacks stop at our own borders, and that our own mail servers do not become the launching point for the next Code Red.

Update on Wildcard certificates

Not that long ago I talked about using wildcard certificates to allow you to move OWA and ActiveSync services from one physical server to another.  Since single certificates are assigned to a single server, failing over or moving to another server would cause the clients to suddenly lose SSL connectivity, as the certificate would not match up, and ActiveSync devices cannot pop up the error about a non-secure connection. OWA can, but it can be troublesome with end-users who suddenly start seeing security warnings.

Following up on this theory of using a domain-assigned wildcard certificate, research has shown that older Windows Mobile devices (WM 5 or earlier) cannot leverage these types of certificates at all.  WM 6 and higher can leverage this technology, but earlier versions were not coded with the required information to recognize that a certificate could possibly be assigned to more than one physical server or networked device.

So, for those using WM 6, OWA and Outlook Anywhere, you can use wildcard certificates to allow for services to move from server to server as required.  For those using earlier versions of WM (or the 3G iPhone - though the jury is still out on that one), you must use server-specific certificates, and re-configure the devices' ActiveSync connection if the server itself moves. 

None of this impacts Blackberries, as they authenticate via the RIM network, and not directly to the Exchange servers.  If you're using Blackberries, wildcard certificates offer the ability for all other mobile systems (OWA, Outlook Anywhere, etc) to move with your servers in the event of a loss of a particular physical machine, while RIM will handle moving the Blackberry devices.

Long story short, if you're on WM 5 or earlier, be ready for a few support calls when you need to move the services or fail over between servers - even if you use wildcard certificates.  If your users are on the new iPhone systems, be sure to keep a close watch on Apple's forums, as new information is being discovered every day.

Get wild with wildcard certificates

Most organizations that are considering securing their email systems are also looking at SSL certificates to aid in that effort.  Security certificates allow for encrypted communications using Outlook Web Access, Outlook Anywhere (RPC over HTTPS) and ActiveSync devices like Windows Mobile Smartphones. The issue is that a normal SSL certificate is bound to a single server, which means you'd have to obtain a signed certificate from a provider for each server you want to put into the solution.

You could use self-signed certificates to get around the issue of having to purchase individual certificates, but only at a cost.  Many applications will not accept self-signed certificates, and even Microsoft-controlled systems like OWA will kick up several layers of security errors - which will increase support headaches for you and your staff.

Alternately, you can obtain a Wildcard Certificate from a provider, which will allow you to use one certificate for multiple servers in your organization.  These certificates are more expensive than single-server options, but can give you much more flexibility with things like server rebuilds, migrations, and failover.  The only restriction is that the servers will have to be part of the same organization. That means that all the servers will have to be part of the same domain and/or forest in the sense of Active Directory.

Wildcard certificates can be defined at either the organizational level - such as *.wildcarddomain.com - or masked at another level, www*.wildcarddomain.com for example.  With the first example, all server identities that are part of wilddcarddomain.com will be able to share the certificate.  In the second, www1, www2 and even wwweb.wildcarddomain.com will be able to share the masked certificate.  Which you use will depend on the depth of security you are looking to implement, as more specific masks allow for less chance of someone spoofing the certificate for a foreign machine.

Once you install the wildcard certificates, you can allow services to move between servers within the same organization without breaking security. This can be extremely helpful for both migrations and disaster recovery, as you can have both of the servers shielded under the same certificate, and therefore you will not get naming errors when the users move or fail over. 

In a worst-case scenario, you can use single-server certificates on a migration or failover target, but your end users will get errors about the certificate being valid, but assigned to a different server name than the one they are connecting to.  You can then purchase another certificate after the fact and assign it to the target server to remove the error if/when needed.  However, wildcard certificates give a much better option for flexibility to you and your team.