S/MIME gateways can create fatal security breakdowns

Over the years, I’ve had several clients use S/MIME to authenticate and encrypt e-mail messages. Unfortunately, encrypting anything end-to-end has problems, including those associated with scanning incoming encrypted messages, checking for data leaks, or indexing for later retrieval. When my clients turn on S/MIME, they are pretty much turning off easy e-mail scanning and retrieval.

Two weeks ago, I asked readers and vendors to submit solutions to the S/MIME problem of anti-malware scanning, indexing, and encrypted message retrieval. You didn’t let me down. In fact, I got so many responses that I felt kind of dumb. But that’s what I love about the computer security world: It’s too big to be an expert in everything.

[ Check out the InfoWorld Test Center’s roundup of e-mail security services. | Learn why securing e-mail but not large file transfers is a bad idea. ]

I knew from the start that the solution I envisioned was some sort of e-mail gateway that could handle the user’s PKI asymmetric keys, performing the desired service before encrypting or decrypting the e-mail and sending it on its way. Essentially, the pathway would look something like this:

S/MIME E-mail Client Anti-malware ScannerArchiverData Leak Prevention Encryption Gateway Internet Other Recipient(s)

The key question is, how would the S/MIME client and encryption gateway interact? Would the S/MIME client even know there was a change? Would the sender be required to share their S/MIME private key with the encryption gateway? How would it handle recipients on the other side?

Martijn Brinkers, author of Djigzo open source e-mail encryption gateway, sent in his solution. Djigzo can function like a normal e-mail server (called an MTA, mail transfer agent), but it can also participate in S/MIME and PDF encryption. Although the user’s existing digital certificate can be imported into Djigzo, the MTA contains its own PKI digital certificate server. You can create and send new PKI digital certificates to both internal and external users, who can then install them locally so that their e-mail clients can participate in S/MIME. Djigzo supports many clients, including Outlook, Gmail, Applemail, Thunderbird, and Lotus Notes. It has fairly good documentation with lots of step-by-step screenshots.

As expected, Djigzo will automatically decrypt incoming messages being sent to internal users. Then the unencrypted message can be scanned, archived, or checked for data leaks.

After decrypting, Djigzo adds headers to the message to indicate whether it was signed or encrypted successfully from the sender. These headers can be checked by the end-user or the client to see if the message was securely delivered to Djigzo. A Djigzo-added header might look something like this:

XDjigzoInfoSignerID01: CN=UTN-USERFirstClient-Authentication and Email, OU=http://www.bannertrust.com, O=The Banneret Root Network, L=Virginia Beach, ST=VA, C=US/778F9874B02A51042E0148D78CCDE1785/ DjigzoInfoSignerVerified01: True

This part of the solution bothered me because by replacing the S/MIME protection with some text-based headers, it essentially prevents the client from doing all the thorough S/MIME tests it would normally perform. On the other hand, it’s completely expected, because I’m asking it to get in the way of the typical encryption pathway.

Sophos has a similar product called the SafeGuard MailGateway. Like Djigzo, SafeGuard supports S/MIME and encrypted PDFs, and it has an integrated PKI certification authority service. And like Djigzo, it offers an encryption solution for non-S/MIME users, using encrypted PDFs or OpenPGP. The Sophos product, as one would expect of commercial software, has a lot more functionality than Djigzo, it supports more formats, and it has superior integration (such as Active Directory support). Its use from both the sender and receiver side is more seamless and professional-looking. As much as I liked Djigzo at first review, I liked the Sophos solution even more.

So why did I have a strange feeling in the pit of my stomach that I was going to have a hard time recommending either one of them? Here were two good solutions to my problem, but I wasn’t sure if I was violating some inherent crypto law by purposefully bypassing the security protections that S/MIME was designed to implement.

As much as I like the idea of S/MIME gateways, their very nature as an intermediate MTA gateway with the user’s private decryption keys means multiple fatal security breakdowns. For one, private keys are meant to be private. Only one person should have them. Add in an S/MIME gateway and that private key exists in at least two places, while the S/MIME gateway administrator becomes a crypto god. Second, the use of headers means the client’s e-mail software cannot be used to do the normal, more thorough, S/MIME security inspection. Without the normal S/MIME encapsulation, how is the e-mail client going to alert the user to a change?

The third and most dire consequence: The S/MIME encryption is no longer end-to-end. Stripping off the crypto before it gets to the client means that malicious things have the ability to tamper with the message before it gets to the final client — not good. What other controls could the e-mail admin add to make sure there wasn’t some sort of mischievousness in the pipeline? None, really. Crypto Ph.D.s around the world are probably having a coronary thinking that anyone would consider such a product. “Why not just give away your house keys and bank account numbers!” they’re probably saying. “Why use S/MIME at all if you’re going to yank away half its protections in the first place?” I told myself.

Then another vendor offered up a S/MIME hybrid encryption approach that appealed to me even more, enough that I could recommend it (and other solutions I’ve seen like it but had forgotten about). The vendor in this case was EchoWorx and its Encrypted Mail Gateway. Here’s how EchoWorx describes the process:

[T]he user composes and sends the message as they normally would. The message is passed through existing e-mail hygiene services such as anti-virus, anti-spam, and content scanning. Pre-defined policies based on who the message is from, who the recipients are, and content in the subject and body determine if the message should be encrypted. If the message does require encryption, it is sent securely to the SMG [secure mail gateway]. Relying on transparent key look-up and exchange functions, SMG automatically determines which users have credentials and which don’t.

Users with standard S/MIME certificates (e.g. encrypted mail users) receive the encrypted message directly to their inbox. Non-subscribers receive a digitally signed notification message with a link to the Message Pickup Center Web site. External users can then click on the link and retrieve the message via a secure SSL/TLS tunnel.

Strangely, even though EchoWorx’s solution suffers from some of the same crypto concerns as above, it offers a few more features that eventually overcame my shyness. First, I like that EchoWorx works pretty seamlessly for users without S/MIME keys. Users click on a link and the e-mail is delivered fairly secure anyway. It’s not S/MIME, but it’s protection and simple-stupid. Second, it includes a policy inspection engine that enforces S/MIME on messages that should be encrypted, based upon some part of the message (such as the subject, sender, contained content, or recipient’s name).

I don’t know about you, but when I’m doing S/MIME, I tend to make mistakes. Every now and then, I forget to encrypt messages to people that I should. Most S/MIME clients don’t make it easy to create rules that enable S/MIME for some recipients and not for others. It’s either all or nothing, or it’s manual. And when it’s manual, people make mistakes. So I like that EchoWorx takes care of the policy issue as well. Solving that problem alone is enough to make me overlook the other end-to-end problems.

I want to thank all the readers and vendors who replied to my request for help. I think all three vendors have good solutions that are perfect for particular situations.

This story, “S/MIME gateways can create fatal security breakdowns,” was originally published at InfoWorld.com. Follow the latest developments in information security and encryption at InfoWorld.com.