In February 1999 I wrote a magazine article entitled, "Free Lunch" that focused on the Advanced Encryption Standard (AES) selection process and the need to avoid proprietary cryptographic algorithms.
The uniqueness of the entire AES process was that it was akin to a large town meeting -- the process was open to the public and anyone could have submitted an algorithm for review.
One of the complaints against DES (data encryption standard), the standard AES was replacing, was that the NSA clandestinely hid a backdoor in the algorithm. To downgrade all conspiracy theories, the NSA decided not to submit any algorithms to NIST during the AES selection process. Rather, they acted as an impartial arbitrator in the process. Notwithstanding, it is implausible that NIST would have approved anything without the divine sanction of the NSA. The foremost effect of AES is that it clearly demonstrated that the entire encryption and security industry started to favor public-based algorithms.
Whoever said that there is no such thing as a free lunch never had in mind encryption algorithms. The paradoxical issue about encryption algorithms is that their true strength is only manifest after extensive and critical open peer review.
The specifics are known as Kerckhoffs' Principle, after Auguste Kerckhoffs, who observed in 1883 that a cryptosystem should be secure even if everything about the system, except the key, is public knowledge.
Using Kerckhoffs principle, every organization that is using encryption functionality, be it in hardware or software, should make the first rule of their encryption selection to avoid any software that uses a secret algorithm. It should be built into RFPs, contracts and the like. Keeping an algorithm concealed is no proof of safety. Vendors that refuse to reveal their algorithms should be dropped.
There hardly seems to be a valid reason why any reputable security vendor in 1999, let alone in 2010, would waste their time developing a proprietary algorithm when there are so many efficient and capable publicly-available algorithms. Whether it be Triple-DES, Twofish, SERPENT, CAST 256 or IDEA, or any other open encryption algorithms -- any vendor that has something to hide in their algorithm should be questioned.
Since the encryption algorithm is the foundation of most security products, a vendor should be proud to show you their algorithm. The reality is, for those who adamantly refuse to reveal their algorithm, one can reverse engineer the software code (albeit this may be illegal in some jurisdictions) to reveal the guts of it.
According to Benjamin Jun, vice president of technology at Cryptography Research, there is much to fear with unreviewed algorithms. "While algorithm failure modes are often subtle, they are nearly always catastrophic. Good cryptosystem designers know that any new system is guilty until proven innocent. Therefore, companies considering encryption functionality should always use widely peer-reviewed cryptographic algorithms, unless absolutely necessary." Jun added that it is an imperative for all involved to thoroughly document all of the engineering design, in order for the system to undergo intense scrutiny and review.
Unfortunately, vendors still think that proprietary cryptography is good. I ran into an example at Interop New York in November. ENC Security Systems had a booth promoting their Encrypt-Stick encryption software system that is used to protect data on USB flash drives. In the glossy handouts, and on its website, ENC states that the Encrypt-Stick uses 512 Bit Polymorphic Encryption from PMC-Ciphers Inc.
In a recent e-mail exchange with the vice president of international business development and IT for Onix International, the Ontario, Canada worldwide distributor of the EncryptStick software application, he replied that "based on past history, open source encryption has always been hacked."
While every algorithm has had it 20 years of fame, it is true that they will eventually be hacked. Even the venerable RSA algorithm, considered effective as of January 2010, is developed on the fact that huge numbers are difficult to factor. Should a method for factoring large numbers be developed, or a production quantum computer made, RSA would indeed be obsolete. But its true strength is based on years of openness and peer review.
The gentleman at Onix International was honest enough to admit that he is not a cryptologist and doesn't profess to be one. But he also stated that "I still feel in my heart that if you have a solid algorithm, private is the way to go." What he is missing is that what one feels in their heart is immaterial within the world of cryptographic algorithms. In fact, authors of code and owners of systems are often oblivious of their products' systemic failings -- which illustrates the need for open-source cryptographic algorithms and public peer review of those algorithms.
Does that mean there is zero use for proprietary ciphers? For the most part, yes. But Jun does note that in some tamper-resistant security modules, they are architected in such a way that data passes through via a standard cipher and a proprietary cipher. In that case, the proprietary cipher is treated as an algorithmic obfuscation tool, so that a cloning attack needs to do more than just extract the AES key. He said that there are a few applications where this is done, but it needs to be done very carefully.
In a world where Intel processors are obsolete after a few years, Kerckhoffs' Principle is completely relevant more than 125 years after he first stated it.
While many caution that there's no such thing as a free lunch, there is indeed a free lunch after all, in the form of open encryption algorithms. Should you want to bypass that free lunch, caveat emptor to you, your data, and all your customers.
Ben Rothke, CISSP, QSA (firstname.lastname@example.org) is a security consultant with BT Professional Services and the author of "Computer Security: 20 Things Every Employee Should Know."