Good Things Fall Apart
CAVP, CMVP, FIPS 140–3, and why we need a better way to measure good cryptography with ideally less acronyms
Cryptography is critical to protecting our information online. In a pandemic-wracked world where companies and countries are being forced at gunpoint into the digital age, figuring out whether or not some cryptography is “good” enough to use for protecting secrets is a very big problem.
Unfortunately there isn’t a commonly accepted way to verify good cryptography in 2020. Outside of a US/Canadian government standard that’s relatively unknown unless you work in defense, there isn’t a commonly accepted way of verifying “strong” encryption that is globally accepted.
The CAVP: the “Hacker News” of crypto
In the western world, the US NIST’s and Canadian CSE’ Cryptographic Algorithm Verification Program (CAVP) is the primary barometer for strong encryption. The CAVP is a list of ciphers and implementations vetted secure by US and Canadian government cryptologists, in partnership with both governments’ military/national intelligence units and the academic communities focusing on cryptologic mathematics and computer science in both countries.
The CAVP covers a list of symmetric and asymmetric encryption algorithms, as well as cryptographic hash algorithms used typically for identity and message content verification. It also covers vetted implementations of these ciphers, as presumably someone could just ship a library saying it uses one of these ciphers that really just posts all of your information to some Pastebin unless that implementation was open source or otherwise externally verified by some entity like NIST/CSE to be legit.
Smashing windows vs. picking locks
Unfortunately the CAVP alone doesn’t cover one key part of implementation: key management. While a cipher could be secure, it could manage keys or other elements in a wildly insecure way. This is a problem because in the real world we typically don’t launch attacks like Alan Turing and implement novel codebreaking techniques that exploit mathematic flaws in a cipher to guess the answer — a practice known as cryptanalysis.
Instead most hackers employ a side channel attack to “go around” the math protecting encrypted messages. If codebreaking was burglarizing a house, modern hackers rarely don’t try to pick the door. They try to find a window into the house that they can smash and climb through to get inside.
Poor key management, or management of key material like initialization vectors, are like big pane glass windows for modern hackers. They make it easy to go around very complicated door locks, ensuring that anyone with a big enough brick can get inside the house.
Worse, modern adversaries easily are able to automate side channel exploits and share them online using security tool suites like Metasploit.
Real world examples of this include Aircrack-ng’s side channel attack on WPA/WPA2 encryption, the encryption used for 802.11 wireless networks. Due to flaws in how WPA handles the transmission of key material, Aircrack-ng is able to steal key material used to create session keys for logins to secured wireless networks.
This vulnerability speeds up the process of “guessing” a Wifi network’s password, ensuring that breaking encryption otherwise meant to take longer than the heat death of the universe to guess can be broken in a few hours.
“Tell me what you hate about me” —The CMVP and FIPS 140
When it comes to verifying both an algorithm and how it handles keys and key material, the western world relies on another NIST/CSE program. This program is FIPS 140, the Federal Information Processing Standard 140.
FIPS 140 is a process for verifying cryptographic modules (aka: crypto modules) for inclusion in the Cryptographic Module Verification Program (or CMVP). Crypto modules are a comprehensive measurement of both the algorithm selected by a system and how keys and key material for that algorithm are selected, created, and handled.
In theory, modules verified under FIPS 140 and later included in the CMVP both use and implement safe cryptography against current and near-term attack by well resourced adversaries. In practice? Not necessarily.
How does FIPS 140 work?
Before we don our tinfoil hats let’s dive a little deeper into FIPS 140 itself. FIPS 140 comes in several versions, each with slightly (or in the case of its most recent version very) different ways of measuring cryptographic security. Most organizations and governments have relied on FIPS 140–2 for the last decade, but the newest version of FIPS 140–3 is set to begin verifying new crypto modules for the CMVP in September 2020.
The testing for each module under FIPS relies on two key areas: a module’s architecture and criteria for that architecture under NIST SP800–140:
Module Architectures
How a cryptographic module is implemented is critical to how it is tested under FIPS 140.
For example: is a crypto module carried within the firmware of a hardware appliance like a Hardware Security Module (HSM)? Is it a purely software driven API library that leverages common system architecture? Or is it a combination of the two — a new type of module under FIPS 140–3 known a “hybrid” module? The architecture for a module determines which test requirements it is audited against.
SP800–140X Test Requirements
The SP800–140 documents are a series of testing requirements for FIPS that audit how CAVP algorithms are implemented as well as handling the generation, transmission, and protection of key and key material for those algorithms. Depending on whether a module is a specific architecture, these criteria are interpreted and tested differently.
One final criteria is used for auditing a cyrpto module under FIPS 140: that module’s security level. In FIPS 140 there are four levels of security that measure how that crypto module responds to tampering — an attack by an adversary that successfully circumvents accessing the host of that module. This is typically addressed in the context of physically accessing a module’s hardware, but it can also be interpreted to mean something like the host of a cloud provider accessing a virtual machine instance running that module under duress due to laws like China’s State Cryptography Law or the US Government’s PATRIOT Act.
There are four security levels under the most current version of FIPS 140–3. These levels rise from being robust conceivable tampering (Level 1), to passively and actively resisting tampering (Levels 2 and 3). The most aggressive level of FIPS 140–3 is Level 4, wherein a system resists tampering against even environmental attacks from physical access.
Where is FIPS 140 required?
Because there has not historically been a program for comprehensively measuring “good” cryptography, FIPS 140 is frequently seen as a requirement by most organizations. But it rarely is explicitly required outside military and federal systems requirements in NATO governments.
FIPS 140 certified encryption is explicitly required in a few key use cases:
FedRAMP Moderate+ Workloads
FedRAMP is a process for streamlining purchase of IT systems for the US Federal Government. FedRAMP maintains a set of Impact Levels which roughly correspond to how damaging the leaking of data from a particular workload could be to the US government. For Moderate+ workloads, FedRAMP systems must use cryptography from FIPS 140-approved crypto modules (i.e.: modules contained within the CMVP).
Common Criteria
The Common Criteria certification is an international program for verifying the security of IT infrastructure. While originally intended for US and NATO military and federal systems, Common Criteria is also commonly used as a barometer for security across US-friendly nations in Asia such as Japan and Singapore.
Common Criteria has different Evaluation Assurance Levels (EALs) that a vendor’s product is audited against. These EALs effectively refer to how comprehensively secure a product is against different types of well resourced adversaries. For example: a virtual machine running on a Windows box in a US Army base may require a Security Target with a EAL 2+. But the same virtual machine managing a weapons system on a Navy destroyer may require deployment on a EAL 4+ system due to its deployment in a battlefield setting.
For many EAL 2+ systems’ testing criteria, FIPS 140 cryptography is necessary for auditing specific security targets. In the US government this is taken to an additional step of requiring specific levels for certification, ensuring that some kinds of classified information (for example: Top Secret Compartmentalized Information) is protected with FIPS 140–2 Level 2+ crypto modules.
The Problems with FIPS 140 and the CMVP
FIPS 140 and the CMVP are far from perfect. And as nations become increasingly regionalized for data and security requirements, there is an increasingly growing call for a non-US replacement to these standards given a history of glaring issues with the FIPS 140 certification process.
First, certification under FIPS 140 is extremely difficult to maintain. Any code changes to a crypto module necessitate a new crypto module certification. It can take months or even years to certify a crypto module under FIPS 140.
The volume of work to assemble as part of a certification package (as well as the in-depth understanding of cryptography and the certification process itself) also ensure that most organizations typically hire consultants to manage this process. Because this process is so difficult and requires a relatively rare skill set, many consultancies will typically charge 6 figure sums in order to manage the completion of a FIPS 140–2 audit .
Because of the difficulty and cost of certifying a crypto module, most “FIPS Versions” of security products are several versions behind the current version of that product. This can lead to serious problems where exploits or vulnerabilities cannot be immediately patched without waiver, as doing so would require an update and re-certification of that product.
This has led to a number of major exploits in crypto modules that are effectively left unpatched due to a combination of vendors being unable to recertify quickly or — in more glaring examples — NIST and CSE deciding that a vulnerability isn’t a big problem and objecting to changing code to close a potential exploit.
For example, in the early 2010s security researchers noted that the random number generators (RNGs) in several FIPS-certified libraries behaved poorly under load. After researching some of the older RNGs used in these libraries, they discovered long-standing vulnerabilities that have been left unpatched even to this day. NIST and CSE have objected to the exploitability of these vulnerabilities due to the difficulty of presumed attacks exploiting these vulnerabilities, and given the context there are many within the infosec community who wonder whether such vulnerabilities are intended by FIPS to enable high level espionage by codebreakers from NSA or UK’s GHCQ.
As a result of the above, most vendors like Microsoft and Oracle highly encourage their enterprise customers not to use “FIPS Mode” versions of their product unless it is absolutely necessary, citing the impact of performance in using dated FIPS certified crypto modules.
TL;DR
We need better ways of agreeing on what’s good cryptography. Unfortunately this is very hard, as good cryptography requires both mathematically-secure ciphers and secure methods to generate and handle keys for those ciphers.
FIPS 140 and the CMVP are probably the most comprehensive ways to verify the security of encryption today. But they are not without fault, and in an increasingly political world of cryptography it is important that we develop an international and non-political standard for measuring the security of encryption.
