In 2014, JPMorgan Chase was under a massive cyber- attack in which the data of 76 million private customers and 7 million business customers was leaked. The attacker was able to get administrative rights due to non-functional two-factor authentication and was able to access user data. The webserver and the web application were secured, but the database remained unencrypted where the data was copied from.

If Format Preserving Encryption had been used, this situation could have been mitigated. With FPE, there would not have been any change to the database schema, and the encryption could be integrated on the fly.

What is Format Preserving Encryption?

For basic information in regard to FPE, please refer to this link

To give you some context, Format Preserving Encryption or FPE is an encryption algorithm used to preserve the format of the clear text while it remains encrypted. However, the strength of FPE is lower compared to AES. FPE is, however, an important mechanism for encrypting data whilst preserving the data length. FPE ensures that while data remains encrypted, all programs, applications and databases continue to be functional.

Why use Format Preserving Encryption?

Implementing a perfectly secure network is harder than just encrypting your data. Encrypting data is cheaper, easier, more secure, and thus better in every way imaginable.There are many organizations with a legacy infrastructure which may not be as secure. Thus, protecting all of the data in the legacy network protects the data even if the network gets compromised. This change can be made with almost no impact to existing infrastructure.Even if the organization has a robust infrastructure, it may face issues while the data is under audit. No one wants to reveal raw customer data which may put their reputation under seize. Thus FPE can be used to de- identify all data, remove all PII (Personal Identifiable Information) of customers and would serve as an extra defence mechanism when data is breached. –

As per NIST 800-38G:

Format-preserving encryption (FPE) is designed for data that is not necessarily binary. In particular, given any finite set of symbols, like the decimal numerals, a method for FPE transforms data that is formatted as a sequence of the symbols in such a way that the encrypted form of the data has the same format, including the length, as the original data. Thus, an FPE encrypted SSN would be a sequence of nine decimal digits.

So, if we convert a 16-digit credit-card number it will return another 16-digit value. A 9-digit Social Security Number would return another 9-digit value.This cannot be achieved with other modes of encryption, such as AES where if we encrypt a credit card it will look like 0B6X8rMr058Ow+z3Ju5wimxYERpomz402++zNozLhv w= which is greater than 16 digits and has not just numbers inside it.This kind of output would not work in most systems or databases where we must follow strict data types. Thus if it expects 16 digit numbers, this type of output would not suffice and may even result in a system-wide crash.

NIST SP 800-38G recommends ways through which we can encrypt this sensitive data in the databases. These solutions would also follow FIPS 140-2. So if someone wishes to use FPE, they can rest assured that they would be following almost all regulations and standards which would be enough to satisfy regulatory requirements of HIPAA, PCI DSS etc.

Now, since we talked about why to use FPE regardless of using a legacy network, let us talk about FPE provided by Google Cloud Platform, and what benefit it provides over other platforms.

FPE By Google Cloud

Firstly, Google is the only cloud provider currently who is providing FPE through their DLP APIs. Now, most of the organizations are currently transitioning to the cloud, but to make that transition happen securely, data should stay encrypted while in transit.

To do that, Google provides FPE under Cloud Data Loss Prevention. Using DLP API, customers can encrypt their data using FPE and de-identify information using predefined info types such as Credit card numbers, phone numbers, etc.This would encrypt the data, and make it safer to transition to the cloud. The transfer of data from a datacenter to a database on the cloud would also maintain their referential integrity as well as their format.

Conclusion

FPE is an encryption mechanism that keeps data encrypted while databases and applications remain functional. FPE preserves the format of the data which allows legacy systems and networks to remain functional while data is encrypted. GCP provides a DLP API which offers FPE through their platform. This helps in making all types of systems and programs functional/available and also improves data auditability by removing all PII data within it.

Sources:

https://www.bbc.com/news/business-29470381

Free Downloads

Datasheet of Encryption Consulting Services

Encryption Consulting is a customer focused cybersecurity firm that provides a multitude of services in all aspects of encryption for our clients.

Download
Encryption Services

About the Author

Anish Bhattacharya is a Consultant at Encryption Consulting, working with PKIs, HSMs, creating Google Cloud applications, and working as a consultant with high-profile clients.

Format Preserving Encryption, or FPE, is an encryption algorithm which preserves the format of the information while it is being encrypted. FPE is weaker than standard Advanced Encryption Standard (AES), but FPE can preserve the length of the data as well as its format. FPE works with existing databases to encrypt data while keeping it in the same format, encrypting data while not harming the function of existing applications.

Table of Contents

Format Preserving Encryption (FPE)

FPE encrypts takes plaintext and converts it to ciphertext, of the same format. An example of this is seen above. By keeping the format the same, an application can do operations on data as if it were the plaintext, while not revealing the sensitive information encrypted. Personally Identifiable Information (PII), credit card information, social security numbers, and other sensitive data are normally encrypted with Format Preserving Encryption. FPE has three different modes of operation: FF1, FF2, and FF3, which are referred to as FFX as a whole.

All three of the operation methods utilize the AES block cipher within their encryption. The second mode, FF2, was created but never approved by the National Institute of Standards and Technology (NIST). Instead, FF1 and FF3 were approved. FF1 is the most commonly used version, because of a cryptanalytic attack performed on the FF3 method which showed it held flaws. This attack found that the proposed 128-bit level of security was not reached. In response, a method called FF3-1 was created in early 2019 which addressed these issues.

FPE works extremely well with existing applications as well as new applications. If an application needed data of a certain length and format, then FPE could be applied to the data to encrypt it while not necessitating the changing of the application. This works particularly well with software that cannot handle long strings of data. Format Preserving Encryption is a valid encryption algorithm to be used for compliance with NIST standards. An NIST publication, called NIST 800-38G, was created to address FPE.  This NIST publication focuses on the three methods of Format Preserving Encryption, describing the technical details of each mode of operation.

Secure your data through Encryption Assessment

FPE on the Cloud

Many vendors offer FPE with their services, including Comforte, HashiCorp, Futurex and Xmart Solutions. Some cloud service providers (CSPs) offer options to utilize FPE within their platform, but far fewer than regular vendors. Of the three biggest CSPs, Microsoft Azure, Amazon Web Services (AWS), and Google Cloud Platform (GCP), only GCP offers users the ability to work with Format Preserving Encryption.

Using FFX, Google Cloud Platform allows users to have access to FF1 and FF3 FPE methods, though FF1 is the more commonly used method. FFX uses multiple rounds of a Feistel function on the plaintext, along with a key, to create the ciphertext. A Feistel function splits the plaintext into two parts, permutates the text to change what it looks like, and then swaps the left half of text to the right and vice versa. The FF1 method uses 10 rounds of a Feistel function, and FF3 uses 8 rounds.

For a user utilizing the Data Loss Prevention (DLP) API in GCP, to encrypt with FFX, an alphabet to be used to encrypt the plaintext must be specified. This can be done by using a shortened name of one of the four most common alphabet types. NUMERIC specifies numbers 0-9, HEXADECIMAL includes the NUMERIC alphabet along with A-F, UPPER_CASE_ALPHA_NUMERIC is for 0-9 and A-Z, and ALPHA_NUMERIC specifies 0-9, A-Z and a-z. Users can also use a radix value specifying the size of the alphabet. Specifying 2 gives an alphabet consisting of the numbers 0 and 1, while specifying 95 gives an alphabet with all numeric, upper-case alpha, lower-case alpha, and symbol characters. The last way to specify the alphabet is by providing the exact numbers, symbols, and letters that are in the alphabet.

Encrypting with FPE on GCP can result in an image like the previous image, or a surrogate annotation may precede the ciphertext, if the data is unstructured. A ciphertext with a surrogate annotation will follow this format:

SURROGATE_INFOTYPE(SURROGATE_LENGTH): SURROGATE_VALUE

The surrogate_infotype is chosen by the user and can be any word. The surrogate_length tells how long the surrogate_value is, and the surrogate_value is the ciphertext itself. An example of ciphertext with a surrogate annotation is:

FaKeInFoTyPe(13): 182-123-3596

FPE with Encryption Consulting

Encryption Consulting offers blogs providing information for any organization or user wanting to learn more about FPE. We also provide resources that utilize Google Cloud Platform’s FPE options. Bucket Protector and Cloud Data Protector both work with the DLP and Key Management Services (KMS) APIs to encrypt data being sent to Google Cloud Storage or data on an organization’s local database with FPE. Bucket Protector and Cloud Data Protector fall under the Cloud Data Lake Protection services offered by Encryption Consulting, protecting data at the Data Storage and Data Ingestion phases of the Cloud Data Lake Lifecycle.

Free Downloads

Datasheet of Encryption Consulting Services

Encryption Consulting is a customer focused cybersecurity firm that provides a multitude of services in all aspects of encryption for our clients.

Download
Encryption Services

About the Author

President at Encryption Consulting LLC focusing on providing consulting to customers in the Applied Cryptography space.

Online data security has always been important, but never more so than now. With more and more of our data being stored on the cloud, users need to look for the best security solutions to ensure their confidential information is secure. While all parts of online data security are necessary to secure data, arguably the most important portion is data encryption.

This is why more and more cloud services are using a type of encryption called Format-Preserving Encryption.

What is Format-Preserving Encryption?

If your company has multiple 16-digit credit card numbers stored in a database, but the encrypted ciphertext needs to be 16-digits as well after encryption, this is where Format-Preserving Encryption [FPE] comes in.FPE encrypts plaintext that is a certain length and produces a ciphertext that is the same length as the plaintext and uses the same set of values as the plaintext. Using the previous example of a 16-digit credit card number with a plaintext of 1483920193402918, the ciphertext created with FPE could produce an output of 1483666666662918.

By using FPE, you can see that the ciphertext and plaintext are the same length and only use numerical values for encryption. One cloud provider that lets users implement FPE in their encryption is Google Cloud.

Format-Preserving Encryption in Google Cloud

Google Cloud gives users access to a de-identification technique called pseudonymization. Pseudonymization is a technique that replaces sensitive data with cryptographically generated tokens. Google Cloud supports three different pseudonymization techniques:

  1. Deterministic encryption using AES-SIV
  2. Format-Preserving Encryption
  3. Cryptographic hashing

All three techniques use cryptographic keys for data transformation, but we will focus on the Format-Preserving Encryption.
Google Cloud uses a type of FPE called FPE-FFX. FFX focuses on two different FPE methods,FF1 and FF3, to encrypt data.At the time of writing this, FF1 is the only method currently supported for encryption. FF2 did not make it to publication at the time of FFX’s creation. FF2 and FF3 derivations are being resubmitted, but after a cryptanalytic attack in 2017, FF3 was considered to be too insecure.FFX uses multiple rounds of a Feistel function on the plaintext, along with the use of a key, to create the ciphertext. A Feistel function splits the plaintext into two parts and does a permutation each round on each half of the plaintext, and then swaps the left half of text to the right and vice versa. The FF1 method uses 10 rounds of the Feistel function, and FF3 uses 8 rounds of the Feistel function. FPE-FFX has several steps necessary to encrypt data.To begin encryption, the alphabet being used to de-identify the data must be specified in one of three ways:

  1. Using one of four values that represent the most common character sets/alphabets
  2. Using a radix value specifying the size of the alphabet. Specifying 2 gives an alphabet consisting of the numbers 0 and 1, while specifying 95 gives an alphabet with all numeric, upper-case alpha, lower-case alpha, and symbol characters
  3. By building an alphabet containing the exact characters to be used

When using FPE-FFX in Google Cloud, the data is encrypted as previously described, but can also be prepended with a surrogate annotation, resulting in a final token. The token takes the following form when a surrogate annotation is included: surrogate_infotype(surrogate_length): surrogate_value. The surrogate annotation is surrogate_infotype(surrogate_length). The infotype is defined by the user and the surrogate value is the resulting ciphertext. If no surrogate annotation is specified, then the final token is just the surrogate value. To re-identify unstructured data, the full token, including a surrogate annotation, is necessary, while structured data only needs the surrogate value.

Conclusion

Format preserving encryption is extremely important for users who wish to keep the ciphertext after encryption as the same length as the plaintext. Of the several different FPE-FFX methods used on Google Cloud, FF1 is the best practice method to use, due to the extra rounds of the Feistel function it goes through.

Structured data requires a surrogate annotation be prepended on the ciphertext to allow for re-identification of data. Google Cloud has a strong implementation of FPE in place for customer use. For those in need of same length plaintext and ciphertext, Google Cloud’s FPE-FFX is their best choice.

Free Downloads

Datasheet of Encryption Consulting Services

Encryption Consulting is a customer focused cybersecurity firm that provides a multitude of services in all aspects of encryption for our clients.

Download
Encryption Services

About the Author

Riley Dickens is a Consultant at Encryption Consulting, working with PKIs, creating Google Cloud applications, and working as a consultant with high-profile clients.

Let's talk