Pseudonymization

Pseudonymization is a de-identification technique that replaces sensitive data values with cryptographically generated tokens. Pseudonymization is widely used in industries like finance and healthcare to help reduce the risk of data in use, narrow compliance scope, and minimize the exposure of sensitive data to systems while preserving data utility and accuracy.

Sensitive Data Protection supports three pseudonymization techniques of de-identification, and generates tokens by applying one of three cryptographic transformation methods to original sensitive data values. Each original sensitive value is then replaced with its corresponding token. Pseudonymization is sometimes referred to as tokenization or surrogate replacement.

Pseudonymization techniques enable either one-way or two-way tokens. A one-way token has been transformed irreversibly, while a two-way token can be reversed. Because the token is created using symmetric encryption, the same cryptographic key that can generate new tokens can also reverse tokens. For situations in which you don't need reversibility, you can use one-way tokens that use secure hashing mechanisms.

It's helpful to understand how pseudonymization can help protect sensitive data while allowing your business operations and analytical workflows easy access to and use of the data they need. This topic explores the concept of pseudonymization and the three cryptographic methods to transform data that Sensitive Data Protection supports.

For instructions on how to implement these pseudonymization methods and for more examples of using Sensitive Data Protection, see De-identifying sensitive data.

Supported cryptographic methods in Sensitive Data Protection

Sensitive Data Protection supports three pseudonymization techniques, all of which use cryptographic keys. Following are the available methods:

• Deterministic encryption using AES-SIV: An input value is replaced with a value that has been encrypted using the AES-SIV encryption algorithm with a cryptographic key, encoded using base64, and then prepended with a surrogate annotation, if specified. This method produces a hashed value, so it does not preserve the character set or the length of the input value. Encrypted, hashed values can be re-identified using the original cryptographic key and the entire output value, including surrogate annotation. Learn more about the format of values tokenized using AES-SIV encryption.
• Format preserving encryption: An input value is replaced with a value that has been encrypted using the FPE-FFX encryption algorithm with a cryptographic key, and then prepended with a surrogate annotation, if specified. By design, both the character set and the length of the input value are preserved in the output value. Encrypted values can be re-identified using the original cryptographic key and the entire output value, including surrogate annotation. (For some important considerations around using this encryption method, see Format preserving encryption later in this topic.)
• Cryptographic hashing: An input value is replaced with a value that has been encrypted and hashed using Hash-based Message Authentication Code (HMAC)-Secure Hash Algorithm (SHA)-256 on the input value with a cryptographic key. The hashed output of the transformation is always the same length and can't be re-identified. Learn more about the format of values tokenized using cryptographic hashing.

These pseudonymization methods are summarized in the following table. Table rows are explained following the table.

• Encryption type: The kind of encryption used in the de-identification transformation.
• Supported input values: Minimum requirements for input values.
• Surrogate annotation: A user-specified annotation that is prepended to encrypted values to provide context to users and to provide information for Sensitive Data Protection to use in the re-identification of a de-identified value. A surrogate annotation is required for re-identification of unstructured data. It is optional when transforming a column of structured, or tabular, data with a RecordTransformation.
• Context tweak: A reference to a data field that "tweaks" the input value so that identical input values can be de-identified to different output values. The context tweak is optional when transforming a column of structured, or tabular data, with a RecordTransformation. To learn more, see Using context tweaks.
• Character set and length preserved: Whether a de-identified value is made up of the same set of characters as the original value, and whether the length of the de-identified value matches that of its original value.
• Reversible: Can be re-identified using the cryptographic key, surrogate annotation, and any context tweak.
• Referential integrity: Referential integrity allows for records to maintain their relationship to one another even after having their data individually de-identified. Given the same crypto key and context tweak, a table of data will be replaced with the same obfuscated form each time it is transformed, which ensures that connections between values (and, with structured data, records) are preserved, even across tables.

How tokenization works in Sensitive Data Protection

The basic process of tokenization is the same for all three methods that Sensitive Data Protection supports.

Step 1: Sensitive Data Protection selects data to tokenize. The most common way to do this is to use a built-in or custom infoType detector to match on the desired sensitive data values. If you are scanning structured data (such as a BigQuery table), you can also perform tokenization on entire columns of data using record transformations.

For more information about the two categories of transformations—infoType and record transformations—see De-identification transformations.

Step 2: Using a cryptographic key, Sensitive Data Protection encrypts each input value. You can provide this key in one of three ways:

• By wrapping it using Cloud Key Management Service (Cloud KMS). (For maximum security, Cloud KMS is the preferred method.)
• By using a transient key, which Sensitive Data Protection generates at the time of de-identification and then discards. A transient key only keeps integrity per API request. If you need integrity or plan to re-identify this data, do not use this key type.
• Directly in raw text form. (Not recommended.)

For more details, see the Using cryptographic keys section, later in this topic.

Step 3 (Cryptographic hashing and deterministic encryption with AES-SIV only): Sensitive Data Protection encodes the encrypted value using base64. With cryptographic hashing, this encoded, encrypted value is the token, and the process continues with Step 6. With deterministic encryption using AES-SIV, this encoded, encrypted value is the surrogate value, which is just one component of the token. The process continues with Step 4.

Step 4 (Format preserving and deterministic encryption with AES-SIV only): Sensitive Data Protection adds an optional surrogate annotation to the encrypted value. The surrogate annotation helps identify encrypted surrogate values by prepending them with a descriptive string that you define. For example, without an annotation you might not be able to tell apart a de-identified phone number and a de-identified Social Security or other identification number. In addition, to re-identify values in unstructured data that have been de-identified using either format preserving encryption or deterministic encryption, you must specify a surrogate annotation. (Surrogate annotations are not required when transforming a column of structured, or tabular, data with a RecordTransformation.)

Step 5 (Format preserving and deterministic encryption with AES-SIV of structured data only): Sensitive Data Protection can use optional context from another field to "tweak" the token generated. This enables you to change the scope of the token. For example, suppose you have a database of marketing campaign data that includes email addresses and you want to generate unique tokens for the same email address "tweaked" by the campaign ID. This would allow someone to join data for the same user within the same campaign but not across different campaigns. If a context tweak is used to create the token, then this context tweak is also required for the de-identification transformations to be reversed. Format preserving and deterministic encryption using AES-SIV support contexts. Learn more about using context tweaks.

Step 6: Sensitive Data Protection replaces the original value with the de-identified value.

Tokenized value comparison

This section demonstrates how typical tokens look after being de-identified using each of the three methods discussed in this topic. The example sensitive data value is a North American telephone number (1-206-555-0123).

Deterministic encryption using AES-SIV

With de-identification using deterministic encryption and AES-SIV, an input value (and, optionally, any specified context tweak) is encrypted using AES-SIV with a cryptographic key, encoded using base64, and then optionally prepended with a surrogate annotation, if specified. This method does not preserve the character set (or "alphabet") of the input value. In order to generate printable output, the resulting value is encoded in base64.

The resulting token, assuming a surrogate infoType has been specified, is in the form:

SURROGATE_INFOTYPE(SURROGATE_VALUE_LENGTH):SURROGATE_VALUE

The following annotated diagram shows an example token—the output of a de-identification operation using deterministic encryption with AES-SIV on the value 1-206-555-0123. The optional surrogate infoType has been set to NAM_PHONE_NUMB:

1. Surrogate annotation
2. Surrogate infoType (defined by user)
3. Character length of transformed value
4. Surrogate (transformed) value

If you do not specify a surrogate annotation, the resulting token is equal to the transformed value, or #4 in the annotated diagram. To re-identify unstructured data, this entire token is required, including the surrogate annotation. When transforming structured data such as a table, the surrogate annotation is optional; Sensitive Data Protection can perform both de-identification and re-identification on an entire column using a RecordTransformation without a surrogate annotation.

Format preserving encryption

With de-identification using format preserving encryption, an input value (and, optionally, any specified context tweak) is encrypted using the FFX mode of format preserving encryption ("FPE-FFX") with a cryptographic key, and then optionally prepended with a surrogate annotation, if specified.

Unlike the other methods of tokenization described in this topic, the output surrogate value is the same length as the input value, and it is not encoded using base64. You define the character set—or "alphabet"—that the encrypted value is comprised of. There are three ways to specify the alphabet for Sensitive Data Protection to use in the output value:

• Use one of four enumerated values that represent the four most common character sets/alphabets.
• Use a radix value, which specifies the size of the alphabet. Specifying the minimum radix value of 2 results in an alphabet that consists of just 0 and 1. Specifying the maximum radix value of 95 results in an alphabet that includes all numeric characters, upper-case alpha characters, lower-case alpha characters, and symbol characters.
• Build an alphabet by listing the exact characters to use. For example, specifying 1234567890-* would result in a surrogate value that is made up of only numbers, hyphens, and asterisks.

The following table lists four common character sets by each one's enumerated value (FfxCommonNativeAlphabet), radix value, and list of the set's characters. The final row lists the full character set, which corresponds to the maximum radix value.