Configure message encryption

By default, Pub/Sub encrypts customer content at rest. Pub/Sub handles encryption for you without any additional actions on your part. This option is called Google default encryption.

If you want to control your encryption keys, then you can use customer-managed encryption keys (CMEKs) in Cloud KMS with CMEK-integrated services including Pub/Sub. Using Cloud KMS keys gives you control over their protection level, location, rotation schedule, usage and access permissions, and cryptographic boundaries. Using Cloud KMS also lets you track key usage, view audit logs, and control key lifecycles. Instead of Google owning and managing the symmetric key encryption keys (KEKs) that protect your data, you control and manage these keys in Cloud KMS.

After you set up your resources with CMEKs, the experience of accessing your Pub/Sub resources is similar to using Google default encryption. For more information about your encryption options, see Customer-managed encryption keys (CMEK).

CMEK with Cloud KMS Autokey

You can either create CMEKs manually to protect your Pub/Sub resources or use Cloud KMS Autokey. With Autokey, key rings and keys are generated on demand to support resource creation in Pub/Sub. Service agents that use the keys for encrypt and decrypt operations are created if they don't already exist and are granted the required Identity and Access Management (IAM) roles. For more information, see Autokey overview.

How CMEK works with Pub/Sub

When you configure Pub/Sub with CMEK, the service automatically encrypts all data using the specified key. Cloud KMS usage for CMEK might incur additional costs depending on your usage patterns.

Every message is encrypted at the following states and layers:

At the application layer, Pub/Sub individually encrypts incoming messages as soon as the message is received. This implementation adds the following features:

Envelope encryption pattern

Pub/Sub uses the envelope encryption pattern with CMEK. In this approach, the messages are not encrypted by Cloud KMS. Instead, Cloud KMS is used to encrypt Data Encryption Keys (DEKs) created by Pub/Sub for each topic. These DEKs are stored only in encrypted, or wrapped, form by Pub/Sub. Before storing a DEK, the service sends the DEK to Cloud KMS to be encrypted with the key encryption key (KEK) specified on the topic. A new DEK is generated for each topic approximately every six hours.

Before Pub/Sub publishes messages to a subscription, it encrypts them using the newest DEK that was generated for the topic. Pub/Sub decrypts the messages shortly before they are delivered to subscribers.

Configure CMEK with Pub/Sub

You can configure CMEK manually or by using Autokey.

Before you begin

You can configure CMEK for Pub/Sub using the Google Cloud console or the Google Cloud CLI.

Complete the following tasks:

  • Enable the Cloud KMS API.

  • Create a key ring and a key in Cloud KMS. Keys and key rings cannot be deleted.

For instructions on how to accomplish these tasks, Create a key ring and Create a key.

Because Pub/Sub resources are global, we strongly recommend that you use global Cloud KMS keys to configure CMEK-enabled topics. Depending on the locations of a topic's publishers and subscribers, the use of a regional Cloud KMS key could introduce unnecessary dependencies on cross-region network links.

Required roles and permissions

Pub/Sub uses a Google Cloud service agent to access Cloud KMS. The service agent is maintained internally by Pub/Sub for each project, and is not visible on the Service Accounts page in the Google Cloud console by default.

The Pub/Sub service agent has the form service-${PROJECT_NUMBER}@gcp-sa-pubsub.iam.gserviceaccount.com.

Pub/Sub requires specific permissions to encrypt and decrypt data using CMEK.

Complete the following steps to set up the required access:

  • Grant the Pub/Sub service agent the Cloud KMS Crypto Key Encrypter/Decrypter (roles/cloudkms.cryptoKeyEncrypterDecrypter) role.

    gcloud kms keys add-iam-policy-binding CLOUD_KMS_KEY_NAME \
    --member=serviceAccount:service-PROJECT_NUMBER@gcp-sa-pubsub.iam.gserviceaccount.com \
    --role=roles/cloudkms.cryptoKeyEncrypterDecrypter

    Replace the following:

    • CLOUD_KMS_KEY_NAME: The name of the Cloud KMS key.

      The key is of the format projects/PROJECT_ID/locations/LOCATION/keyRings/KEY_RING/cryptoKeys/CRYPTO_KEY.