Secure shared data in use with Confidential Space

1. Overview

Confidential Space offers secure multi-party data sharing and collaboration, while allowing organizations to preserve the confidentiality of their data. This means that organizations can collaborate with each other while still maintaining control over their data and protecting it from unauthorized access.

Confidential Space unlocks scenarios where you want to gain mutual value from aggregating and analyzing sensitive, often regulated, data, while retaining full control over it. With Confidential Space, organizations can gain mutual value from aggregating and analyzing sensitive data such as personally identifiable information (PII), protected health information (PHI), intellectual property, and cryptographic secrets — while retaining full control over it.

What you'll need

• A Google Cloud Platform Project
• A Browser, such Chrome or Firefox
• Basic knowledge of Google Compute Engine ( codelab), Confidential VM, Containers and remote repositories
• Basic knowledge of Cloud KMS ( codelab)
• Basic knowledge of Service Accounts, Workload identity federation and attribute conditions.

What you'll learn

• How to configure the necessary Cloud resources for running Confidential Space
• How to run the workload in a Confidential VM running the Confidential Space VM image
• How to authorize access to protected resources based on the attributes of the workload code (what), the Confidential Space environment (where) and the account that is running the workload (who).

In this codelab you will set up a Confidential Space between Primus and Secundus Bank to determine their common customers without sharing full account lists with each other. It involves the following steps:

• Step 1: Set up required cloud resources for Primus and Secundus Banks. These cloud resources include cloud storage buckets, KMS keys, workload identity pools and service accounts for Primus and Secundus Banks. Primus Bank and Secundus Bank store their customer data in Cloud Storage buckets and they encrypt the data using Cloud Key Management service keys.
• Step 2: Create a workload service account which will be used by the workload VM. Secundus Bank who will be the operator of the workload will launch the workload VM. Primus Bank would be authoring the workload code.
• Step 3: Create a workload which includes two CLI commands, one for counting the customers from the provided location and another for finding common customers of Primus and Secundus Bank. The workload would be authored by Primus Bank and it will be packaged as a Docker image. This Docker image will be published to Artifact Registry.
• Step 4: Authorize a workload. Primus Bank would use a workload identity pool to authorize workloads to access their customer data based on attributes of who is running the workload, what the workload does and where the workload is running.
• Step 5: When workload would run, it would request access to the cloud resources of data collaborators (Primus Bank and Secundus Bank) by offering an Attestation Verifier service token with workload and environment claims. If the workload measurement claims in the token match the attribute condition in the workload identity pools of Primus and Secundus Banks, it returns the service account access token that has permission to access the respective cloud resources. The cloud resources will only be accessible to the workload running inside Confidential Space.
• Step 5(a): Run the first workload which counts Primus Bank customers from specific locations. For this workload, Primus Bank would be a data collaborator and workload author, which would provide the encrypted customer list to the workload running in Confidential Space. Secundus Bank would be a workload operator and it will run the workload in a Confidential Space.
• Step 5(b): Run the second workload which finds the common customers of Primus and Secundus Banks. For this workload, Primus Bank and Secundus Bank would both be data collaborators. They would provide the encrypted customer lists to the workload running in Confidential Space. Secundus Bank would again be a workload operator. This workload would be authorized by Secundus Bank too because the workload needs to access the encrypted customer lists of Secundus Bank as well for finding the common customers. In this case, Secundus Bank would authorize the workload to access their customer data based on attributes of who is running the workload, what the workload does and where the workload is running as mentioned in step 4 for Primus Bank.

2. Set Up Cloud Resources

Before you begin

• Clone this repository using the below command to get the required scripts that are used as part of this codelab.

git clone https://github.com/GoogleCloudPlatform/confidential-space.git

• Change the directory for this codelab.

cd confidential-space/codelabs/bank_data_analysis_codelab

• Ensure you have set the required project environment variables as shown below. For more information on creating a GCP project, please refer to this codelab. You can refer this to get details about how to retrieve project id and how it is different from project name and project number.

export PRIMUS_PROJECT_ID=<GCP project id of Primus bank>
export SECUNDUS_PROJECT_ID=<GCP project id of Secundus bank>

• Enable Billing for your projects.
• Enable Confidential Computing API and following APIs for both the projects.

gcloud services enable \
    cloudapis.googleapis.com \
    cloudkms.googleapis.com \
    cloudresourcemanager.googleapis.com \
    cloudshell.googleapis.com \
    container.googleapis.com \
    containerregistry.googleapis.com \
    iam.googleapis.com \
    confidentialcomputing.googleapis.com

• Set the variables for resource names as mentioned below using this command. You can override the resource names using these variables (e.g export PRIMUS_INPUT_STORAGE_BUCKET='my-input-bucket')
• You can set the following variables with existing cloud resource names in Primus project. If the variable is set, then the corresponding existing cloud resource from the Primus project would be used. If the variable is not set, cloud resource name would be generated from project-name and new cloud-resource would be created as part of the following:

• You can set the following variables with existing cloud resource names in the Secundus project. If the variable is set, then the corresponding existing cloud resource from the Secundus project would be used. If variable is not set, cloud resource name would generated from project-name and new cloud-resource would be created as part of the following:

• There are few artifacts used as part of this codelab as mentioned below:
• primus_customer_list.csv: The file that contains the customer data of Primus Bank. Here is the sample file used in this codelab.
• secundus_customer_list.csv: The file that contains the customer data of Secundus Bank. Here is the sample file used in this codelab.
• You will need certain permissions for these two projects:
• For the $PRIMUS_PROJECT_ID, you will need Cloud KMS Admin, Storage Admin, Artifact Registry Administrator, Service Account Admin, IAM Workload Identity Pool Admin.
• For the $SECUNDUS_PROJECT_ID, you will need Compute Admin, Storage Admin, Service Account Admin, Cloud KMS Admin, IAM Workload Identity Pool Admin, Security Admin (optional).
• Run the following script to set the remaining variable names to values based on your project ID for resource names.

source config_env.sh

Set up cloud resources for Primus Bank

Following cloud resources are required for the Primus Bank. Run this script to set up the resources for Primus Bank:

• Cloud storage bucket ($PRIMUS_INPUT_STORAGE_BUCKET) to store the encrypted customer data file of Primus Bank.
• Encryption key ($PRIMUS_ENC_KEY) and keyring ($PRIMUS_ENC_KEYRING) in KMS to encrypt the customer data file of Primus Bank.
• Workload Identity Pool ($PRIMUS_WORKLOAD_IDENTITY_POOL) to validate claims based on attributes conditions configured under its provider.
• Service account ($PRIMUS_SERVICE_ACCOUNT) attached to above mentioned workload identity pool ($PRIMUS_WORKLOAD_IDENTITY_POOL) has an access to decrypt data using the KMS key (using roles/cloudkms.cryptoKeyDecrypter role), read data from the cloud storage bucket (using objectViewer role) and connecting the service-account to the workload identity pool (using roles/iam.workloadIdentityUser).

./setup_primus_bank_resources.sh

Set up cloud resources for Secundus Bank

For the Secundus Bank, following cloud resources are required. Run this script to set up Secundus Bank resources. As part of this steps below mentioned resources will be created:

• Cloud storage bucket ($SECUNDUS_INPUT_STORAGE_BUCKET) to store the encrypted customer data file of Secundus Bank.
• Encryption key ($SECUNDUS_ENC_KEY) and keyring ($SECUNDUS_ENC_KEYRING) in KMS to encrypt the data file of Secundus Bank.
• Workload Identity Pool ($SECUNDUS_WORKLOAD_IDENTITY_POOL) to validate claims based on attributes conditions configured under its provider.
• Service account ($SECUNDUS_SERVICE_ACCOUNT) attached to above mentioned workload identity pool ($SECUNDUS_WORKLOAD_IDENTITY_POOL) has an access to decrypt data using the KMS key (using roles/cloudkms.cryptoKeyDecrypter role), read data from the cloud storage bucket (using objectViewer role) and connecting the service-account to the workload identity pool (using roles/iam.workloadIdentityUser).
• Cloud storage bucket ($SECUNDUS_RESULT_STORAGE_BUCKET) to store the result of workload execution by Secundus Bank.

./setup_secundus_bank_resources.sh

3. Create Workload

Create workload service-account

Now, you will create a service-account for the workload with required roles and permissions as mentioned below. Run the following script to create a workload service account in the Secundus Bank project. The VM that runs the workload would use this service-account.

This workload service-account ($WORKLOAD_SERVICE_ACCOUNT) will have the following roles:

• Grant the confidentialcomputing.workloadUser role to the workload service account . This will allow the user account to generate an attestation token.
• Grant the logging.logWriter role to the workload service account permission. This allows the Confidential Space environment to write logs to Cloud Logging in addition to the Serial Console, so logs are available after the VM is terminated.
• objectViewer to read data from the $PRIMUS_INPUT_STORAGE_BUCKET cloud storage bucket.
• objectViewer to read data from the $SECUNDUS_INPUT_STORAGE_BUCKET cloud storage bucket.
• objectAdmin to write the workload result to the $SECUNDUS_RESULT_STORAGE_BUCKET cloud storage bucket.

./create_workload_service_account.sh

Create workload

As part of this step, you will create a Docker image for the workload used in this codelab. The workload is a simple GoLang application which:

• Counts customers in a specified geographical location.
• Finds common customers of Primus and Secundus Bank from the customer lists stored in their respective cloud storage buckets.

Run the following script to create a workload in which the following steps are being performed:

• Create Artifact Registry ($PRIMUS_ARTIFACT_REPOSITORY) owned by Primus Bank where the workload would be published.
• Generate the code and update it with required resources names. The workload code used in this codelab can be found here.
• Build the code and package it in a Docker image. The corresponding Dockerfile can be found here.
• Publish the Docker image to the Artifact Registry ($PRIMUS_ARTIFACT_REGISTRY) owned by Primus Bank.
• Grant the service account $WORKLOAD_SERVICE_ACCOUNT read permission for the Artifact Registry ($PRIMUS_ARTIFACT_REGISTRY).

./create_workload.sh

4. Authorize and Run Workload(s)

Authorize Workload

Primus Bank wants to authorize workloads to access their customer data based on attributes of the following resources:

• What: Code that is verified
• Where: An environment that is secure
• Who: An operator that is trusted

Primus uses Workload identity federation to enforce an access policy based on these requirements.

Workload identity federation allows you to specify attribute conditions. These conditions restrict which identities can authenticate with the workload identity pool (WIP). You can add the Attestation Verifier Service to the WIP as a workload identity pool provider to present measurements and enforce the policy.

Workload identity pool was already created earlier as part of the cloud resources setup step. Now Primus Bank will create a new OIDC workload identity pool provider. The specified --attribute-condition authorizes access to the workload container. It requires:

• What: Latest $WORKLOAD_IMAGE_NAME uploaded to the $PRIMUS_ARTIFACT_REPOSITORY repository.
• Where: Confidential Space trusted execution environment is running on the fully supported Confidential Space VM image.
• Who: Secundus Bank $WORKLOAD_SERVICE_ACCOUNT service account.

gcloud config set project $PRIMUS_PROJECT_ID

gcloud iam workload-identity-pools providers create-oidc $PRIMUS_WIP_PROVIDER \
  --location="global" \
  --workload-identity-pool="$PRIMUS_WORKLOAD_IDENTITY_POOL" \
  --issuer-uri="https://confidentialcomputing.googleapis.com/" \
  --allowed-audiences="https://sts.googleapis.com" \
  --attribute-mapping="google.subject='assertion.sub'" \
  --attribute-condition="assertion.swname == 'CONFIDENTIAL_SPACE' && 
   'STABLE' in assertion.submods.confidential_space.support_attributes &&
   assertion.submods.container.image_reference == 'us-docker.pkg.dev/$PRIMUS_PROJECT_ID/$PRIMUS_ARTIFACT_REPOSITORY/$WORKLOAD_IMAGE_NAME:$WORKLOAD_IMAGE_TAG' && 
'$WORKLOAD_SERVICE_ACCOUNT@$SECUNDUS_PROJECT_ID.iam.gserviceaccount.com' in assertion.google_service_accounts"

Similar to the WIP created for Primus Bank, Secundus Bank wants to authorize workloads to access their customer data based on:

• What: The workload.
• Where: The Confidential Space Environment.
• Who: The account ($WORKLOAD_SERVICE_ACCOUNT) which is running the workload.

Primus Bank uses the image_reference claim, which includes the image tag, to determine whether they should authorize access. They control the remote repository, so they can be sure to only tag images which do not leak their data.

In comparison, Secundus Bank does not control the repository where they are getting the image, so they cannot safely make that assumption. Instead, they choose to authorize access to the workload based on its image_digest. Unlike the image_reference, which Primus Bank could change to point to a different image, Primus Bank could not have the image_digest refer to an image other than the one that Secundus Bank audited in the earlier step.

Before creating workload identity pool providers, we would collect the image_digest for the workload container image which would be used in the attribute conditions of the provider.

export WORKLOAD_IMAGE_DIGEST=$(docker images ${PRIMUS_PROJECT_REPOSITORY_REGION}-docker.pkg.dev/$PRIMUS_PROJECT_ID/$PRIMUS_ARTIFACT_REPOSITORY/$WORKLOAD_IMAGE_NAME:$WORKLOAD_IMAGE_TAG  --no-trunc --quiet)

gcloud config set project $SECUNDUS_PROJECT_ID

gcloud iam workload-identity-pools providers create-oidc $SECUNDUS_WIP_PROVIDER \
  --location="global" \
  --workload-identity-pool="$SECUNDUS_WORKLOAD_IDENTITY_POOL" \
  --issuer-uri="https://confidentialcomputing.googleapis.com/" \
  --allowed-audiences="https://sts.googleapis.com" \
  --attribute-mapping="google.subject='assertion.sub'" \
  --attribute-condition="assertion.swname == 'CONFIDENTIAL_SPACE' && 
'STABLE' in assertion.submods.confidential_space.support_attributes && 
assertion.submods.container.image_digest == ‘${WORKLOAD_IMAGE_DIGEST}'&&
 assertion.submods.container.image_reference == '${PRIMUS_PROJECT_REPOSITORY_REGION}-docker.pkg.dev/$PRIMUS_PROJECT_ID/$PRIMUS_ARTIFACT_REPOSITORY/$WORKLOAD_IMAGE_NAME:$WORKLOAD_IMAGE_TAG' && 
'$WORKLOAD_SERVICE_ACCOUNT@$SECUNDUS_PROJECT_ID.iam.gserviceaccount.com' in assertion.google_service_accounts"

Run Workload(s)

As part of this step, Secundus Bank will be running the workload on Confidential Space. This workload will get the access tokens from the Primus's workload identity pool and Secundus's workload identity pool to read and decrypt the customer data of Primus Bank and Secundus Bank respectively.

Required TEE arguments are passed using the metadata flag. Arguments for the workload container are passed using the "tee-cmd" portion of the flag. The result of workload execution will be published to $SECUNDUS_RESULT_STORAGE_BUCKET.

Run first workload

As part of the first workload execution, The workload will count the customers of Primus Bank from the provided location in workload container arguments. As shown below, the first workload will execute the "count-location" command and the result will be stored at $SECUNDUS_RESULT_STORAGE_BUCKET/seattle-result.

gcloud config set project $SECUNDUS_PROJECT_ID

gcloud compute instances create first-workload-cvm \
 --confidential-compute \
 --shielded-secure-boot \
 --maintenance-policy=TERMINATE \
 --scopes=cloud-platform --zone=${SECUNDUS_PROJECT_ZONE} \
 --image-project=confidential-space-images \
 --image-family=confidential-space \
--service-account=${WORKLOAD_SERVICE_ACCOUNT}@${SECUNDUS_PROJECT_ID}.iam.gserviceaccount.com \
 --metadata "^~^tee-image-reference=${PRIMUS_PROJECT_REPOSITORY_REGION}-docker.pkg.dev/${PRIMUS_PROJECT_ID}/${PRIMUS_ARTIFACT_REPOSITORY}/${WORKLOAD_IMAGE_NAME}:${WORKLOAD_IMAGE_TAG}~tee-restart-policy=Never~tee-cmd="[\"count-location\",\"Seattle\",\"gs://${SECUNDUS_RESULT_STORAGE_BUCKET}/seattle-result\"]""

View results

In the Secundus project, view the results of the first workload. Wait for 3-5 minutes for workload to complete the execution and result to be available on cloud storage bucket.

gcloud config set project $SECUNDUS_PROJECT_ID

gsutil cat gs://$SECUNDUS_RESULT_STORAGE_BUCKET/seattle-result

The result should be 3, as this is the number of people from Seattle, are listed in the primus_customer_list.csv file!

Run second workload

As part of the second workload execution, we will find the common customers of the Primus Bank and Secundus Bank. As shown below, the second workload will execute the "list-common-customers" command and the result will be stored at $SECUNDUS_RESULT_STORAGE_BUCKET/list-common-count.

gcloud config set project $SECUNDUS_PROJECT_ID

gcloud compute instances create second-workload-cvm \
 --confidential-compute \
 --shielded-secure-boot \
 --maintenance-policy=TERMINATE \
 --scopes=cloud-platform --zone=${SECUNDUS_PROJECT_ZONE} \
 --image-project=confidential-space-images \
 --image-family=confidential-space \
--service-account=${WORKLOAD_SERVICE_ACCOUNT}@${SECUNDUS_PROJECT_ID}.iam.gserviceaccount.com \
  --metadata "^~^tee-image-reference=${PRIMUS_PROJECT_REPOSITORY_REGION}-docker.pkg.dev/${PRIMUS_PROJECT_ID}/${PRIMUS_ARTIFACT_REPOSITORY}/${WORKLOAD_IMAGE_NAME}:${WORKLOAD_IMAGE_TAG}~tee-restart-policy=Never~tee-cmd="[\"list-common-customers\",\"gs://$SECUNDUS_RESULT_STORAGE_BUCKET/list-common-result\"]""

View results

In the Secundus project, view the results of the second workload. Wait for 3-5 minutes for workload to complete the execution and result to be available on cloud storage bucket.

gcloud config set project $SECUNDUS_PROJECT_ID

gsutil cat gs://$SECUNDUS_RESULT_STORAGE_BUCKET/list-common-result

The result should be the following list as these are the common customers between Primus and Secundus Bank.

Output:

Eric
Clinton
Ashley
Cooper

Run Unauthorized Workload

Primus Bank's contract allowing Secundus Bank access to their data expires. So Primus Bank updates their attribute condition to allow VMs with the service account from their new partner, Tertius Bank.

Primus Bank modifies the Workload Identity Pool provider

In the $PRIMUS_PROJECT_ID, update the attribute condition for the Attestation Verifier identity provider to authorize workloads at a new location.

1. Set the project to $PRIMUS_PROJECT_ID.

gcloud config set project $PRIMUS_PROJECT_ID

2. Export the GCP Project-Id of Tertius Bank using the below command. Later Primus Bank would use this to update the attribute condition of the workload identity pool provider. Primus bank will not stop authorizing the Secundus Bank workload service-accounts. It will now allow Tertius Bank workload service accounts.

export TERTIUS_PROJECT_ID=<GCP project-id of Tertius Bank>

3. Update the OIDC provider in the workload identity pool. Here '$WORKLOAD_SERVICE_ACCOUNT@$SECUNDUS_PROJECT_ID.iam.gserviceaccount.com' in assertion.google_service_accounts is changed to '$WORKLOAD_SERVICE_ACCOUNT@$TERTIUS_PROJECT_ID.iam.gserviceaccount.com' in assertion.google_service_accounts. Instead of authorizing the workload service account from Secundus Bank, now the workload service account from Tertius Bank will be authorized.

gcloud iam workload-identity-pools providers update-oidc $PRIMUS_WIP_PROVIDER \
  --location="global" \
  --workload-identity-pool="$PRIMUS_WORKLOAD_IDENTITY_POOL" \
  --issuer-uri="https://confidentialcomputing.googleapis.com/" \
  --allowed-audiences="https://sts.googleapis.com" \
  --attribute-mapping="google.subject='assertion.sub'" \
  --attribute-condition="assertion.swname == 'CONFIDENTIAL_SPACE' && 
   'STABLE' in assertion.submods.confidential_space.support_attributes &&
   assertion.submods.container.image_reference == '${PRIMUS_PROJECT_REPOSITORY_REGION}-docker.pkg.dev/$PRIMUS_PROJECT_ID/$PRIMUS_ARTIFACT_REPOSITORY/$WORKLOAD_IMAGE_NAME:$WORKLOAD_IMAGE_TAG' && 
'$WORKLOAD_SERVICE_ACCOUNT@$TERTIUS_PROJECT_ID.iam.gserviceaccount.com' in assertion.google_service_accounts"

Re-run the workload

When Secundus Bank tries to run the original workload, it fails. To view the error, delete the original results file and VM instance, and then try to run the workload again.

Delete existing results file and VM instance

1. Set the project to the $SECUNDUS_PROJECT_ID project.

gcloud config set project $SECUNDUS_PROJECT_ID

2. Delete the results file.

gsutil rm gs://$SECUNDUS_RESULT_STORAGE_BUCKET/seattle-result

3. Delete the Confidential VM instance.

gcloud compute instances delete second-workload-cvm

Run the unauthorized workload:

gcloud compute instances create second-workload-cvm \
 --confidential-compute \
 --shielded-secure-boot \
 --maintenance-policy=TERMINATE \
 --scopes=cloud-platform --zone=${SECUNDUS_PROJECT_ZONE}\
 --image-project=confidential-space-images \
 --image-family=confidential-space \
--service-account=${WORKLOAD_SERVICE_ACCOUNT}@${SECUNDUS_PROJECT_ID}.iam.gserviceaccount.com \
 --metadata "^~^tee-image-reference=${PRIMUS_PROJECT_REPOSITORY_REGION}-docker.pkg.dev/${PRIMUS_PROJECT_ID}/${PRIMUS_ARTIFACT_REPOSITORY}/${WORKLOAD_IMAGE_NAME}:${WORKLOAD_IMAGE_TAG}~tee-restart-policy=Never~tee-cmd="[\"list-common-customers\",\"gs://$SECUNDUS_RESULT_STORAGE_BUCKET/list-common-result\"]""

View error

Instead of the results of the workload, you see an error (The given credential is rejected by the attribute condition).

gsutil cat gs://$SECUNDUS_RESULT_STORAGE_BUCKET/seattle-result

Similar to this, If Primus Bank secretly modifies the workload to send Secundus Bank's entire customer list to a bucket Primus Bank owns then that attempt would fail as the malicious workload's digest would be different from the image digest that was authorized in Secundus Bank's Workload Identity Pool.

5. Clean Up

Here is the script that can be used to clean up the resources that we have created as part of this codelab. As part of this cleanup, the following resources will be deleted:

• Input cloud storage bucket of the Primus Bank ($PRIMUS_INPUT_STORAGE_BUCKET).
• A service-account of the Primus Bank ($PRIMUS_SERVICE_ACCOUNT).
• An artifact registry of the Primus Bank which holds image signatures ($PRIMUS_COSIGN_REPOSITORY).
• A workload identity pool of the Primus Bank($PRIMUS_WORKLOAD_IDENTITY_POOL).
• A workload service account of the Secundus Bank ($WORKLOAD_SERVICE_ACCOUNT).
• Input cloud storage bucket of the Secundus Bank ($SECUNDUS_INPUT_STORAGE_BUCKET).
• A service-account of the Secundus Bank ($SECUNDUS_SERVICE_ACCOUNT).
• An artifact registry of Secundus Bank which holds image signatures ($SECUNDUS_COSIGN_REPOSITORY).
• A workload identity pool of Secundus Bank($SECUNDUS_WORKLOAD_IDENTITY_POOL).
• A workload service account of Secundus Bank ($WORKLOAD_SERVICE_ACCOUNT).
• Workload Compute Instances.
• The result storage bucket of Secundus Bank ($SECUNDUS_RESULT_STORAGE_BUCKET).
• An artifact repository of Primus Bank ($PRIMUS_ARTIFACT_REPOSITORY).

./cleanup.sh

If you are done exploring, please consider deleting your project.

• Go to the Cloud Platform Console.
• Select the project you want to shut down, then click ‘Delete' at the top: this schedules the project for deletion.

Congratulations

Congratulations, you've successfully completed the codelab!

You learned how to secure shared data while retaining its confidentiality using Confidential Space.

What's next?

Check out some of these similar codelabs...

• Signed container image codelab
• How to transact digital assets with multi-party computation and confidential spaces
• Analyze confidential data with Confidential spaces

Further reading

• Feeling isolated? Confidential computing to the rescue
• Confidential Computing at GCP
• Confidential Space: The future of privacy-preserving collaboration
• How Google and Intel make Confidential Computing more secure
• Privacy vs. Progress - Advancing Security with Google Cloud Confidential Computing