Manage data products

This document is intended for data product owners who want to update, delete, and grant access to data products in Knowledge Catalog (formerly Dataplex Universal Catalog).

For more information about the architecture and key concepts of data products, see About data products.

Before you begin

  1. Enable the Dataplex, BigQuery APIs.

    Roles required to enable APIs

    To enable APIs, you need the serviceusage.services.enable permission. If you created the project, then you likely already have this permission through the Owner role (roles/owner). Otherwise, you can get this permission through the Service Usage Admin role (roles/serviceusage.serviceUsageAdmin). Learn how to grant roles.

    Enable the APIs

Required roles

To get the permissions that you need to manage data products, ask your administrator to grant you the following IAM roles on the project:

For more information about granting roles, see Manage access to projects, folders, and organizations.

These predefined roles contain the permissions required to manage data products. To see the exact permissions that are required, expand the Required permissions section:

Required permissions

The following permissions are required to manage data products:

  • Edit the overview system aspect type: dataplex.entryGroups.useOverviewAspect
  • Edit the refresh cadence system aspect type: dataplex.entryGroups.useRefreshCadenceAspect
  • Approve access request for a data product: dataplex.dataProducts.approve

You might also be able to get these permissions with custom roles or other predefined roles.

Access control for BigQuery models

Access to BigQuery models within a data product is managed through IAM conditions applied to the parent dataset's IAM policy.

If you delete and then recreate a BigQuery model using the same name, Knowledge Catalog restores the permissions previously granted to Google Groups or service accounts through the data product.

Set permissions to make data product discoverable

To enable potential consumers to search for, view, and request access to a data product, data product owners must ensure the product is discoverable by granting the appropriate IAM roles on the data product resource:

  • Search for, view, and request access to data products: Dataplex Data Product Consumer (dataplex.dataProductsConsumer)

  • View data product definitions and metadata (read-only access): Dataplex Data Product Viewer (dataplex.dataProductsViewer)

Manage data product access requests

When a data product consumer requests access, the data product owner receives an email notification. Owners can review, approve, or reject these requests using either the Google Cloud console or the API.

Console

Review access requests for a specific data product

  1. In the Google Cloud console, go to the Knowledge Catalog Data products page.

    Go to Data products

  2. Click the data product for which you want to review access requests.

  3. Click the Access request management tab.

  4. Select the requester whose request you want to review.

  5. Click Actions and then click Approve or Reject.

  6. Click Save.

Review access requests across multiple data products

  1. In the Google Cloud console, go to the Knowledge Catalog Governance workflows page.

    Go to Governance workflows

  2. Click the Pending Approvals tab.

  3. Optional: Filter the list by the specific data product name to isolate its requests.

  4. Select the target request and click Approve or Reject.

  5. Click Save.

Post-approval behavior

  • User requests: Consumers who requested access for their user identity are automatically added as members to the Google Group mapped to the access group.

  • Service account requests: Consumers who requested access for a service account are granted permissions to impersonate the data producer service account mapped to the access group.

Once processed, the request status updates automatically and the entry moves to the Approval log tab.

REST

List all pending access requests

To list all access requests that are pending your review within a specific project and location, send a GET request using the custom collection method :listReviewable:

curl -X GET \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
"https://dataplex.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/changeRequests:listReviewable"

Replace the following:

  • PROJECT_ID: the ID of your Google Cloud project

  • LOCATION: the region where the data product exists (for example, us-central1)

Approve an access request

Data product owners with the dataplex.dataProducts.approve permission can approve a pending request by sending a POST request to the custom method :approve:

curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
"https://dataplex.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/changeRequests/CHANGE_REQUEST_ID:approve"

Replace the following:

  • PROJECT_ID: the ID of your Google Cloud project

  • LOCATION: the region where the data product exists (for example, us-central1)

  • CHANGE_REQUEST_ID: the unique ID of your access request you want to approve

Unlike console-driven approvals, approving a change request using the REST API doesn't automatically modify Google Group memberships or configure service account token impersonation. You must complete these downstream access management steps manually:

  • For user identity access requests: Add the data product consumer to the mapped Google Group. For more information, see Add a membership to a Google Group.

  • For service account access requests: Grant the Service Account Token Creator (roles/iam.serviceAccountTokenCreator) IAM role to the data product consumer's service account on the resource by running the following command:

    gcloud iam service-accounts add-iam-policy-binding PRODUCER_SERVICE_ACCOUNT \
      --member="serviceAccount:CONSUMER_SERVICE_ACCOUNT" \
      --role="roles/iam.serviceAccountTokenCreator"
    

    Replace the following:

    • PRODUCER_SERVICE_ACCOUNT: the fully qualified email address of the service account mapped to the data product's access group

    • CONSUMER_SERVICE_ACCOUNT: the fully qualified email address of the service account belonging to the consumer requesting access

Reject an access request

Data product owners with the dataplex.dataProducts.approve permission can reject a pending request by sending a POST request to the custom method :reject along with a justification payload:

curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
-d '{
  "comment": "REJECTION_REASON"
}' \
"https://dataplex.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/changeRequests/CHANGE_REQUEST_ID:reject"

Replace the following:

  • PROJECT_ID: the ID of your Google Cloud project

  • LOCATION: the region where the data product exists (for example, us-central1)

  • CHANGE_REQUEST_ID: the unique ID of your access request you want to reject

Update a data product

You can update the basic details (for example, data product name, description, owner details), assets, access groups, permissions, contract, aspects, and additional documentation for an existing data product.

Update the basic details of a data product

Console

  1. In the Google Cloud console, go to the Knowledge Catalog Data products page.

    Go to Data products

  2. Click the data product that you want to update.

  3. Click Edit.

  4. Update the Data product name, Data product icon, Description, Data product owner(s) email address, Data product approver(s) email address, and Labels as required.

  5. Click Save.

C#

C#

Before trying this sample, follow the C# setup instructions in the Knowledge Catalog quickstart using client libraries. For more information, see the Knowledge Catalog C# API reference documentation.

To authenticate to Knowledge Catalog, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.

using Google.Cloud.Dataplex.V1;
using Google.LongRunning;
using Google.Protobuf.WellKnownTypes;

public sealed partial class GeneratedDataProductServiceClientSnippets
{
    /// <summary>Snippet for UpdateDataProduct</summary>
    /// <remarks>
    /// This snippet has been automatically generated and should be regarded as a code template only.
    /// It will require modifications to work:
    /// - It may require correct/in-range values for request initialization.
    /// - It may require specifying regional endpoints when creating the service client as shown in
    ///   https://cloud.google.com/dotnet/docs/reference/help/client-configuration#endpoint.
    /// </remarks>
    public void UpdateDataProductRequestObject()
    {
        // Create client
        DataProductServiceClient dataProductServiceClient = DataProductServiceClient.Create();
        // Initialize request argument(s)
        UpdateDataProductRequest request = new UpdateDataProductRequest
        {
            DataProduct = new DataProduct(),
            UpdateMask = new FieldMask(),
            ValidateOnly = false,
        };
        // Make the request
        Operation<DataProduct, OperationMetadata> response = dataProductServiceClient.UpdateDataProduct(request);

        // Poll until the returned long-running operation is complete
        Operation<DataProduct, OperationMetadata> completedResponse = response.PollUntilCompleted();
        // Retrieve the operation result
        DataProduct result = completedResponse.Result;

        // Or get the name of the operation
        string operationName = response.Name;
        // This name can be stored, then the long-running operation retrieved later by name
        Operation<DataProduct, OperationMetadata> retrievedResponse = dataProductServiceClient.PollOnceUpdateDataProduct(operationName);
        // Check if the retrieved long-running operation has completed
        if (retrievedResponse.IsCompleted)
        {
            // If it has completed, then access the result
            DataProduct retrievedResult = retrievedResponse.Result;
        }
    }
}

Go

Go

Before trying this sample, follow the Go setup instructions in the Knowledge Catalog quickstart using client libraries. For more information, see the Knowledge Catalog Go API reference documentation.

To authenticate to Knowledge Catalog, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.


//go:build examples

package main

import (
	"context"

	dataplex "cloud.google.com/go/dataplex/apiv1"
	dataplexpb "cloud.google.com/go/dataplex/apiv1/dataplexpb"
)

func main() {
	ctx := context.Background()
	// This snippet has been automatically generated and should be regarded as a code template only.
	// It will require modifications to work:
	// - It may require correct/in-range values for request initialization.
	// - It may require specifying regional endpoints when creating the service client as shown in:
	//   https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options
	c, err := dataplex.NewDataProductClient(ctx)
	if err != nil {
		// TODO: Handle error.
	}
	defer c.Close()