Knowledge Catalog (formerly Dataplex Universal Catalog) makes it easier to understand and analyze your data by automatically profiling your BigQuery and Iceberg REST Catalog tables.
Profiling is like getting a detailed health report for your data. It gives you key statistics, such as common values, how the data is spread out (distribution), and how many entries are missing (null counts). This information speeds up your analysis.
Data profiling recommends data quality check rules to ensure your data stays reliable.
Conceptual model
Knowledge Catalog lets you better understand the profile of your data by creating a data profile scan. A data profile scan is a type of Knowledge Catalog data scan that analyzes a BigQuery or an Iceberg REST Catalog table to generate statistical insights.
The following diagram shows how Knowledge Catalog scans data to report on statistical characteristics.
A data profile scan is associated with one BigQuery table or Iceberg REST Catalog table and scans the table to generate the data profiling results. A data profile scan supports several configuration options.
Configuration options
This section describes the configuration options available for running data profile scans.
Profiling modes
You can choose between the following profiling modes:
Standard: This is the default mode. It provides a comprehensive and customizable profile by scanning your data based on the sampling and filters that you specify. Standard mode is suitable for detailed analysis and long-term monitoring of data characteristics.