About media documents and data stores

Builder for media.

This page provides information about documents and data stores for media. If you're using media recommendations or media search, review the schema requirements for your documents and data stores on this page before uploading your data.

Overview

A document is any item that you upload into an Agent Search data store. For media, a document typically contains metadata information about media content, such as videos, news articles, music files, or podcasts. The Document object in the API captures this metadata information.

Your data store contains a collection of documents that you have uploaded. When you create a data store, you specify that it will contain media documents. Data stores for media can only be attached to media apps, not to other app types such as custom search and recommendations. Data stores are represented in the API by the DataStore resource.

The quality of the data that you upload has a direct effect on the quality of the results that media apps provide. In general, the more accurate and specific information you can provide, the higher quality your results.

The data that you upload to the data store must be formatted in a specific JSON schema. The data arranged in that schema must be in a BigQuery table, a file or set of files in Cloud Storage, or in a JSON object that can be uploaded directly using the Google Cloud console.

Google predefined schema versus custom schema

You have two options for your media data schema:

  • The Google predefined schema. If you haven't already designed a schema for your media data, the Google predefined schema is a good choice.

  • Your own schema. If you have your data already formatted in a schema, you can use your own schema. For more information, see Custom schema below.

With either option, you can add fields to the schema after the initial data import. However, with the Google predefined schema, for the initial import, your data field names and types must exactly match those in the Document fields tables.

Key properties

Properties are used to train the models for search and recommendations. Property fields represent all fields in your schema.

Key properties are a special fixed set of properties in the Google schema. The key properties identify important information that is used to understand semantic meanings of the data.

If you use a custom schema, make sure to map your fields to as many of the key properties as possible. You do the mapping in the Google Cloud console after importing the data; see Create a media data store.

Google predefined JSON Schema for Document

When using media, documents can use the Google predefined JSON schema for media.

Documents are uploaded with either a JSON or Struct data representation. Make sure the document JSON or Struct conforms to the following JSON schema. The JSON schema uses JSON Schema 2020-12 for validation. For more about JSON Schema, also see the JSON Schema specification documentation at json-schema.org.

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "type": "object",
  "properties": {
    "title": {
      "type": "string",
    },
    "description": {
      "type": "string",
    },
    "media_type": {
      "type": "string",
    },
    "language_code": {
      "type": "string",
    },
    "categories": {
      "type": "array",
      "items": {
        "type": "string",