Integrate fundamental AI concepts in Google Chat apps

1. Before you begin

What are Google Chat apps with AI?

Google Chat apps with AI do the following:

Bring your services and resources into Google Chat, which lets users get information and take action without leaving the conversation.
Integrate with generative AI models to create, search, and edit data like text or images.
Support an agentic experience by applying conversational AI concepts for more practical, natural, sophisticated and helpful interactions.

Why integrate Google Chat apps with AI?

The typical use cases fall in the following categories:

Content creation and edition. Generate marketing copy, craft social media posts, create realistic images, compose music, or aid in the creation of video content.
Data search and analysis. Extract key insights from an unstructured knowledge base, summarize lengthy texts, classify contents, or translate languages with enhanced accuracy and speed.
Conversation. Engage in natural, informative, and efficient conversations like you would with an assistant.
Task automation. Take actions on behalf of the user, such as creating a new calendar event, sending a document, or managing a ticket in an external system.

The ability to integrate these capabilities directly within the familiar interface of Google Chat is a huge opportunity for anyone who wants to improve the experience and productivity of their users.

Prerequisites

Basic knowledge of Google Cloud and Node.js.
Basic knowledge of Google Chat apps, including messages, cards, authentication, APIs, and HTTP endpoints.

What you will build

In this codelab, you will build eight minimalist Google Chat apps that integrate fundamental AI concepts to show how they can be applied in real-world applications. They are all built as Google Workspace add ons and rely on the HTTP architecture:

It works as follows:

A user sends a message in Google Chat to a Chat app, either as a direct message or in a Chat space.
An HTTP request is sent to the web server running as a Node.js Google Cloud Run function that contains the Chat app logic.
Optionally, the Chat app logic can integrate with Google Workspace services (like Calendar and Sheets), other Google services (like Maps, YouTube, and Vertex AI), or other web services (like a project management system or ticketing tool).
The web server sends an HTTP response back to the Chat app service in Chat.
The response is delivered to the user.
Optionally, the Chat app can call the Chat API to asynchronously post messages or perform other operations.

Each Google Chat app's Node.js Google Cloud Run function contains their own version of the following source files to take the necessary actions in steps #3 and #6 above:

package.json: A central manifest that acts as a blueprint for the Node.js project. It's used to define the metadata, dependencies, and scripts.
env.js: A script that sets constants required for execution. It should be edited based on the environment and configuration.
index.js: The main script that handles the logic for the Google Chat interaction events. Only the message event type is implemented in this codelab but it would typically include other types such as card click, slash command, and dialog in real-life applications.

Prompt app

This app relies on a Gemini model to converse with users in their natural languages using concise and plain text answers.

Format app

This app builds on the Prompt app by adding support for rich text answers compliant with the specific text format of Google Chat messages.

Ground app

This app builds on the Format app by adding support for the Google Search tool and returning sources in answer messages with cards.

MCP app

This app builds on the Format app by adding support for the Google Workspace Developer Assist Model Context Protocol (MCP).

Multi-turn app

This app builds on the Format app by adding support for conversational memory with a Google Cloud Firestore database.

Custom tool app

This app builds on the Multi-turn app by adding support for a function calling custom tool that calls the Google Workspace Calendar API based on information provided by the user.

Stream app

This app relies on a Gemini model to generate short stories based on themes provided by users. The Google Chat API is used to send results and statuses in messages as progress is made.

Multimodal app

This app relies on a Gemini model to edit images based on textual instructions from users. Google Chat APIs are used to download and upload the images as message attachments.

What you will learn

The fundamental AI concepts are relevant to Google Chat apps and how to apply them.
To access Vertex AI using the Google Gen AI SDK.
To use Google Workspace APIs to develop delightful and powerful features.
To leverage Cloud Run to build scalable Google Chat apps.

What you will need

Completion of the Build an HTTP Google Chat app quickstart with Node.js. This codelab builds on the resulting Google Cloud project, Google Chat app, and Google Cloud Run function.

2. Get set up

Initialize and access resources

In this section, you access and configure the following resources from your preferred web browser.

Google Chat API configuration

Open the Google Cloud console in a new tab, then follow these steps:

Select your project.
In the Google Cloud search field, search for "Google Chat API", then click Google Chat API, click Manage, and click Configuration.

Set the App name and Description to Gen AI App.
Click Save.

Google Chat space

Open Google Chat in a new tab, then follow these steps:

If not already done, open a direct message space with the Chat app.
Type Hello and press enter, the Chat app should reply with your name and avatar image.

Google Cloud Run function service

Open the Google Cloud console in a new tab, then follow these steps:

Select your project.
Click Menu ☰ > Cloud Run > Services.

In the list of services, click addonchatapp, then open the Source tab.

Download source code and resources locally

Download this GitHub repository.

In your preferred local development environment, open the node/chat/gen-ai-apps directory.

3. Prompt app

This app prompts Gemini on Vertex AI to converse with users in their natural languages using concise and plain text answers. The implementation relies on the Google Gen AI SDK for Node.js.

Review concepts

Natural language

Any language spoken or written by humans for everyday communication, in contrast to artificial or computer-based languages.

Cloud Run functions

Cloud Run functions are great for building serverless backends, doing real-time data processing, and creating intelligent apps. There are no servers to provision, manage, patch, or update. They automatically scale, and are highly available and fault-tolerant.

Prompting

Prompting refers to the technique of crafting input (prompts) to guide a generative AI model to produce a desired output. It usually involves carefully phrasing questions, providing context, giving instructions, or giving examples to obtain specific and relevant responses from the model.

Vertex AI

Vertex AI offers everything you need to build and use generative AI, including AI solutions, search and conversation, more than 130 foundation models, and a unified AI platform.

Gemini

Gemini is a multimodal LLM from Google accessible through Vertex AI. It helps people unlock their human potential so that they can augment their imagination, expand their curiosity, and enhance their productivity.

Google Gen AI SDK

The Google Gen AI SDK is designed for developers to build applications powered by Gemini, it provides a unified interface compatible with both the Gemini Developer API and Vertex AI. It comes with client libraries in Python, Go, Node.js, and Java.

Review flow

Review source code

env.js

...
// Replace with your GCP project ID.
projectID: process.env.PROJECT_ID || 'your-google-cloud-project-id',

// Replace with your GCP project location.
location: process.env.LOCATION || 'your-google-cloud-project-location',

// Replace with the Gemini model to use.
model: process.env.MODEL || 'gemini-2.5-flash-lite',
...

index.js

// Import the Google Gen AI SDK.
import { GoogleGenAI } from '@google/genai';
...
// Use Vertex AI.
const genAI = new GoogleGenAI({vertexai: true, project: env.projectID, location: env.location});

http('gen-ai-app', async (req, res) => {
 // Send a new Chat message with the generated answer
 return res.send({ hostAppDataAction: { chatDataAction: { createMessageAction: { message: {
   text: await generateAnswer(req.body.chat.messagePayload.message.text)
 }}}}});
});

async function generateAnswer(message) {
 // The prompt is made of the user's message and specific instructions for the model.
 const prompt = 'In a consice and with plain text only (no formatting), '
                 + 'answer the following message in the same language: ' + message;
 const aiResponse = await genAI.models.generateContent({model: env.model, contents: prompt});
 return aiResponse.candidates[0].content.parts[0].text;
};
...

package.json

...
"main": "index.js",
"type": "module",
"scripts": {
  "start": "node index.js"
},
"dependencies": {
  "@google-cloud/functions-framework": "^4.0.0",
  "@google/genai": "1.15.0"
},
...

Enable Vertex AI API

In the Google Cloud console, enable the Vertex AI API:

Click Menu ☰ > APIs & Services > Enabled APIs & Services and then confirm that Vertex AI API is in the list.

Update Google Cloud Run Node.js function

In your local development environment, change the current directory to node/chat/gen-ai-apps/1-prompt. It contains the entire source code and resources.
Open env.js in an editor and set the following:
projectID: The ID of your Google Cloud project. It can be retrieved from the Google Cloud console welcome page.

location: The region of your Google Cloud Run function service. It can be retrieved from the Google Cloud Run function service details page.

model: The model to use. You can find all available models from the Vertex AI documentation. The model set by default is Flash for a fast and cheap execution.

Go to the Source tab of the Google Cloud Run function service details page.

Click Edit source.
Set Function entry point to gen-ai-app.
Click ➕, type env.js, and click ✔️to create the missing source file.
Replace the entire contents of the index.js, env.js, and package.json files with those in your local development environment.
Click Save and redeploy.
Wait for the successful completion of the revision deployment.

Try it

In the direct message space with the Chat app in Google Chat, type Hello, how are you? and press enter. The app should answer concisely in plain text as per our instructions in the prompt.

In the direct message space with the Chat app in Google Chat, type Bonjour comment allez-vous? and press enter. The app should answer in French as per our instructions in the prompt.

4. Format app

This app builds on the Prompt app by adding support for rich text answers compliant with Google Chat text message format. Instructions in the prompt are updated with an exhaustive description of the different options that the model can use.