Translating text from a photo

This page shows how to detect text in an image, how to personalize translations, and how to generate synthetic speech from text. This tutorial uses Cloud Vision to detect text in an image file. Then, this tutorial shows how to use Cloud Translation to provide a custom translation of the detected text. Finally, this tutorial uses Text-to-Speech to provide machine dictation of the translated text.

Objectives

  1. Pass text recognized by the Cloud Vision API to the Cloud Translation API.

  2. Create and use Cloud Translation glossaries to personalize Cloud Translation API translations.

  3. Create an audio representation of translated text using the Text-to-Speech API.

Costs

Each Google Cloud API uses a separate pricing structure.

For pricing details, refer to the Cloud Vision pricing guide, the Cloud Translation pricing guide, and the Text-to-Speech pricing guide.

Before you begin

Make sure that you have:

Setting up client libraries

This tutorial uses Vision, Translation, and Text-to-Speech client libraries.

To install the relevant client libraries, run the following commands from the terminal.

Python

  pip install --upgrade google-cloud-vision
  pip install --upgrade google-cloud-translate
  pip install --upgrade google-cloud-texttospeech

Node.js

  npm install @google-cloud/vision
  npm install @google-cloud/translate
  npm install @google-cloud/text-to-speech

Setting up permissions for glossary creation

Creating Translation glossaries requires using a service account key with "Cloud Translation API Editor" permissions.

To set up a service account key with Cloud Translation API Editor permissions, do the following:

  1. Create a service account:

    1. In the Google Cloud console, go to the Service Accounts page.

      Go to Service Accounts

    2. Select your project.

    3. Click Create Service Account.

    4. In the Service account name field, enter a name. The Google Cloud console fills in the Service account ID field based on this name.

    5. Optional: In the Service account description field, enter a description for the service account.

    6. Click Create and continue.

    7. Click the Select a role field and select Cloud Translation > Cloud Translation API Editor

    8. Click Done to finish creating the service account.

      Do not close your browser window. You will use it in the next step.

  2. Download a JSON key for the service account you just created:

    1. In the Google Cloud console, click the email address for the service account that you created.
    2. Click Keys.
    3. Click Add key, then click Create new key.

    4. Click Create. A JSON key file is downloaded to your computer.

      Make sure to store the key file securely, because it can be used to authenticate as your service account. You can move and rename this file however you would like.

    5. Click Close.

  3. In your terminal, set the GOOGLE_APPLICATION_CREDENTIALS variable using the following command. Replace path_to_key with the path to the downloaded JSON file containing your new service account key.

    Linux or macOS

    export GOOGLE_APPLICATION_CREDENTIALS=path_to_key

    Windows

    set GOOGLE_APPLICATION_CREDENTIALS=path_to_key

Importing libraries

This tutorial uses the following system imports and client library imports.

Python

Before trying this sample, follow the Python setup instructions in the Cloud Translation quickstart using client libraries. For more information, see the Cloud Translation Python API reference documentation.

To authenticate to Cloud Translation, set up Application Default Credentials. For more information, see Set up authentication for a local development environment. 

import html
import os

# Imports the Google Cloud client libraries
from google.api_core.exceptions import AlreadyExists
from google.cloud import texttospeech
from google.cloud import translate_v3beta1 as translate
from google.cloud import vision

Node.js

Before trying this sample, follow the Node.js setup instructions in the Cloud Translation quickstart using client libraries. For more information, see the Cloud Translation Node.js API reference documentation.

To authenticate to Cloud Translation, set up Application Default Credentials. For more information, see Set up authentication for a local development environment. 

// Imports the Google Cloud client library
const textToSpeech = require('@google-cloud/text-to-speech');
const translate = require('@google-cloud/translate').v3beta1;
const vision = require('@google-cloud/vision');

// Import other required libraries
const fs = require('fs');
//const escape = require('escape-html');
const util = require('util');

Setting your project ID

You must associate a Google Cloud project with each request to a Google Cloud API. Designate your Google Cloud project by setting the GCLOUD_PROJECT environment variable from the terminal.

In the following command, replace project-id with your Google Cloud project ID. Run the following command from the terminal.

Linux or macOS

export GCLOUD_PROJECT=project-id

Windows

set GCLOUD_PROJECT=project-id

Using Vision to detect text from an image

Use the Vision API to detect and extract text from an image. The Vision API uses Optical Character Recognition (OCR) to support two text-detection features: detection of dense text, or DOCUMENT_TEXT_DETECTION, and sparse text detection, or TEXT_DETECTION.

The following code shows how to use the Vision API DOCUMENT_TEXT_DETECTION feature to detect text in a photo with dense text.

Python

Before trying this sample, follow the Python setup instructions in the Cloud Translation quickstart using client libraries. For more information, see the Cloud Translation Python API reference documentation.

To authenticate to Cloud Translation, set up Application Default Credentials. For more information, see Set up authentication for a local development environment. 

def pic_to_text(infile: str) -> str:
    """Detects text in an image file

    Args:
    infile: path to image file

    Returns:
    String of text detected in image
    """

    # Instantiates a client
    client = vision.ImageAnnotatorClient()

    # Opens the input image file
    with open(infile, "rb") as image_file:
        content = image_file.read()

    image = vision.Image(content=content)

    # For dense text, use document_text_detection
    # For less dense text, use text_detection
    response = client.document_text_detection(image=image)
    text = response.full_text_annotation.text
    print(f"Detected text: {text}")

    return text