Downloading, preprocessing, and uploading the COCO dataset

COCO is a large-scale object detection, segmentation, and captioning dataset. Machine learning models that use the COCO dataset include:

Mask-RCNN
Retinanet
ShapeMask

Before you can train a model on a Cloud TPU, you must prepare the training data.

This document describes how to prepare the COCO dataset for models that run on Cloud TPU. The COCO dataset can only be prepared after you have created a Compute Engine VM. The script used to prepare the data, download_and_preprocess_coco.sh, is installed on the VM and must be run on the VM.

After preparing the data by running the download_and_preprocess_coco.sh script, you can bring up the Cloud TPU and run the training.

To fully download and preprocess and upload the COCO dataset to a Cloud Storage bucket takes approximately 2 hours.

In your Cloud Shell, configure gcloud with your project ID.

export PROJECT_ID=project-id
gcloud config set project ${PROJECT_ID}

In your Cloud Shell, create a Cloud Storage bucket using the following command:

Note: In the following command, replace bucket-name with the name you want to assign to your bucket.
```
gcloud storage buckets create gs://bucket-name --project=${PROJECT_ID} --location=us-central2
```

Create a Compute Engine VM to download and preprocess the dataset. For more information, see Create and start a Compute Engine instance.

$ gcloud compute instances create vm-name \
    --zone=us-central2-b \
    --image-family=ubuntu-2204-lts \
    --image-project=ubuntu-os-cloud \
    --machine-type=n1-standard-16 \
    --boot-disk-size=300GB \