Agentic vision

Gemini Robotics ER models can write and execute Python code to manipulate images and apply logic before answering. This page covers code execution examples: object detection with zoom and crop, instrument reading, fluid measurement, circuit board reading, and image annotation.

To adapt these examples to your own use case, replace the prompt text and the uploaded image file with your own. You can also adjust the requested JSON schema in the prompt to match the output structure your application needs, or add a system_instruction to enforce output format and precision.

For full runnable code, see the Robotics cookbook.

Thinking level

You can control the thinking level of the model to trade latency for accuracy. Spatial tasks like object detection perform well with a low thinking level. Complex tasks like counting or weight estimation benefit from a higher thinking level.

The following example sets the thinking level to high for a complex counting task:

from google import genai

client = genai.Client()

uploaded_file = client.files.upload(file="scene.jpeg")

interaction = client.interactions.create(
    model="gemini-robotics-er-2-preview",
    input=[
        {
            "type": "image",
            "uri": uploaded_file.uri,
            "mime_type": uploaded_file.mime_type
        },
        {"type": "text", "text": "Identify and count all objects on the table."}
    ],
    generation_config={
        "thinking_level": "high"  # Use "minimal" or "low" for faster spatial tasks
    }
)

print(interaction.output_text)

See Thinking for details.

Object detection (Zoom and crop)

The following example uses code execution to zoom and crop an image for a clearer view when detecting objects and returning bounding boxes.

from google import genai

client =