AI Hypercomputer has several consumption options for acquiring compute resources. These options address needs such as workload duration, fault tolerance, and infrastructure model. Learn about these options and choose the best one for your use case.
As an alternative to manual evaluation, you can prompt Gemini in the Google Cloud console to compare consumption options for your workload and budget. For more information, see Design your AI infrastructure with Gemini.
Overview of consumption options
To compare consumption options across key characteristics, use the following table:
| Consumption option | Provisioning model | Best for | Capacity assurance | Lifespan | Preemptible | Quota | Discounts | Allocation model |
|---|---|---|---|---|---|---|---|---|
| Flex-start | Flex-start | Short-duration workloads up to seven days that can wait for capacity. | Best-effort | Up to seven days | No | Preemptible quota | Discounted (up to 53%) on supported series | Dense for MIG resize requests; best-effort for standalone VMs. |
| Spot VMs | Spot | Fault-tolerant, short-duration general GPU workloads. | Best-effort | Preemptible | Yes | Preemptible quota | Deeply discounted (up to 91%) | Standard |
| Standard reservations | Standard | Critical general GPU workloads that require a very high level of assurance for capacity. | Very high | Very high | User-defined | No | No quota consumed | Discounted rates with committed use discounts (CUDs) |
| Future reservations for capacity blocks | Reservation-bound | Large-scale, long-running training on clustered GPU. | Very high | Unlimited within the reservation period. | No | Automatically increased | Discounted (up to 53%) with CUDs | Dense |
| Future reservations in calendar mode | Reservation-bound | clustered GPU workloads up to 90 days that require reserved capacity. | Very high | Up to 90 days | No | No quota consumed | Discounted (up to 53%) | Dense |
| On-demand VMs | Standard | general GPU workloads with no specific duration. | Best-effort | Unlimited | No | On-demand quota | None (pay-as-you-go) | Standard |
The consumption option that you use to deploy your infrastructure depends on whether you use General GPU or Clustered GPU.
General GPU: designed for small-scale inference, development, and smaller-scale training workloads. This type of GPU offers operational flexibility through asynchronous maintenance. To view the available options, see Consumption options for general GPU.
Clustered GPU: designed for large-scale, tightly coupled training workloads and large-scale inference, RL, and reasoning model workloads that require high capacity assurance and dense resource allocation to minimize network latency. To view the available options, see Consumption options for clustered GPU.
Consumption options for general GPUs
Use the following consumption options to acquire capacity for general GPU.
Use Flex-start
To run short-duration workloads that require densely allocated resources, you can request compute resources for up to seven days by using Flex-start. Whenever resources are available, Compute Engine creates your requested number of VMs. You can stop standalone Flex-start VMs, but you can't stop Flex-start VMs that a managed instance group (MIG) creates through resize requests. Flex-start VMs exist until you delete them, or until Compute Engine deletes the VMs at the end of their run duration.
Ideal workloads
- Small model pre-training
- Model fine-tuning
- High performance computing (HPC) simulation
- Batch inference
Key characteristics
- Provisioning model: Flex-start.
- Broad hardware support: request any clustered GPU machine type, except A4X Max and A4X.
- Latency optimization: apply a compact placement policy to standalone VMs to minimize network latency.
- Cost efficiency: receive a discount of up to 53% on vCPUs, memory, GPUs, and Local SSD disks for A4, A3, A2, and G4 machine series. Standard on-demand rates apply to other supported series. For more information, see Flex-start pricing.
Use Spot
To run fault-tolerant workloads, you can obtain compute resources immediately based on availability. You get resources at the lowest price possible. However, Compute Engine can preempt VMs at any time to reclaim capacity.
Ideal workloads
- Batch processing and data analytics
- High performance computing (HPC) and media encoding