Google Cloud GPU instances give you access to NVIDIA H100, A100, and RTX PRO 6000 hardware through Compute Engine virtual machines, Google Kubernetes Engine, Vertex AI, and Cloud Run. Navigating five machine series, a quota system that starts at zero, and on-demand rates that can exceed $11 per GPU-hour takes real effort.
This guide covers what developers and ML engineers need to make the right call.
Key takeaways:
- GCP GPU instances fall into five machine series: A3 (H100), A2 (A100), G2 (L4), G4 (RTX PRO 6000), and N1 with attachable T4/V100/P100 GPUs, plus Cloud Run for serverless inference
- The only on-demand A3 configuration is the a3-highgpu-8g at $88.48/hr; smaller A3 SKUs require Spot or Flex-start
- Spot pricing cuts costs by up to 91%, but instances can be preempted with 30-second notice
- GCP's sustained use discounts apply automatically to N1 GPU instances with no commitment required
- For training and fine-tuning workloads, specialized GPU clouds can be 3-5x cheaper with no quota overhead
What Are Google Cloud GPU Instances?
Google Cloud GPU instances are virtual machines with NVIDIA accelerators attached, available through several delivery models. Compute Engine gives you the most control: attach a GPU to a VM and manage drivers, storage, and networking yourself. Google Kubernetes Engine (GKE) supports GPU node pools for containerized workloads. Vertex AI adds a managed ML layer on top of the same underlying compute. Cloud Run GPU lets you run inference containers without managing a VM at all.
GCP's GPU lineup is entirely NVIDIA hardware, rented across machine series optimized for different workloads, from large-scale training to inference to professional visualization.
GPUs vs TPUs on Google Cloud
Google Cloud also offers their proprietary Tensor Processing Units (TPUs), designed for large-scale transformer training and inference. TPUs deliver more performance per watt for matrix-heavy JAX workloads at pod scale, but require code to pass through the XLA compiler and are only available on GCP. GPUs support PyTorch natively, run across dozens of providers, and handle any workload without framework constraints.
How to Read a GCP GPU Machine Type Name
GCP machine type names encode the series, configuration, and scale in a compact string. Take a3-highgpu-8g from left to right:
a3is the machine series (third-generation accelerator-optimized)highgpuis the configuration variant (high GPU density)8gis the GPU count
g2-standard-4 means G2 series, standard configuration, 4 vCPUs, with one L4 GPU attached.
N1 instances are the exception, as GPUs are attached separately from the machine type name rather than encoded in it.
Google Cloud GPU Machine Types
A3 Series: NVIDIA H100
The A3 series is GCP's highest-performance GPU offering, built around NVIDIA H100 80GB SXM5 GPUs. The a3-highgpu-8g packs eight H100s into a single VM with 1.87 TB of RAM, designed for distributed LLM training and large-scale inference. GCP also offers A3 Mega instances with additional host RAM for memory-intensive multi-tenant workloads.
Provisioning a3-highgpu-1g, a3-highgpu-2g, and a3-highgpu-4g requires Spot VMs or Flex-start VMs; these are not available as standard on-demand instances. The a3-highgpu-8g is the only A3 option available on demand.
| Instance | GPU | GPU VRAM | vCPUs | RAM | $/hr (Spot/Flex-start for 1g–4g) |
|---|---|---|---|---|---|
| a3-highgpu-1g1 | 1x NVIDIA H100 80GB | 80 GB HBM3 | 26 | 234 GB | $11.06 |
| a3-highgpu-2g1 | 2x NVIDIA H100 80GB | 160 GB HBM3 | 52 | 468 GB | $22.12 |
| a3-highgpu-4g1 | 4x NVIDIA H100 80GB | 320 GB HBM3 | 104 | 936 GB | $44.24 |
| a3-highgpu-8g | 8x NVIDIA H100 80GB | 640 GB HBM3 | 208 | 1,872 GB | $88.48 |
| 1 a3-highgpu-1g, -2g, and -4g must be created as Spot VMs or Flex-start VMs. | |||||
A2 Series: NVIDIA A100
The A2 series uses NVIDIA A100 GPUs in two variants: a2-highgpu with 40GB A100s and a2-ultragpu with 80GB A100s. A2 is the workhorse for most fine-tuning and training workloads that do not require H100-class performance. The a2-highgpu-1g is the entry point for single-GPU A100 access on GCP.
| Instance | GPU | GPU VRAM | vCPUs | RAM | On-Demand $/hr |
|---|---|---|---|---|---|
| a2-highgpu-1g | 1x NVIDIA A100 40GB | 40 GB HBM2 | 12 | 85 GB | $3.67 |
| a2-highgpu-2g | 2x NVIDIA A100 40GB | 80 GB HBM2 | 24 | 170 GB | $7.35 |
| a2-highgpu-4g | 4x NVIDIA A100 40GB | 160 GB HBM2 | 48 | 340 GB | $14.69 |
| a2-highgpu-8g | 8x NVIDIA A100 40GB | 320 GB HBM2 | 96 | 680 GB | $29.39 |
| a2-ultragpu-1g | 1x NVIDIA A100 80GB | 80 GB HBM2e | 12 | 170 GB | $5.07 |
| a2-ultragpu-2g | 2x NVIDIA A100 80GB | 160 GB HBM2e | 24 | 340 GB | $10.14 |
| a2-ultragpu-4g | 4x NVIDIA A100 80GB | 320 GB HBM2e | 48 | 680 GB | $20.27 |
| a2-ultragpu-8g | 8x NVIDIA A100 80GB | 640 GB HBM2e | 96 | 1,360 GB | $40.55 |
| On-demand pricing, us-central1. | |||||
G4 Series: NVIDIA RTX PRO 6000 Blackwell
The G4 series is on of GCP's newest GPU family, built around NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs with 96 GB of GDDR7 memory per GPU. It targets professional visualization, 3D rendering, ray tracing, and AI inference workloads that benefit from the Blackwell architecture's tensor core performance.
G4 VMs are available in a range of sizes and represent GCP's most recent GPU hardware addition as of 2026.
| Instance | GPU | GPU VRAM | vCPUs | RAM | On-Demand $/hr |
|---|---|---|---|---|---|
| g4-standard-48 | 1x NVIDIA RTX PRO 6000 | 96 GB GDDR7 | 48 | 180 GB | $4.50 |
| g4-standard-96 | 2x NVIDIA RTX PRO 6000 | 192 GB GDDR7 | 96 | 360 GB | $9.00 |
| g4-standard-192 | 4x NVIDIA RTX PRO 6000 | 384 GB GDDR7 | 192 | 720 GB | $18.00 |
| g4-standard-384 | 8x NVIDIA RTX PRO 6000 | 768 GB GDDR7 | 384 | 1,440 GB | $36.00 |
| On-demand pricing, us-central1. | |||||
G2 Series: NVIDIA L4
The G2 series uses NVIDIA L4 24GB GPUs, optimized for inference, video processing, and lighter AI workloads. At $0.70/hr on-demand for a single L4, G2 is the most cost-effective GCP GPU option for models that fit within 24GB of VRAM. The Ada Lovelace architecture makes the L4 efficient at FP16 and INT8 inference.
| Instance | GPU | GPU VRAM | vCPUs | RAM | On-Demand $/hr |
|---|---|---|---|---|---|
| g2-standard-4 | 1x NVIDIA L4 24GB | 24 GB GDDR6 | 4 | 16 GB | $0.70 |
| g2-standard-8 | 1x NVIDIA L4 24GB | 24 GB GDDR6 | 8 | 32 GB | $0.84 |
| g2-standard-12 | 1x NVIDIA L4 24GB | 24 GB GDDR6 | 12 | 48 GB | $0.98 |
| g2-standard-16 | 1x NVIDIA L4 24GB | 24 GB GDDR6 | 16 | 64 GB | $1.12 |
| g2-standard-24 | 2x NVIDIA L4 24GB | 48 GB GDDR6 | 24 | 96 GB | $1.55 |
| g2-standard-48 | 4x NVIDIA L4 24GB | 96 GB GDDR6 | 48 | 192 GB | $3.10 |
| g2-standard-96 | 8x NVIDIA L4 24GB | 192 GB GDDR6 | 96 | 384 GB | $6.20 |
| On-demand pricing, us-central1, as of July 2026. Verify on the GCP GPU pricing page before budgeting. | |||||
N1 with Attached GPUs: T4, V100, and P100
The N1 series is GCP's general-purpose machine family. It supports attaching T4, V100, or P100 GPUs separately, letting you tune the CPU-to-GPU ratio for your workload. The T4 is the most cost-efficient option for inference and development. Note that NVIDIA P100 GPUs reach end of support on September 15, 2026.
| GPU (attached to N1) | VRAM | Architecture | GPU count | Cost ($/hr) | Best For |
|---|---|---|---|---|---|
| NVIDIA T4 | 16 GB GDDR6 | Turing | 1x–4x | $0.39 (1x, n1-standard-1) – $7.08 (4x, n1-standard-96) | Inference, development, budget training |
| NVIDIA Tesla P4 | 8 GB GDDR5 | Pascal | 1x–4x | $0.64 (1x, n1-standard-1) – $8.08 (4x, n1-standard-96) | Video transcoding, virtual workstations, light inference |
| NVIDIA P100 | 16 GB HBM2 | Pascal | 1x–4x | $1.50 (1x, n1-standard-1) – $11.52 (4x, n1-standard-96) | Legacy workloads (end of support Sep 2026) |
| NVIDIA V100 | 16 GB HBM2 | Volta | 1x–8x | $2.52 (1x, n1-standard-1) – $25.52 (8x, n1-standard-96) |
Legacy training workloads |
Cloud Run GPUs: Serverless GPU Inference
Cloud Run GPU lets you attach an L4 GPU to a Cloud Run instance without managing a VM. Billing is per-second with no minimum, and the container scales to zero when idle, making it cost-effective for inference endpoints with variable traffic. No quota request is required, unlike Compute Engine. It suits smaller model serving via API when persistent VM overhead is not justified.
Google Colab: Google's Notebook GPU Option
Google Colab is Google's browser-based notebook environment, backed by the same Google Cloud infrastructure. The free tier assigns a shared T4 for sessions up to 12 hours, though GPU access is not guaranteed and sessions are pre-emptible.
Google Colab GPU Tiers and Pricing
Colab's paid tiers use a compute-unit model. A T4 burns roughly 1.5 to 2.0 compute units per hour; an A100 burns 13 to 15. The table below shows approximate effective hourly costs.
| GPU | VRAM | Architecture | Compute Units/hr | Approx. $/hr | Available On |
|---|---|---|---|---|---|
| T4 | 16 GB | Turing | 1.5-2.0 | $0.15-$0.20 | Free, Pro, Pro+ |
| L4 | 22.5 GB | Ada Lovelace | 2.5-5.0 | $0.25-$0.50 | Pro, Pro+ |
| V100 | 16 GB | Volta | 5.0-6.0 | $0.50-$0.60 | Pro, Pro+ |
| A100 | 40 / 80 GB | Ampere | 13.0-15.0 | $1.30-$1.50 | Pro+ |
| Approximate effective hourly costs based on current compute-unit consumption rates. Colab does not guarantee a specific GPU model on any tier. | |||||
Colab's Limits: When to Move to a Dedicated Instance
Colab stops being practical once a project needs consistent GPU access. GPU assignment is not guaranteed, meaning you may get a T4 when you need an A100, and that can change between sessions. Sessions are pre-emptible, compute units run out unpredictably, and interrupted runs cannot be resumed mid-epoch.
Once a project needs overnight runs, reproducible hardware, or predictable billing, a dedicated GPU instance is the right move.
See a full comparison of Colab tiers against dedicated GPU alternatives.
Quick-Pick: Which GCP GPU Is Right for Your Workload?
| Workload | Recommended Instance | GPU | On-Demand $/hr |
|---|---|---|---|
| Large-scale LLM training (70B+ parameters) | a3-highgpu-8g | 8x H100 80GB | $88.48 |
| Single-GPU fine-tuning and applied AI | a2-ultragpu-1g | 1x A100 80GB | $5.07 |
| Batch inference and medium training runs | a2-highgpu-1g | 1x A100 40GB | $3.67 |
| LLM inference (7B to 13B models) | g2-standard-4 | 1x L4 24GB | $0.70 |
| Visualization, rendering, and graphics AI | g4-standard-48 | 1x RTX PRO 6000 96GB | $4.50 |
| Serverless inference (variable traffic) | Cloud Run GPU | 1x L4 24GB | Per-second billing |
| Budget experimentation and development | N1 + T4 | 1x T4 16GB | ~$0.35 + N1 rate |
| Fault-tolerant training (spot) | a2-highgpu-1g (spot) | 1x A100 40GB | ~$1.10 (spot) |
Google Cloud GPU Pricing
On-Demand GPU Pricing
On-demand pricing charges by the second (with a 1-minute minimum) and requires no commitment. The rates below cover common single-GPU configurations.
| Instance | GPU | VRAM | On-Demand $/GPU/hr |
|---|---|---|---|
| a3-highgpu-8g (per GPU) | NVIDIA H100 SXM5 | 80 GB HBM3 | $11.06 |
| a2-ultragpu-1g | NVIDIA A100 | 80 GB HBM2e | $5.07 |
| a2-highgpu-1g | NVIDIA A100 | 40 GB HBM2 | $3.67 |
| g4-standard-48 | NVIDIA RTX PRO 6000 | 96 GB GDDR7 | $4.50 |
| g2-standard-4 | NVIDIA L4 | 24 GB GDDR6 | $0.70 |
| N1 + T4 | NVIDIA T4 | 16 GB GDDR6 | ~$0.35 (GPU add-on) |
| N1 + V100 | NVIDIA V100 | 16 GB HBM2 | ~$2.48 (GPU add-on) |
| On-demand pricing, us-central1, as of July 2026. N1 GPU add-on pricing excludes the underlying N1 machine type rate. Verify on the GCP GPU pricing page before budgeting. | |||
Spot (Preemptible) Pricing: Up to 91% Off
GCP Spot VMs run on excess capacity at discounts of up to 91% off on-demand rates. Google can preempt them with approximately 30 seconds of notice, making them suitable only for fault-tolerant workloads that checkpoint frequently. An A100 40GB drops from $3.67/hr to around $1.10/hr on Spot; an H100 can fall from $11.06/hr to approximately $3.30/hr. Spot pricing varies by region and fluctuates daily.
Committed Use Discounts (CUDs): 1-Year and 3-Year
Committed Use Discounts lock in a GPU type and region for one or three years in exchange for lower rates. A 1-year CUD saves approximately 25 to 37% over on-demand; a 3-year CUD saves roughly 50 to 57% depending on the GPU family. CUDs are resource-specific: a us-central1 A100 commitment does not cover a europe-west4 workload.
CUD discounts on GPU-accelerated machines are shallower than on CPU instances. The g2-standard-4 (L4) saves only about 8% on a 1-year CUD and 11% on a 3-year CUD. For GPU instances, Spot pricing almost always delivers greater savings than CUDs when the workload can tolerate interruption.
Sustained Use Discounts: GCP's Automatic Savings
Sustained use discounts (SUDs) apply automatically when a VM runs for a significant portion of a billing month, with no commitment required. The discount reaches up to 30% for instances that run the full month, and neither AWS nor Azure offers an equivalent. SUDs apply only to N1 instances with attached GPUs; accelerator-optimized families (A3, A2, G2, G4) are not eligible.
Dynamic Workload Scheduler
Dynamic Workload Scheduler (DWS) queues batch AI and HPC jobs and starts them when capacity is available at a reduced rate. It suits non-time-critical training runs and can deliver savings beyond Spot pricing in some configurations, without the interruption risk of a preemptible VM.
Hidden Costs of GCP GPU Instances
The advertised hourly rate is not the full bill. Several cost categories stack on top of GPU compute and are easy to underestimate before the first invoice.
Storage. Persistent Disk is billed separately from the instance. Balanced Persistent Disk costs $0.10/GB/month; SSD Persistent Disk runs $0.17/GB/month. A 500GB SSD dataset adds $85/month before you run a single GPU-hour. Disk charges continue even when the VM is stopped.
Egress. GCP charges approximately $0.12/GB for data leaving its network to the internet, with the first 200GB/month free. Downloading a 100GB model checkpoint costs around $12 in egress alone. Teams serving inference externally or moving artifacts between regions accumulate these costs on top of GPU compute.
Read a full breakdown of how egress fees add up across providers.
Idle GPU time. GCP bills for provisioned time regardless of GPU utilization. A single A100 40GB left running idle for 8 hours costs $29.36 at on-demand rates.
Vertex AI surcharges. Using Vertex AI as a managed layer on top of raw Compute Engine GPU instances adds a meaningful premium to compute costs. Budget for this overhead rather than treating Compute Engine and Vertex AI rates as equivalent.
Stopped vs. terminated billing. Stopping a VM does not stop storage billing. Only deleting the persistent disk removes storage charges. Build shutdown scripts to explicitly stop the VM and detach or snapshot storage when the disk is no longer needed.
Google Cloud GPU vs AWS, Azure, and Specialized Providers
AWS cut its H100 rates by 44% in June 2025, and GCP's H100 at $11.06/hr per GPU now sits above both AWS and Azure. Specialized GPU clouds are substantially cheaper across all tiers.
| Provider | A100 80GB $/hr | H100 80GB $/hr |
|---|---|---|
| Thunder Compute | $1.09 | $2.19 |
| RunPod | $1.39 | $2.89 |
| Lambda Labs | $2.79 | $3.99 |
| AWS | $3.43 (p4de.24xlarge) | $6.88 (p5.4xlarge) |
| Azure | $4.41 (NC24ads A100 v4) | $8.30 (NC40ads H100 v5) |
| Google Cloud | $5.07 (a2-ultragpu-1g) | $11.06 (a3-highgpu-8g, per GPU) |
| On-demand, single-GPU pricing as of July 2026. AWS A100 is per-GPU from the p4de.24xlarge (8-GPU node). GCP H100 is per-GPU from the a3-highgpu-8g. Prices vary by region and change frequently. | ||
Thunder Compute's on-demand H100 rate of $2.19/hr is lower than GCP's best-case Spot price for H100, without any preemption risk or upfront commitment. There are no egress fees and persistent storage is included by default.
For a deeper look at the cost and setup differences, see the Thunder Compute vs GCP comparison.
Google Cloud GPU Cost Optimization
Spot vs CUDs vs On-Demand
- On-demand suits short or unpredictable workloads and anything where interruption is not acceptable.
- Spot is the highest-impact lever for fault-tolerant training: switching an A100 run from on-demand to Spot typically cuts the compute bill by 60-70%, provided your script checkpoints every 10 to 15 minutes.
- CUDs make sense when you have a clear 12-month projection of consistent GPU usage and the workload is not eligible for Spot.
For most training workloads, Spot beats CUDs on savings. For production inference where SLA matters, on-demand or a CUD is the right choice.
Right-Size Your Instances
Using an A3 (H100) instance for a fine-tuning job that fits in 40GB of VRAM wastes most of what you are paying for.
- G2 (L4) for models under 13B parameters at FP16,
- A2 (A100 40GB) for medium training and fine-tuning
- A3 (H100) only when you genuinely need 80GB of HBM3 or multi-node scale.
Running nvidia-smi dmon during a training job shows real-time GPU memory utilization and helps identify over-provisioning.
Sustained Use Discounts vs Committed Use
For N1 instances with attached GPUs, SUDs kick in automatically once the instance runs more than 25% of a billing month and grow the longer it runs. This suits long-running inference servers or persistent development environments.
SUDs do not apply for A3, A2, and G2.
Minimize Egress Costs
- Keep training data and model artifacts in the same GCP region as your GPU VMs to avoid cross-region transfer fees.
- Use Cloud Storage for dataset storage rather than pulling from external sources repeatedly.
- For inference workloads serving external users, a CDN or response caching layer reduces data leaving GCP per request.
- Monitor "Network Egress" in your billing dashboard by filtering exports for SKUs with "Network Egress."
When Google Cloud GPU Makes Sense (and When It Doesn't)
GCP is the right choice for teams embedded in the Google Cloud ecosystem. If your data lives in BigQuery, your pipelines run on GKE, and your ML platform is Vertex AI, keeping GPU compute on GCP is simpler.
GCP is also the only provider with TPUs, which matters for JAX workloads at hyperscale. Moreover, automatic sustained use discounts on N1 GPU instances are a genuine differentiator for continuous workloads that do not want multi-year commitments.
GCP is a harder choice when GPU compute is the primary cost. On-demand H100 rates at $11.06/hr are the highest among the major hyperscalers. The quota system adds days to first-launch timelines. Egress fees and billing complexity across machine types, committed use contracts, storage tiers, and Vertex AI surcharges make cost prediction difficult before the first invoice.
For workloads where cost and setup speed matter most, specialized GPU providers offer the same NVIDIA hardware at 3 to 5 times lower cost, with no quota delays, no egress fees, and no managed-service overhead.
Last Thoughts on Google Cloud GPU Instances
GCP's GPU lineup covers every major NVIDIA accelerator, from the T4 for budget inference to the H100 for large-scale training, and its ecosystem integrations with Vertex AI, GKE, and BigQuery are genuinely useful. The cost of that ecosystem is steep on-demand pricing, quota friction, and egress fees.
Teams that need the GCP stack should use it; teams that need GPU compute without it should compare the alternatives before committing.
FAQ
What GPUs are available on Google Cloud?
GCP offers NVIDIA H100 80GB (A3 series), A100 40GB and 80GB (A2 series), L4 24GB (G2 series), RTX PRO 6000 96GB (G4 series), and T4, V100, and P100 as attachable GPUs on N1 instances. GCP also offers TPUs as a separate accelerator class.
How much does a GPU cost on Google Cloud per hour?
On-demand pricing in us-central1 ranges from ~$0.35/hr for a T4 add-on to $11.06/hr per GPU on the a3-highgpu-8g. A100 40GB is $3.67/hr; A100 80GB is $5.07/hr; L4 is $0.70/hr. Spot pricing cuts those rates by 60 to 91%.
How do I get GPU quota on Google Cloud?
Navigate to IAM and Admin, then Quotas in the Cloud Console, filter for the GPU type and region you need, and submit an increase request. Common GPU quotas are approved within a few hours to a couple of days. A3 (H100) quota can take longer.
What is the difference between Spot and Preemptible VMs on GCP?
Spot VMs and preemptible VMs work the same way: both run on excess capacity at deep discounts and can be interrupted with approximately 30 seconds of notice. Spot VMs are the current naming convention with no maximum runtime limit; the older preemptible type had a 24-hour cap.
Does Google Cloud charge egress fees for GPU workloads?
Yes. GCP charges approximately $0.12/GB for internet egress, with the first 200GB/month free. GPU workloads that download model checkpoints, serve inference externally, or move data between regions accumulate these costs on top of the hourly GPU rate.
What is the difference between Google Cloud GPU instances and Google Colab?
Colab offers shared GPU access through a compute-unit model with no guaranteed GPU type, pre-emptible sessions, and unpredictable availability. Compute Engine GPU instances are dedicated VMs with a fixed GPU, full environment control, and per-second billing with no session limits.
Is Google Cloud cheaper than AWS or Azure for GPU instances?
No, not on-demand. GCP's H100 at $11.06/hr per GPU is higher than AWS ($6.88/hr) and Azure ($8.30/hr). GCP's advantage is its automatic sustained use discounts on N1 GPU instances, which neither AWS nor Azure offers.
When should I use Cloud Run GPUs instead of Compute Engine GPUs?
Use Cloud Run GPU for inference with variable traffic. It scales to zero, bills per second of actual request processing, and requires no VM management. Use Compute Engine for training, fine-tuning, or any workload needing persistent storage or full OS control.
Can I run a single H100 GPU on Google Cloud on-demand?
Not in the standard sense. The a3-highgpu-1g, -2g, and -4g configurations must be provisioned as Spot VMs or Flex-start VMs. The a3-highgpu-8g is the only A3 variant available on demand, billed at $88.48/hr for all 8 GPUs.
Do sustained use discounts apply to A100 and H100 instances on GCP?
No. Sustained use discounts apply only to N1 instances with attached GPUs (T4, V100, P100). Accelerator-optimized families including A3, A2, G2, and G4 are not eligible. For those series, Spot pricing or Committed Use Discounts are the main savings options.