NVLink is NVIDIA's high-speed GPU-to-GPU interconnect. It matters enormously for large multi-GPU training and very little for single-GPU work. An eight-GPU H100 SXM server can move data between GPUs at 900 GB/s, while a PCIe-based machine is capped near 128 GB/s.
That bandwidth gap decides whether some multi-GPU training jobs are practical. It is also why NVLink comes up constantly when people rent GPUs. Single-GPU inference, smaller fine-tunes, and batch jobs run the same on PCIe, so paying a premium added bandwidth wastes money.
What Is NVLink?
NVLink is NVIDIA's proprietary high-speed interconnect for direct GPU-to-GPU communication. Instead of routing traffic through the shared PCIe bus, NVLink creates a dedicated path between GPUs inside the same server or rack. That dedicated path gives NVLink several times the bandwidth of PCIe.
On a typical server, PCIe is the general-purpose road shared by the CPU, storage, and every GPU. NVLink is a private lane between the GPUs. NVLink carries no contention from other devices and no CPU arbitration, so it delivers higher bandwidth and lower latency between GPUs.
NVLink also links CPUs to GPUs on NVIDIA's Grace-based systems, where an NVLink-C2C (chip-to-chip) connection joins the Grace CPU and GPU at 900 GB/s. On mainstream x86 servers, the CPU still reaches the GPUs over PCIe, so NVLink stays between the GPUs.
NVLink is intra-node only, meaning it connects GPUs within a single server or rack, not across separate machines. Communication between separate nodes travels over InfiniBand or Ethernet instead.
How NVLink Works
NVLink is a point-to-point serial link built from multiple high-speed lanes. A single GPU can carry several links at once. Because each device holds multiple NVLinks, small groups of GPUs form a direct mesh instead of routing everything through one central hub. Direct meshing separates NVLink from PCIe, where devices connect through a shared root complex.
NVLink supports cache coherence across connected GPUs, letting them share a unified memory pool. One GPU can read another GPU's memory directly, without copying data through system RAM. For memory-bound multi-GPU work, that coherent access is much of NVLink's advantage.
NVLink Bandwidth by Generation
NVLink bandwidth has roughly doubled with every generation, climbing from 160 GB/s on Pascal to 1.8 TB/s on Blackwell. Earlier generations added bandwidth by adding more links. NVLink 5.0 kept 18 links and doubled per-link speed instead.
| Generation | GPU Architecture | Year | Total Bandwidth per GPU (Bidirectional) |
|---|---|---|---|
| NVLink 1.0 | Pascal (P100) | 2016 | 160 GB/s |
| NVLink 2.0 | Volta (V100) | 2017 | 300 GB/s |
| NVLink 3.0 | Ampere (A100) | 2020 | 600 GB/s |
| NVLink 4.0 | Hopper (H100, H200) | 2022 | 900 GB/s |
| NVLink 5.0 | Blackwell (B200, GB200) | 2024 | 1.8 TB/s |
| NVLink 6.01 | Rubin (R100) | 2026+ | ~3.6 TB/s |
| 1NVLink 6.0 figures are based on NVIDIA roadmap data. The Rubin generation is not yet shipping. | |||
NVLink vs PCIe: When the Interconnect Actually Matters
NVLink delivers roughly 7x the bandwidth of PCIe Gen5 on Hopper and about 14x on Blackwell, but that advantage only helps workloads with heavy GPU-to-GPU traffic. NVLink and PCIe solve the same problem at different bandwidth tiers and price points.
| Dimension | NVLink 4.0 (H100) | NVLink 5.0 (B200) | PCIe Gen5 x16 |
|---|---|---|---|
| Bidirectional bandwidth | 900 GB/s | 1.8 TB/s | 128 GB/s |
| GPU-to-GPU latency | ~1 microsecond | ~1 microsecond | ~4 to 8 microseconds |
| Topology | All-to-all via NVSwitch | All-to-all via NVSwitch | Shared PCIe root complex |
| CPU involvement | None | None | DMA through PCIe |
The gap matters most in multi-GPU training. Splitting a model across GPUs transfers activations on every forward pass, and an all-reduce sums gradients after every step. At 900 GB/s these transfers take microseconds. At PCIe Gen5's 128 GB/s they take several times longer and show up in step time.
For workloads with little GPU-to-GPU traffic, PCIe is not a bottleneck. A model that fits on one GPU never crosses the interconnect, so NVLink offers no benefit. The form factor decision depends greatly on the workload.
NVSwitch: Scaling NVLink Beyond Eight GPUs
NVSwitch is a dedicated switch chip that turns NVLink into a full all-to-all fabric. In plain terms, every GPU in the node can reach every other GPU directly, at full speed, at the same time. NVLink on its own is just the link between GPUs. Without a switch, GPUs connect only in pairs or rings, so traffic hops through other GPUs to reach its destination, which slows group operations.
On an eight-GPU H100 SXM node, four NVSwitch chips create a non-blocking crossbar. Every GPU can send 900 GB/s to every other GPU at the same time, with none waiting on the others. Rack-scale NVSwitch goes further: a GB200 NVL72 connects 72 Blackwell GPUs into roughly 130 TB/s of all-to-all bandwidth, behaving like one large GPU.
PCIe systems cannot replicate NVSwitch. PCIe GPUs communicate through the shared root complex, so they lack the non-blocking, all-to-all behavior NVSwitch provides on SXM systems.
Do You Actually Need NVLink?
Most AI workloads do not need NVLink, and assuming otherwise is a common way to overpay. NVLink matters only when activations or gradients move between GPUs on a tight latency budget.
| Workload | NVLink Needed? | Why |
|---|---|---|
| Pre-training large models (70B and up, multi-GPU) | Yes | All-reduce runs every step, and PCIe bandwidth becomes the bottleneck at scale |
| Multi-GPU fine-tuning with FSDP or tensor parallelism (roughly 13B and up) | Usually | Activations and gradients move between GPUs frequently |
| Single-GPU fine-tuning or inference | No | The GPU works alone, so there is no inter-GPU traffic |
| Inference on models that fit one GPU | No | No cross-GPU communication is involved |
| Batch embedding or classification | No | Work is independent per sample |
Workloads That Need NVLink
Large-scale distributed training is the clearest case for NVLink. A model sharded across 4 or 8 GPUs with tensor or pipeline parallelism moves traffic on every step, and PCIe overhead can consume a large share of each training step on large models. Multi-GPU inference with a shared KV cache also needs NVLink, since the cache must stay accessible across the node.
Workloads That Run Fine on PCIe
Any workload where the GPU operates alone runs the same on PCIe:
- Single-GPU inference
- Fine-tuning a model that fits in one GPU's memory
- Batch embedding or classification jobs
- Most day-to-day development and experimentation
For these workloads, an SXM instance adds cost without adding speed.
How to Check Whether Your Instance Has NVLink
You can confirm NVLink status directly on any instance. Start with link status, which reports the speed of each active link:
# Check NVLink link status and speed per link
nvidia-smi nvlink -s
If NVLink is active, the output lists a speed for each link. An all-inactive result on a multi-GPU instance means the GPUs are connected over PCIe rather than NVLink. To see the topology directly, check the GPU-to-GPU matrix:
# Check GPU-to-GPU topology
nvidia-smi topo -m
Check the cells between GPU pairs. An entry like NV4 or NV18 means the pair shares that many NVLink links; SYS means the connection runs through the PCIe root complex with no NVLink.
In virtualized cloud environments like Thunder Compute,
topo -mmay fail to build the matrix, in which casenvlink -sor a peer-to-peer bandwidth test is the reliable check.
Running GPU Workloads on Thunder Compute
Thunder Compute offers cost-efficient A100 and H100 instances that fit the large majority of workloads that do not need NVLink. A single-GPU fine-tune or inference job runs the same on these instances as on far more expensive hardware, without paying for interconnect bandwidth it will not use. Instances come in 1x, 2x, 4x, and 8x configurations with per-minute billing, 100GB storage per GPU, and no egress fees.
| GPU | On-Demand Price |
|---|---|
| RTX A6000 | $0.35/hr |
| L40 | $0.79/hr |
| A100 80GB | $1.09/hr |
| H100 80GB | $2.19/hr |
Thunder Compute integrates with VS Code, Cursor, and Windsurf extensions, so you can launch a GPU and run the nvidia-smi checks without leaving your editor. To persist an environment across sessions, snapshot an instance and relaunch it with a different configuration.
One exception applies: jobs that genuinely need NVLink-class bandwidth for large distributed training. An SXM plus InfiniBand setup is the right tool for those, and a provider built around that hardware serves them better. For where that line falls, see Thunder Compute's comparison of single-node and distributed-training options.
Run the
nvidia-smichecks on a live instance before paying an SXM premium you may not need, then compare Thunder Compute GPU pricing.
Last Thoughts on NVLink
NVLink matters enormously for large multi-GPU training and very little for everything else. It is only worth paying for when your workload moves heavy traffic between GPUs. For jobs that run on a single GPU or tolerate PCIe bandwidth, the SXM premium buys nothing. Check your topology first and match the hardware to the job. When a PCIe GPU fits, you can spin up an A100 or H100 on Thunder Compute at a fraction of SXM pricing.
FAQ
What is NVLink in simple terms?
NVLink is NVIDIA's high-speed connection for moving data directly between GPUs. It replaces the shared PCIe path with a dedicated link, giving GPUs several times more bandwidth inside a single server or rack.
What is the difference between NVLink and PCIe?
NVLink is a dedicated GPU-to-GPU link; PCIe is a general-purpose bus shared by the CPU, storage, and GPUs. NVLink offers 7-14x the bandwidth of PCIe Gen5, but only matters for workloads moving large data volumes between GPUs.
What is NVSwitch?
NVSwitch is the switch chip that connects many NVLinks into a full all-to-all fabric. On an eight-GPU H100 node, every GPU reaches every other GPU at full NVLink speed simultaneously; at rack scale it links dozens of GPUs.
Do I need NVLink for AI inference?
Usually not. A model that fits on a single GPU never crosses the interconnect, so PCIe is enough. NVLink matters for inference only when a large KV cache or tensor-parallel model spans multiple GPUs on one node.
How much bandwidth does NVLink have?
It depends on the generation. NVLink 4.0 on Hopper (H100) delivers 900 GB/s per GPU; NVLink 5.0 on Blackwell (B200) delivers 1.8 TB/s. Both far exceed PCIe Gen5's 128 GB/s.
How do I check if my GPU has NVLink?
Run nvidia-smi nvlink -s to see per-link status, and nvidia-smi topo -m to view the topology matrix. Entries beginning with NV indicate an NVLink connection; SYS means the GPUs connect over PCIe.
Does NVLink work between servers?
No. NVLink is intra-node only, connecting GPUs within one server or rack. Communication between separate nodes travels over InfiniBand or Ethernet instead.
Which NVIDIA GPUs support NVLink?
SXM data center GPUs support full NVLink and NVSwitch fabric, including A100 SXM, H100 SXM, H200, and B200. PCIe cards connect over the PCIe root complex and do not join an NVSwitch fabric.