The NVIDIA L40 and L40S share the same AD102 die family, 48 GB GDDR6 ECC, and PCIe form factor. The key difference is the Transformer Engine: the L40S ships with it enabled, unlocking automatic FP8/FP16 per-layer switching in vLLM and doubling FP8 throughput to 733 TFLOPS dense.
For teams running production LLM inference on 7B-34B models at batch size 8 or above, that difference moves the cost-per-token needle. For rendering, VDI, and low-concurrency inference where FP8 is not the bottleneck, the L40's lower hourly rate is the more relevant spec.
Takeaways
- The L40 and L40S differ in clock configuration and Transformer Engine availability, despite sharing the same CUDA core count, memory, and bandwidth.
- The L40S has 2x the FP8 throughput (733 vs 362 TFLOPS dense) and the Transformer Engine for automatic FP8/FP16 switching in vLLM via a single flag.
- The L40 wins on cost for image generation, VDI, and low-batch inference where the Transformer Engine is not the bottleneck.
Quick Comparison
| Specification | NVIDIA L40 | NVIDIA L40S |
|---|---|---|
| Architecture | Ada Lovelace (AD102) | |
| Process Node | TSMC 4N | |
| CUDA Cores | 18,176 | |
| Tensor Cores | 568 (4th Gen) | |
| GPU Memory | 48 GB GDDR6 ECC | |
| Memory Bandwidth | 864 GB/s | |
| Transformer Engine | No | Yes |
| FP16 Tensor Performance | 181 TFLOPS | 362 TFLOPS |
| FP8 Tensor Performance (dense) | 362 TFLOPS | 733 TFLOPS |
| FP8 Tensor Performance (sparse) | 724 TFLOPS | 1,466 TFLOPS |
| AI TOPS | 362 | 733 |
| NVLink | Not supported | |
| MIG Support | Not supported | |
| PCIe Interface | Gen 4 x16 | |
| Power (TBP) | 300 W | 350 W |
| Launch Date | Oct 2022 | Aug 2023 |
| Thunder Compute pricing | from $0.79/hr | Not offered1 |
| Sources: NVIDIA L40S Specs; NVIDIA L40 Datasheet. 1 For L40S cloud options, see the pricing table below. | ||
Why the L40S Outperforms: Same Silicon, Different Configuration
Both GPUs use the NVIDIA AD102 die manufactured on TSMC's 4N process, but they are different die revisions: the L40 uses AD102-895 and the L40S uses AD102-896. The two chip variants share the same CUDA core count (18,176), Tensor Cores, RT cores, and memory subsystem, but differ in clock configuration. The L40's boost clock is 2,490 MHz; the L40S reaches 2,520 MHz, with a notably higher base clock (1,065 MHz vs 735 MHz).
The L40S is NVIDIA's data center re-positioning of Ada Lovelace, purpose-configured for AI inference with the Transformer Engine enabled and a higher power budget. The same die family ships in both cards, but the revision, clocks, and firmware differ.
The Transformer Engine dynamically switches between FP8 and FP16 precision per layer during inference, using per-tensor scaling factors computed at runtime. The L40 has 4th-gen Tensor Cores that support FP8 at the hardware level but lacks the Transformer Engine software framework. FP8 deployment on the L40 requires manual per-layer configuration and is less reliable in production inference pipelines, particularly when using vLLM's --dtype fp8 flag, which routes automatically to the Transformer Engine on the L40S.
FP8 vs FP16 Throughput Explained
FP8 uses half the memory of BF16 (Brain Float 16) per operand, so Tensor Cores process roughly twice as many values per cycle. On the L40S this yields 733 TFLOPS FP8 dense versus 362 TFLOPS FP16 dense, reaching 1,466 TFLOPS with structured sparsity. The L40's FP8 figure (362 TFLOPS dense) matches the L40S's FP16 figure exactly. In practice, the L40S FP8 path delivers roughly 1.5-2x the token throughput of the L40S FP16 path at equivalent batch sizes.
Power Consumption
The L40 draws 300W; the L40S draws 350W. The 50W premium is the direct trade-off for higher compute throughput. For large GPU fleets, 8x L40 GPUs running 24/7 consume roughly 9.6 kWh/day less than 8x L40S, saving approximately $60/month at typical US data center electricity rates.
L40 Inference Benchmark: Tokens Per Second by Batch Size
At batch sizes 1-8, the performance gap between L40 and L40S is driven almost entirely by FP8 compute advantage, not memory bandwidth. At batch sizes 16 and above, both GPUs become bandwidth-bound and the gap narrows.
The table below shows representative token throughput for Llama 3.1 8B per GPU via vLLM, based on controlled Ada Lovelace hardware benchmarks and hardware TFLOPS ratios (note: L40 FP16 figures are estimated from TFLOPS ratios, not directly measured on the L40).
| GPU | Precision | Batch Size | Tokens/sec (approx.) |
|---|---|---|---|
| L40 | FP16 | 1 | 35 |
| L40 | FP16 | 8 | 168 |
| L40S | FP16 | 1 | 46 |
| L40S | FP16 | 8 | 336 |
| L40S | FP8 | 1 | 70 |
| L40S | FP8 | 8 | 504 |
| Approximate figures from Ada Lovelace vLLM benchmarks. Actual throughput varies with model, sequence length, KV cache utilization, and system configuration. | |||
At batch size 8, the L40S FP8 path delivers roughly 3x the throughput of the L40 FP16 path. For production inference endpoints serving concurrent users, that gap directly affects latency, queue depth, and the number of instances needed.
L40 vs L40S: Cost Per Million Tokens
Hourly rate comparisons are incomplete without accounting for throughput. The L40S costs more per hour but generates more tokens per hour, making it cheaper per 1M output tokens for workloads where FP8 is effective.
The table below shows cost per 1M tokens for Llama 3.1 8B inference on Thunder Compute's L40 at the base rate of $0.79/hr (1x GPU, base vCPU/RAM config), compared against indicative L40S market rates:
| GPU | Precision | Batch Size | Tokens/sec | Rate ($/hr) | Cost / 1M Tokens |
|---|---|---|---|---|---|
| L40 (Thunder) | FP16 | 1 | 35 | $0.79 | $6.27 |
| L40 (Thunder) | FP16 | 8 | 168 | $0.79 | $1.31 |
| L40S (market avg) | FP16 | 1 | 46 | $0.99 | $5.98 |
| L40S (market avg) | FP16 | 8 | 336 | $0.99 | $0.82 |
| L40S (market avg) | FP8 | 8 | 504 | $0.99 | $0.55 |
| Cost per 1M tokens = (hourly rate / tokens per second) x (1,000,000 / 3,600). Throughput figures are approximate. Thunder L40 rate shown is the base 1x GPU configuration ($0.79/hr); higher vCPU/RAM configs run up to $1.03/hr. Market average L40S rate based on on-demand pricing from major providers as of August 2026. | |||||
At batch size 8 in FP16, the L40 on Thunder delivers approximately $1.31 per 1M tokens at the base rate. The L40S at FP8 reaches approximately $0.55 per 1M tokens at market rates. For lower-concurrency workloads where batch size stays below 4, Thunder's L40 starting rate keeps cost-per-token competitive.
Check current L40 pricing on Thunder Compute
LLM Training and Fine-Tuning Performance
The L40S delivers approximately 2x the FP16 throughput (362 vs 181 TFLOPS) and 2x the FP8 throughput (733 vs 362 TFLOPS dense) on transformer workloads. Without the Transformer Engine, the L40's FP8 must be configured manually per layer, making the L40S the more practical choice for teams using vLLM or TensorRT-LLM in production. Neither GPU supports NVLink, so for large-scale multi-GPU training, A100 or H100 SXM with NVLink are the better options.
LoRA Fine-Tuning on L40 and L40S
Both GPUs have 48 GB GDDR6, making them well-suited to LoRA and QLoRA fine-tuning of models up to 30B parameters. A 7B or 13B model in FP16 fits on either GPU with room for optimizer states and LoRA adapter weights. QLoRA (Quantized LoRA) at 4-bit brings 30B-class models within reach of a single card.
The L40S completes comparable fine-tuning runs in roughly half the time at BF16, and the total job cost is often lower despite the higher hourly rate. For budget-first teams or short experimental runs, the L40 from $0.79/hr is the practical starting point.
FP8 mixed-precision training via the Transformer Engine is technically supported on the L40S but remains experimental in most standard frameworks as of mid-2026. Most production fine-tuning on L40S uses BF16.
L40S vs A100 for Inference
The L40S and A100 sit in different inference regimes depending on batch size. At batch size 1, the A100's approximately 2 TB/s memory bandwidth (more than double the L40S's 864 GB/s) gives it roughly 2x the token throughput in memory-bound single-request inference. At batch size 8 and above, the L40S FP8 path with the Transformer Engine makes it competitive with or faster than the A100 on a cost-per-token basis.
The A100 also supports NVLink and MIG partitioning, which neither Ada Lovelace GPU offers. For multi-GPU distributed training or multi-tenant MIG deployments, the A100 is the better fit. For single-GPU FP8 inference on 7B-34B models at moderate batch sizes, the L40S delivers comparable throughput at a lower hourly rate than most A100 cloud options.
For a full breakdown of when to choose each, see the A100 vs H100 comparison and the GPU selection guide for AI workflows.
Which Models Run on the L40 and L40S
Both GPUs carry 48 GB GDDR6 ECC, so the VRAM ceiling is identical. The L40S can serve quantized models more efficiently via the Transformer Engine. The table below maps common models to each GPU:
| Model | L40 (48 GB) | L40S (48 GB) | Notes |
|---|---|---|---|
| Llama 3.1 8B (FP16, ~16 GB) | Yes | Yes | Both comfortable; L40S faster at FP8 |
| Mistral 7B (FP16, ~14 GB) | Yes | Yes | Both comfortable; L40S faster at FP8 |
| Llama 3.1 13B (FP16, ~26 GB) | Yes | Yes | Both fit; L40S preferred for throughput |
| Qwen 2.5 32B (FP16, ~64 GB) | No | No | Requires INT4 (~18 GB) or multi-GPU |
| Llama 3.1 70B (FP16, ~140 GB) | No | No | Requires INT4 (~35 GB) on single card |
| Llama 3.1 70B (INT4, ~35 GB) | Yes | Yes | ~13 GB left for KV cache; limits batch size |
| Stable Diffusion XL (~7 GB) | Yes | Yes | Both comfortable; negligible throughput difference |
Llama 3.1 70B fits on a single card at INT4 but leaves only approximately 13 GB for KV cache, limiting batch size and context length. For 70B models at production scale, two-GPU setups are more practical. Multi-GPU communication runs over PCIe Gen 4 on both GPUs, which is sufficient for inference but less efficient than NVLink for large distributed training.
Workload Decision Guide
| Workload | Best Choice | Why |
|---|---|---|
| LLM inference (7B-34B, FP8 via vLLM) | L40S | Transformer Engine makes FP8 plug-and-play; approximately 2-3x throughput vs L40 FP16 |
| LLM inference (7B-34B, FP16, low batch) | L40 | Lower hourly rate; competitive cost-per-token at batch sizes 1-4 |
| Fine-tuning (LoRA, 7B-30B) | L40S | Approximately 2x faster wall-clock time; lower total cost per run where FP8 is effective |
| Generative image / Stable Diffusion XL | L40 | Throughput difference is negligible; lower hourly rate wins on cost per image |
| 3D rendering / Omniverse / VDI | L40 | RT core and visualization driver optimizations; L40S tuned away from graphics |
| Multi-GPU training (70B+) | Neither ideal | No NVLink on either; consider A100 or H100 SXM for large-scale training |
| Budget experimentation and prototyping | L40 | Lower hourly rate; sufficient for testing and iteration cycles |
L40S Cloud Pricing Comparison
The L40S is available across several cloud providers at on-demand rates. Thunder does not currently offer the L40S. For teams that need the Transformer Engine and FP8 throughput, the providers below carry it on-demand with no minimum commitment.
| Provider | L40 / hr | L40S / hr | Notes |
|---|---|---|---|
| Thunder Compute | from $0.79 | Not offered | Per-minute billing, storage included; 1x GPU base config |
| RunPod | ~$0.79 | $0.79 | Secure Cloud on-demand; per-second billing |
| Vast.ai | ~$0.65* | ~$0.72* | Marketplace; rates vary by host |
| AWS (G6e) | N/A | from $1.86/hr2 | g6e instances; CPU/RAM/storage bundled |
| Pricing as of August 2026. * Vast.ai is a marketplace; rates vary by host and region. 2 AWS g6e.xlarge (1x L40S) starts at approximately $1.86/hr in us-east-1 as of August 2026; bundled with CPU and RAM. | |||
For workloads where the Transformer Engine is not required, the Thunder L40 starting from $0.79/hr is at or below most on-demand L40S options on cost per hour and often on cost per token.
Not sure which GPU fits your workload? See the Thunder Compute GPU selection guide for AI workflows.
Renting the L40 on Thunder Compute
Thunder Compute offers the NVIDIA L40 from $0.79/hr on-demand with per-minute billing and no minimum commitment. The VS Code and Cursor extensions connect directly from your IDE, so you can start running inference or fine-tuning jobs without configuring SSH or remote containers. Instance snapshots let you save and restore your exact environment at any time.
See L40 availability and pricing on Thunder Compute →
Last Thoughts on L40 vs L40S
The Transformer Engine is the only thing separating these two GPUs, but it is a meaningful one for FP8 inference at batch size 8 and above. For image generation, VDI, rendering, and low-concurrency FP16 inference, the L40 from $0.79/hr on Thunder Compute is the more cost-efficient option and is at or below most on-demand L40S rates elsewhere.
FAQ
What Is the Difference Between the NVIDIA L40 and L40S?
The primary difference is the Transformer Engine. Both are AD102-based GPUs with 48 GB GDDR6, but they use different die revisions (AD102-895 vs AD102-896). The L40S has the Transformer Engine enabled, doubling FP8 throughput to 733 TFLOPS dense versus 362 TFLOPS on the L40, and draws 350W versus 300W.
Do the L40 and L40S Use the Same GPU Silicon?
Both use AD102 silicon on TSMC's 4N process, but they are different die revisions: L40 is AD102-895, L40S is AD102-896. They share the same CUDA core count and memory subsystem but differ in base clock (735 vs 1,065 MHz), boost clock (2,490 vs 2,520 MHz), TDP (300W vs 350W), and Transformer Engine availability. They are closely related but not identical silicon.
Does the L40 Support FP8?
Yes, via its 4th-gen Tensor Cores at 362 TFLOPS dense. The L40 lacks the Transformer Engine framework, so FP8 deployment requires more manual configuration compared to the L40S, where vLLM's --dtype fp8 flag handles precision switching automatically.
Is the L40S Worth the Premium Over the L40?
For AI inference at batch size 8 or higher, often yes. The L40S FP8 throughput advantage can reduce cost per 1M tokens even at a higher hourly rate. For lighter workloads, rendering, or VDI, the L40 is the better value.
Do the L40 or L40S Support NVLink or MIG?
No. Both GPUs use PCIe Gen 4 x16 only and do not support NVLink or MIG. For distributed training at 70B+ scale, A100 SXM or H100 SXM with NVLink are more efficient.
Where Can I Rent an NVIDIA L40 GPU?
Thunder Compute offers the NVIDIA L40 from $0.79/hr on-demand with no commitments, per-minute billing, and VS Code and Cursor extensions that connect directly from your IDE.
Which Models Run on the L40 and L40S?
Both GPUs have 48 GB GDDR6, so the VRAM ceiling is identical. Models up to roughly 30B parameters fit comfortably in FP16. Llama 3.1 70B requires INT4 quantization on a single card, leaving approximately 13 GB for KV cache.
How Does the L40 Compare to the L40S for Image Generation?
For Stable Diffusion XL and similar workloads, throughput is nearly identical on both GPUs. The L40's lower hourly rate makes it the more cost-efficient choice for image generation pipelines.
At What Batch Size Does the L40S Become Cheaper Per Token Than the L40?
At batch size 8 with FP8 enabled, the L40S costs approximately $0.55 per 1M tokens versus $1.31 on the L40 at FP16. For batch sizes below 4, the L40's lower hourly rate keeps it competitive on cost per token.
How Does the L40S Compare to the A100 for Inference?
At batch size 1, the A100's approximately 2 TB/s memory bandwidth gives it roughly 2x the throughput of the L40S in memory-bound single-request inference. At batch size 8 and above, the L40S FP8 path with the Transformer Engine becomes competitive on cost per token. The A100 also supports NVLink and MIG, making it better suited for multi-GPU training and multi-tenant deployments.
Can I Use the L40S for FP8 Training?
FP8 mixed-precision training via the Transformer Engine is technically supported on the L40S but remains experimental in most standard frameworks as of mid-2026. Most production fine-tuning on the L40S uses BF16. For inference, FP8 via vLLM's --dtype fp8 flag is stable and widely used.
What Is the Transformer Engine and Why Does It Matter?
The Transformer Engine dynamically switches between FP8 and FP16 precision per layer during inference using per-tensor scaling factors computed at runtime. On the L40S, this works automatically with vLLM's --dtype fp8 flag. The L40 has the same FP8-capable Tensor Cores but lacks this framework, requiring manual per-layer configuration that is less reliable in production pipelines.
How Much Power Does the L40S Use Compared to the L40?
The L40 draws 300W; the L40S draws 350W. For large GPU fleets running 24/7, 8x L40 GPUs consume roughly 9.6 kWh/day less than 8x L40S, saving approximately $60/month at typical US data center electricity rates.