Mistral Small 4 folded four capabilities into one model which the company previously split across separate products. The 119B checkpoint handles instruction following, reasoning, vision, and agentic coding. Only 6.5B of those parameters activate per token, so Mistral Small 4 answers at the speed of a much smaller model.
Mistral Large 3 is the more capable model. Released in December 2025, this 675B Mixture of Experts model activates 41B parameters per token, versus Small 4's 6.5B, and carries a 256K context window. The larger active count buys stronger reasoning at a higher cost per token and a much heavier memory footprint.
This guide covers which Mistral models are worth self-hosting, how much memory each one needs at different quantization levels, and how self-hosting costs compare against Mistral's own API.
The Current Mistral Lineup and What Each Model Needs
Only the top two Mistral tiers require cloud GPUs. The current small tier is the Ministral 3 family at 3B, 8B, and 14B, alongside older releases like Mistral 7B that many teams still run.
Mistral Large 3 arrived in December 2025 as a 675B-parameter Mixture of Experts model with 41B active per token. It ships under Apache 2.0 with a 256K context window.
Mistral Small 4 followed in March 2026, consolidating separate instruction, reasoning, vision, and coding products into a single 119B MoE checkpoint, also under Apache 2.0.
| Model | Architecture | Approximate VRAM1 | Self-Host Hardware |
|---|---|---|---|
| Ministral 3 (3B / 8B / 14B) | Dense | ~2-9GB at Q4 | Runs locally, no cloud GPU needed |
| Mistral Small 24B / Codestral | 24B dense | ~13GB at Q4, ~24GB at Q6+ | Fits a single consumer or workstation GPU |
| Mistral Small 4 | 119B MoE, 6.5B active | ~111GB at FP8, ~60-75GB at Q4 | 2x A100 |
| Mistral Medium 3.5 | 128B dense | ~128GB at FP8, ~256GB at BF162 | 2-4x A100 |
| Mistral Large 3 | 675B MoE, 41B active | ~355GB at INT4, ~710GB at FP8 | 8x A100, 8x H200 |
| 1 Mistral does not publish per-quantization VRAM tables. These are community estimates for model weights only before KV cache.
2 Mistral Medium 3.5 ships under a Modified MIT license that permits commercial use but carries exceptions for companies above a revenue threshold. |
|||
The practical dividing line sits between Mistral Small 24B and Mistral Small 4. Below that line, self-hosting is a single-card project. Above that line, you need multiple GPUs and the cost math starts to matter.
Why "6.5B Active" Still Needs Multiple GPUs
Mixture of Experts models size their VRAM requirement by total parameters, not active parameters. Mistral Small 4 uses 128 experts and activates 4 of them per forward pass, which produces the 6.5B active figure. The router cannot choose between experts it has not loaded, so all 128 experts stay resident in memory.
Two separate numbers do two separate jobs. Active parameters determine inference speed and compute cost per token, which is why Mistral Small 4 runs as fast as a small dense model. Total parameters determine how many GPUs you rent.
The name can be misleading as Mistral Small 4 is not a bigger version of Mistral Small 24B. It is a 119B model that happens to run at small-model speed.
Mistral Large 3 follows the same rule at a larger scale. Its 41B active parameters make it cheaper per token than a 675B dense model. But you still need enough VRAM for all 675B parameters before the first token generates.
Mistral Small 4 VRAM Requirements
Mistral Small 4 needs roughly 111GB of VRAM for weights in FP8. This means you need an H200 at 141GB or a B200 at 192GB to load the weights on a single card. And even then 141GB leaves little room for KV cache. Community 4-bit builds land in the 60-75GB range depending on quantization method.
| Format | Approximate Weight Size | Fits On | Notes |
|---|---|---|---|
| Community GGUF Q4 | ~60-75GB | 1x A100 or H100 | llama.cpp path, minimal context headroom |
| FP8 | ~111GB | 2x A100, or 1x H200 | Near-lossless, Mistral documents TP=2 |
| BF16 | ~238GB | 4x H100, or 2x H200 | Full precision, rarely necessary |
Mistral's own model card serves Mistral Small 4 with tensor parallelism across two GPUs, which reflects what Mistral tested rather than a hard memory floor. On H100s, the FP8 weights genuinely need both. A single H100 or A100 can host a community 4-bit GGUF build through llama.cpp, so treat single-GPU serving as an experiment to validate rather than a default.
KV cache sits on top of the weights, though Mistral Small 4 is better behaved than most models its size. It uses Multi-head Latent Attention (MLA), which compresses the cache considerably compared with standard attention. The cache is still a real allocation at 256K context. Cap context lower on tight configurations and add VRAM for long-document workflows.
Consumer hardware does not work well for Mistral Small 4. On a 24GB card like the RTX 4090, most of the model spills into system RAM, and community reports put inference in the single digits of tokens/second. That speed suits an overnight batch job and fails for anything interactive.
Mistral Large 3 GPU Requirements
Mistral Large 3 requires around 710GB of VRAM at FP8, counting weights plus roughly 5% runtime overhead. That figure makes Mistral Large 3 a genuine datacenter workload, and it pushes most teams toward INT4 instead.
Community estimates put the INT4 footprint at roughly 338GB of weights and 355GB with overhead. On paper, Mistral Large 3 at INT4 is reachable on 8x H100, which total 640GB and should leave ample KV cache headroom. FP8 does not fit in that envelope, which is worth confirming before you provision anything.
INT4 costs quality, with reasoning-heavy benchmarks typically dropping 3-5% compared to FP8. That tradeoff suits batch inference and summarization, and it undercuts the point of choosing a 675B model when reasoning accuracy is the goal.
Multi-GPU serving also introduces a parallelism decision. Tensor parallelism splits every weight matrix across GPUs and synchronizes on each layer, which works well on a single node with fast interconnects. Expert parallelism assigns whole expert layers to specific GPUs and communicates only at routing boundaries, which scales better across multiple nodes. Tensor parallelism is simpler and usually sufficient for a single 8-GPU node.
Running Mistral on Thunder Compute
Thunder Compute rents A100 instances at $1.09/hr and H100 PCIe instances at $2.19/hr, billed per minute. The configurations below cover both deployments: a 2x A100 instance for Mistral Small 4, and an 8x H100 configuration for Mistral Large 3 at INT4.
| Deployment | GPU Config | Hourly Cost | Notes |
|---|---|---|---|
| Mistral Small 4 (FP8) | 2x A100 | $2.18/hr | Entry point, cap context near 32K |
| Mistral Small 4 (FP8, long context) | 4x H100 PCIe | $11.56/hr | Headroom for 128K+ context |
| Mistral Small 4 (Q4, budget) | 2x A100 | $2.18/hr | No FP8 tensor cores, use 4-bit |
| Mistral Large 3 (INT4) | 8x H100 PCIe | $23.12/hr | 640GB total, INT4 only |
Mistral Large 3 runs at INT4 on Thunder Compute rather than FP8, because 710GB of FP8 weights exceeds the 640GB an 8x H100 instance provides.
See current Thunder Compute GPU pricing and instance configurations.
Is Self-Hosting Mistral Cheaper Than the API?
Mistral's API undercuts self-hosting for most workloads. Mistral Small 4 costs $0.15/$0.60 per 1M input/output tokens on la Plateforme, and Mistral Large 3 costs $0.50/$1.50. Those rates put break-even at volumes most teams never reach.
| Model | Mistral API (1M in / out) | Self-Host Config | Monthly Compute | Break-Even Volume1 |
|---|---|---|---|---|
| Mistral Small 4 | $0.15 / $0.60 | 2x A100 | ~$1,591 | ~6B tokens/month |
| Mistral Large 3 | $0.50 / $1.50 | 8x H100 | ~$16,878 | ~22.5B tokens/month |
| 1 Break-even assumes a 3:1 input-to-output mix and continuously saturated GPUs, which blends to roughly $0.26 per 1M tokens for Mistral Small 4 and $0.75 for Mistral Large 3. Idle time pushes the threshold higher. | ||||
Roughly 6B tokens/month works out to 200M tokens/day through saturated hardware. Few production workloads sustain that, which is why token math alone rarely justifies self-hosting Mistral.
Cost is rarely the deciding factor. Teams self-host Mistral for data residency, the ability to modify the serving stack, Apache 2.0 licensing without usage restrictions, and freedom from rate limits. Those reasons hold regardless of the token math.
Last Thoughts on Self-Hosting Mistral
Size your GPUs for total parameters rather than active ones, and check Mistral's model card for the current recommended configuration before you deploy. Mistral Small 4 on two A100s is a sensible entry point, giving you instruction following, reasoning, vision, and code in a single deployment under a license that permits commercial use.
Frequently Asked Questions
How much VRAM does Mistral Small 4 need?
Mistral Small 4 needs roughly 111GB of VRAM for weights in FP8, plus KV cache. Community 4-bit builds land around 60-75GB, and full BF16 runs near 238GB. The FP8 checkpoint needs two A100s, or one H200.
Can I run Mistral Large 3 on H100 GPUs?
Yes, with INT4 quantization. The INT4 footprint of roughly 355GB fits on eight H100s with KV cache headroom. FP8 needs around 710GB and exceeds the 640GB available on 8x H100.
Which Mistral model should I self-host?
Pick by workload. Ministral 3 and Codestral cover lighter tasks and coding on one consumer GPU. Mistral Small 4 handles instruction following, reasoning, vision, and agentic coding on two GPUs. Mistral Large 3 needs an eight-GPU server.
How much does it cost to self-host Mistral Small 4?
Two A100s cost $2.18/hr on Thunder Compute, or roughly $1,591/month in compute before storage. Mistral's API charges $0.15/$0.60 per 1M input/output tokens for Small 4, putting break-even near 6B tokens/month.
How much does the Mistral API cost?
Mistral Small 4 costs $0.15/$0.60 per 1M input/output tokens on la Plateforme, and Mistral Large 3 costs $0.50/$1.50. Mistral Medium 3.5 is the premium tier at $1.50/$7.50.