Deploying DeepSeek V4 locally is straightforward for V4-Flash and a serious infrastructure commitment for V4-Pro. Both variants ship as open weights under the MIT license, but their memory footprints put them in entirely different hardware tiers. This guide covers the real VRAM numbers for both, explains why A100 GPUs are a cost-effective choice for V4-Flash despite lacking native FP8, and walks through the serving options for each.
DeepSeek shipped V4 on April 24, 2026 as a two-model family. Both variants are text-only, support a 1M-token context window, and have weights available on Hugging Face under the deepseek-ai organization.
Takeaways
- V4-Flash requires ~175 GB of VRAM: ~158 GB for weights, ~10 GB for KV cache, and a few GB of runtime overhead.
- V4-Pro's ~865 GB checkpoint requires an 8x H200 node or a multi-node H100 cluster with InfiniBand.
- Self-hosting beats the API for data privacy.
What DeepSeek V4 Actually Is (Flash vs Pro)
DeepSeek V4 is a Mixture-of-Experts (MoE) family with two variants:
- V4-Flash - 284B total parameters, 13B active/token
- V4-Pro - 1.6T total parameters, 49B active/token
The instruct weights for both models ship in a mixed FP4/FP8 format:
- FP4 precision for MoE expert parameters
- FP8 precision for attention, normalization, and router parameters.
Both use a hybrid attention design combining Compressed Sparse Attention (CSA) and Heavily Compressed Attention (HCA), which is what makes 1M-token context practical without a prohibitive KV cache footprint.
MoE models require VRAM for every expert, not just the active ones. With V4-Flash, 13B parameters fire per token, but all 284B weights must fit in memory. MoE reduces compute per token, not memory.
V4's compressed attention uses roughly 7% of DeepSeek V3.2's KV footprint at equivalent context length
DeepSeek V4 Flash: Hardware Requirements
V4-Flash needs roughly 175 GB of VRAM to serve at the full 1M-token context window. That budget breaks down into:
- 158 GB for the native FP4/FP8 weights
- Around 10 GB for the KV cache at 1M context
- A few GB of runtime overhead.
Possible minimum setups for full 1M-token inference are 4x A100 80GB (320 GB), 2x H200 (282 GB) or 2x RTX Pro 6000 Blackwell (192 GB).
The VRAM requirement changes with quantization. Unsloth's community GGUFs compress the model significantly, making smaller multi-GPU configs viable at the cost of some quality.
| Precision | Approx. Weight Size | Total VRAM1 | Practical Config |
|---|---|---|---|
| 3-bit (UD-Q3_K_XL) | 129 GB | ~131 GB | 2x A100 80GB (1M context) |
| 4-bit (UD-Q4_K_XL) | 155 GB | ~167 GB | 2x A100 80GB (128K context), 4x A100 (1M context) |
| 8-bit / lossless (UD-Q8_K_XL) | 162 GB | ~174 GB | 4x A100 80GB (1M context), 2x H200 |
| Native FP4/FP8 (official instruct) | 167 GB | ~180 GB | 2x H200, 2x B200 for FP4 kernels |
| 1KV cache estimate at full 1M-token context. | |||
The 4-bit Unsloth GGUF is a great starting point for most teams. At ~155 GB of weights, it fits on 2x A100 80GB for development and 128K-context production serving, and scales to 4x A100 for full 1M-token context.
Why A100 GPUs Work for V4-Flash Despite Lacking Native FP8
The A100 lacks native FP8 tensor-core support, which arrived with the H100's Hopper architecture. Running DeepSeek V4's FP8-converted weights at full tensor-core speed is not possible on an A100. On top of that, native FP4 expert kernels require Blackwell hardware. For teams using the official FP4/FP8 instruct checkpoint, H100 or H200 is the correct choice.
For teams using Unsloth's GGUF quantizations, neither constraint applies. The A100 accelerates precisions from FP32 down to INT4, and GGUF weights are served directly by llama.cpp without touching FP8 or FP4 kernels. The A100 also retains native INT4 tensor-core support that the H100 dropped, a concrete advantage for the quantized-inference path.
The trade-off relative to an H100 is throughput. The A100's 2 TB/s HBM2e memory bandwidth is roughly 40% lower than the H100's 3.35 TB/s, which translates directly into fewer tokens/second. For development workflows and low-concurrency serving, A100s deliver acceptable throughput at a lower cost and with greater availability than H100s.
DeepSeek V4 Pro: Hardware Requirements
V4-Pro's native FP4/FP8 instruct checkpoint weighs approximately 865 GB, which alone exceeds the 640 GB of a single 8x H100 80GB node. The minimum single node deployment is 8x H200 (1,128 GB of HBM3e). Community GGUF quantizations can compress V4-Pro enough to fit on smaller configs, but at a meaningful quality and throughput cost.
Unlike V4-Flash, where quantization buys you a lot (162 GB lossless Q8 vs 865 GB native), V4-Pro's QAT-trained FP4 experts mean sub-Q4 compression degrades output quality sharply. The practical floor for usable quality is Q2, and even that requires either a very large VRAM pool or CPU offload.
The VRAM requirement by quantization tier:
| Precision | Approx. Weight Size | Total VRAM1 | Minimum Config |
|---|---|---|---|
| Q2_K-XL (community GGUF) | ~498 GB | ~520 GB | 8x H100 80GB + CPU offload (-cmoe flag); or 4x H200 GPU-only |
| Q4_K_M-XL (community GGUF) | ~828 GB | ~870 GB | 8x H200 141GB (1,128 GB) |
| Native FP4/FP8 (official instruct) | ~865 GB | ~900 GB+ | 8x H200 141GB (tight); 2x 8x H100 InfiniBand recommended |
| FP8-only (base checkpoint, Hopper path) | ~960 GB | ~1,000 GB+ | 2x 8x H100 InfiniBand or 8x H200 with context capping |
| 1 KV cache estimate at 128K context. Full 1M-token context adds substantial headroom on top of these figures. | |||
The Q2_K-XL build deserves a note: it fits on a single 8x H100 node only with the -cmoe flag, which offloads expert weights to CPU RAM during inference. Expert parameters are moved from GPU to RAM and back each forward pass, which drops decode speed to a few tokens per second. It is viable for offline batch jobs but not for interactive serving.
The Q4_K_M-XL build at ~828 GB requires ~870 GB of total VRAM headroom, which means 8x H200 is the minimum GPU-only config. Quality is better than Q2 but still meaningfully below the native checkpoint, since the source weights are already FP4-quantized and a second Q4 quantization compounds the error.
Self-Hosting vs API: DeepSeek V4 Pricing Math
Self-hosting beats the API only past a sustained usage threshold. V4-Flash API pricing is $0.14/1M input tokens and $0.28/1M output tokens. V4-Pro is $0.435/1M input and $0.87/1M output. At those rates, light or bursty workloads are almost always cheaper via the API. Operational overhead eats the savings at small scale.
At $5.96/hr for a 4x A100 80GB setup on Thunder Compute, the math rarely beats the V4-Flash API on cost alone. The case for self-hosting is data privacy: if your data cannot leave your infrastructure, the API is not an option regardless of price.
| Scenario | Recommended Path | Why |
|---|---|---|
| Light or occasional use | DeepSeek API | $0.14/$0.28 per 1M tokens, zero ops overhead. |
| Privacy-sensitive data | Self-host V4-Flash | Nothing leaves your infrastructure. |
| Fine-tuning or research | Self-host on base checkpoints | MIT license covers commercial redistribution. |
Last Thoughts on Deploying DeepSeek V4 Locally
V4-Flash on 4x A100 80GB is the practical self-hosting target for teams with a data privacy requirement or an experimental workload that benefits from direct model access. The A100's lack of FP8 is a non-issue on the Unsloth GGUF path, and the MIT license means you can modify, fine-tune, and redistribute the weights without restriction. For everything else, the V4-Flash API at $0.14/$0.28 per 1M tokens is hard to beat on cost.
Frequently Asked Questions
Can you run DeepSeek V4 Pro locally?
Only with a large cluster. V4-Pro's native checkpoint is ~865 GB, and the minimum realistic deployment is 8x H200 SXM5 (1,128 GB) on a single node, or a multi-node H100 setup with InfiniBand.
Can an A100 run DeepSeek V4 without native FP8 support?
Yes, via Unsloth GGUF quantizations served with llama.cpp. The A100 has native INT4 tensor-core support that the H100 dropped, so quantized inference runs without a hardware gap. The trade-off is lower concurrent throughput compared to vLLM with native FP8 on a Hopper GPU.
Is self-hosting DeepSeek V4 cheaper than the API?
Only at sustained scale. V4-Flash API pricing is $0.14/$0.28 per 1M tokens. Self-hosting on rented GPUs beats per-token pricing for continuous high-volume inference, or when data privacy prevents using a hosted API.
What is the difference between V4-Flash and V4-Pro for local deployment?
V4-Flash (284B total, 13B active) needs ~170-175 GB total VRAM and fits on 4x A100 80GB or 2x H200. V4-Pro (1.6T total, 49B active) needs ~865 GB and requires an 8x H200 node or multi-node cluster. Flash is the practical self-hosting target for most teams.
Does a self-hosted DeepSeek V4 endpoint work with standard LLM clients?
Yes. Both llama.cpp and vLLM expose a REST API that follows the same request and response format as the OpenAI SDK. You can point Cursor, Claude Code, or any OpenAI-compatible client at your self-hosted endpoint by changing the base URL.