Open source large language models have closed the gap with proprietary systems faster than most researchers expected. Models that required billion-dollar training runs are now downloadable, modifiable, and deployable by any team with the right hardware.
Takeaways
- Kimi K3 leads on raw capability, but its 2.8T parameters need a multi-GPU cluster to self-host.
- GLM-5.2 tops open-weight coding benchmarks like SWE-bench Pro and Terminal-Bench 2.1.
- DeepSeek V4-Flash and Mistral Small 4 bring near-frontier quality to 2-GPU setups.
- Open weight rarely means fully open source: most models publish weights, not training data.
- Thunder Compute runs any of these on on-demand A100 and H100 GPUs billed by the minute.
What Makes an LLM "Open Source"?
The term "open source" means something different in the LLM world than in traditional software. A fully open model can be replicated because it publishes weights, training data, architecture code, and training pipeline under a license allowing free use, modification, and redistribution.
In reality, most models called "open source" are really "open weight": only the model weights are publicly available, but the training data and pipeline remain proprietary.
The Open Source Initiative published its Open Source AI Definition (OSAID) to formalize these distinctions. By its strictest reading, models like DeepSeek R1 and Llama 4 are open weight rather than fully open source, because their training datasets are not released.
For most, the definition is simpler. A model is open source if it lets you:
- Download weights
- Run them locally
- Use them commercially
LLM Model Licenses
MIT and Apache 2.0 licenses grant unrestricted use. Custom community licenses, like Meta's Llama 4 license or the Modified MIT used by Kimi K2, add constraints such as user-count thresholds or attribution requirements.
Always read the license before deploying a model commercially.
LLM Evaluation Metrics
Comparing models requires a shared vocabulary of benchmarks. The most commonly cited ones evaluate different capabilities, and no single score captures overall quality.
Benchmarks
Six benchmarks stress-test models across advanced logic, autonomous software engineering, and multi-domain reasoning.
| Benchmark | What It Measures | Why It Matters | Reference |
|---|---|---|---|
| GPQA Diamond | Graduate-level science reasoning | Resists memorization; tests true understanding | Rein et al. (arXiv 2023) |
| AIME | Multi-step math problem solving | Measures structured logical reasoning | Mathematical Association of America |
| SWE-Bench Verified | Real-world code issue resolution | Best proxy for agentic software engineering | OpenAI & SWE-bench Team (2024) |
| Humanity's Last Exam | Extremely hard cross-domain questions | Near-ceiling test for general intelligence | Center for AI Safety & Scale AI (2025) |
| MMMLU | Multilingual reasoning | Evaluates language breadth beyond English | Alibaba / EvalScope Project |
| LiveCodeBench | Live coding problem solving | Tests coding with contamination-resistant problems | Jain et al. (2024) |
Inference Performance & Efficiency
A model's operational efficiency determines its practical viability. A model that aces every reasoning test but needs 8x H100 GPUs to serve one user is not production-ready.
When evaluating models, teams must consider:
Throughput (Tokens/Second): Measures the volume of text a system generates concurrently. High throughput matters for background processing, large-scale data analysis, and keeping infrastructure costs sustainable under heavy load.
Latency (Time to First Token): Measures how quickly the model begins its response after receiving a prompt. Low latency is critical for user-facing applications, where even a two-second delay breaks the illusion of conversation.
The optimal choice often means sacrificing some accuracy to achieve the speed and cost efficiency your application requires.
Open Source LLM Leaderboard
There is no uncontested leaderboard featuring only open source LLMs. Although now archived, the Open LLM Leaderboard by the Hugging Face community was the reference for comparing open-weight models on standardized benchmarks.
These rank among the strongest open-weight models, but they optimize for different goals: reasoning benchmarks, efficiency, long context, or multimodal breadth. The table is ordered by release date.
| Model | Parameters (Active) | GPQA Diamond | SWE-Bench | AIME 2025 | Humanity's Last Exam | Live Code Bench | Source |
|---|---|---|---|---|---|---|---|
| Gemini 3.1 Pro1 | N/A | 94.3 | 80.6 | – | 44.4 | 91.7 | Google DeepMind |
| Claude Opus 4.61 | N/A | 91.3 | 80.8 | – | 40.0 | 88.8 | Anthropic |
| Kimi K3 | 2.8T (~50B active) | 93.5 | 76.8 | 96.1 | 43.54 | – | Moonshot AI (2026) |
| GLM-5.2 | 744B (40B active) | 91.2 | 62.12 | 99.23 | 40.5 | – | Z.ai (2026) |
| DeepSeek V4-Pro5 | 1.6T (49B active) | 90.1 | 80.6 | – | 37.7 | 93.5 | DeepSeek AI (2026) |
| DeepSeek V4-Flash5 | 284B (13B active) | 88.1 | 79.0 | – | – | 91.6 | DeepSeek AI (2026) |
| Mistral Small 46 | 119B (6.5B active) | – | – | – | – | – | Mistral AI (2026) |
| Kimi K2.5 | 1T (32B) | 87.6 | 76.8 | 96.1 | – | 85.0 | Moonshot AI (2026) |
| Mistral Large 36 | 675B (41B active) | 43.9 | – | – | – | 82.8 | Mistral AI (2025) |
| Kimi K2 Thinking | 1T (32B active) | 84.5 | 71.3 | 99.1 | 44.9 | 83.1 | Moonshot AI (2025) |
| Llama 4 Maverick | 400B (17B active) | 69.8 | 65.0 | – | – | 43.4 | Meta AI (2025) |
| Llama 4 Scout | 109B (17B active) | 73.7 | 68.0 | 6.7 | – | 33.3 | Meta AI (2025) |
| Nemotron Ultra 253B | 253B (dense) | 76.0 | – | 72.5 | – | – | NVIDIA (2025) |
| DeepSeek V3 0324 | 685B (37B active) | 59.1 | 42.0 | 58.1 | 20.3 | 74.1 | DeepSeek AI (2025) |
| DeepSeek-R1 | 671B (37B active) | 71.5 | 49.2 | 74.0 | – | 65.9 | DeepSeek AI (2025) |
1 Added for comparison; closed-model scores from the DeepSeek-V4 technical report (2026). |
|||||||
Kimi K3
Launched July 16, 2026, Kimi K3 is the largest open-weight model at the time of writing. The 2.8T-parameter Mixture-of-Experts model activates 16 of 896 experts per token (roughly 50B active), with a 1M-token context window and native multimodal input.
K3 is built on two architectural changes Moonshot developed in-house:
- Kimi Delta Attention (KDA)
- Attention Residuals (AttnRes).
Along with a Stable LatentMoE framework, these changes deliver roughly 2.5x the scaling efficiency of Kimi K2, converting compute into capability rather than relying on parameter count alone.
The catch is hardware. At 2.8T parameters, K3 is a multi-GPU cluster model: the native MXFP4 weights are around 1.56TB, and serving frameworks like vLLM target 16x B200 GPUs.
Learn how to run and self-host Kimi K3.
Kimi K2
Moonshot AI's Kimi K2 was one of the most significant open-weight releases of 2025. It demonstrated that a Chinese lab could release models to compete with or beat GPT-5 on key benchmarks. Both K2 models use a Mixture-of-Experts (MoE) architecture with 1T total parameters and 32B active per inference. They ship under a Modified MIT License that allows broad commercial use.
Learn how to run Kimi K2 with Ollama.
Kimi K2.5
Kimi K2.5 is a native multimodal agentic model built through continual pretraining on approximately 15T mixed visual and text tokens. It pairs strong reasoning and agentic coding with native image and video understanding, and introduced Moonshot's Agent Swarm for coordinating parallel sub-agents. Moonshot's later K3 and K2.x releases have since moved ahead of it on most benchmarks.
Kimi K2 Thinking
Kimi K2 Thinking is the reasoning-focused variant of the K2 family, optimized for test-time scaling by expanding thinking tokens and tool-call rounds together. It is built for long-horizon agentic reasoning and can perform 200 to 300 consecutive tool calls without manual intervention.
GLM-5.2
Z.ai's GLM-5.2 is a 744B MoE model with 40B parameters active per token, released under an MIT license in June 2026. It is tuned for long-horizon coding and agentic workflows.
It performs strongly on reasoning benchmarks like GPQA Diamond and AIME, and on long-horizon coding suites such as Terminal-Bench 2.1 and SWE-bench Pro.
GLM-5.2 does not support image understanding, which limits it on visual analysis workflows. It requires significant multi-GPU infrastructure for self-hosted inference: the 2-bit quantized variant alone needs roughly 245 GB of combined memory.
Learn how to run GLM-5.2 with Unsloth and see the full hardware requirements and cost breakdown.
DeepSeek
DeepSeek's January 2025 release of R1 was a turning point for open-source AI. A peer-reviewed Nature paper confirmed R1's reasoning training cost just $294K, on top of the ~$6M spent building the DeepSeek V3 base model it builds on. That disclosure forced the industry to reassess how much compute was required to reach top-tier performance.
DeepSeek's R1, V3, and V4 models all ship under the MIT License, and the lab has continued releasing new open-weight generations at a rapid pace.
DeepSeek-R1
DeepSeek-R1 is a 671B parameter model focused on advanced reasoning through reinforcement learning post-training.
It achieves 74% on AIME 2025 and 49.2% on SWE-Bench Verified, performing comparably to models that cost far more to build. Its distilled variants, ranging from 1.5B to 70B parameters, bring strong reasoning to hardware ranging from consumer laptops to mid-range workstations.
Read a full guide on DeepSeek R1 and learn to run it locally.
DeepSeek V3 0324
Released in March 2025, DeepSeek-V3-0324 updates the original V3 with an improved post-training pipeline that borrows reinforcement learning techniques from R1.
The update brought the parameter count to 685B. At release, it outperformed GPT-4.5 in math and coding evaluations, making it a strong general-purpose open-weight baseline.
DeepSeek V4
DeepSeek V4 shipped on April 24, 2026 as a two-model, text-only family with a 1M-token context window, released as open weights under the MIT License.
It comes in two variants sized for very different hardware:
- V4-Flash at 284B total parameters (13B active per token)
- V4-Pro at 1.6T total parameters (49B active per token).
Both ship in a mixed FP4/FP8 format, using FP4 for the MoE experts and FP8 for attention, normalization, and routing.
The 1M-token context is made practical by a hybrid attention design that combines Compressed Sparse Attention and Heavily Compressed Attention, which keeps the KV cache small at long context.
Learn how to deploy DeepSeek V4 locally, with a full hardware guide covering the V4-Flash and V4-Pro variants.
Llama 4
Meta's Llama family is the infrastructure layer of the open-weight ecosystem. The Llama 4 generation, released in April 2025, introduced MoE architecture and added native multimodality across text, images, and video.
Both Scout and Maverick are available under Meta's Llama 4 Community License, which permits free commercial use for products serving under 700M monthly active users.
Learn how to run Llama 4 with Ollama.
Llama 4 Maverick
Llama 4 Maverick is the flagship generalist model of the Llama 4 family, with 400B total parameters and 17B active per token across 128 experts.
Its 1M-token context window and multimodal capabilities position it as a direct competitor to GPT-4o and Gemini 2.0 Flash. On the multilingual reasoning benchmark MMMLU, Maverick scores 84.6%, and its inference cost and speed make it a practical choice for production deployments. See the Thunder Compute guide to fine-tuning Llama 4 for a step-by-step walkthrough using a single A100 80GB.
Llama 4 Scout
Llama 4 Scout is an efficiency-focused sibling with 109B total parameters and 17B active per token distributed across 16 experts.
Scout runs high-speed inference on a much smaller GPU footprint, with a 192K-token context window and native multimodal support for text and vision. It excels at multi-document summarization, live tool-calling, and parsing dense codebases, making it well suited for local development, edge deployments, and high-volume agentic pipelines.
Mistral
Mistral AI is one of Europe's most influential open-weight labs, and its two cloud-scale models sit at very different points on the cost curve. Both ship under Apache 2.0, which permits commercial use without usage restrictions.
Mistral Small 4
Released in March 2026, Mistral Small 4 is a 119B MoE checkpoint that activates just 6.5B parameters per token. It folds instruction following, reasoning, vision, and agentic coding into a single model that runs at small-model speed.
Its 128 experts mean the full 119B needs roughly 111GB of VRAM at FP8, making a 2x A100 pair the lowest configuration that serves the full-precision weights. Community 4-bit builds compress it to the around 75GB, low enough to experiment on a single GPU, and its Multi-head Latent Attention keeps the KV cache small at long context.
Mistral Large 3
Released in December 2025, Mistral Large 3 is a 675B MoE model with 41B active parameters per token and a 256K context window. It buys stronger reasoning than Small 4 at a much heavier footprint.
Its FP8 weights need around 710 GB of VRAM, which pushes most teams to INT4 at roughly 355 GB so the model fits on an 8x H100 node. As a generalist rather than a reasoning specialist, it trades a few points of hardest-benchmark accuracy for broad knowledge and throughput, so it suits batch inference more than accuracy-critical workloads.
Learn how to self-host Mistral, including the GPU requirements for Mistral Small 4 and Large 3.
Nemotron
NVIDIA's Nemotron Ultra 253B takes a different approach from the large MoE models. It uses Neural Architecture Search to optimize a dense transformer derived from Llama 3.1-405B, matching competitive benchmark scores with a smaller, more hardware-efficient architecture.
Nemotron Ultra 253B
Released in April 2025, Nemotron Ultra 253B is a dense decoder-only transformer with 253B parameters optimized for reasoning, RAG, and tool-calling tasks.
It operates on a single 8x H100 node using FP8 precision. With reasoning mode enabled, it jumps from approximately 80% to over 97% accuracy on MATH-500 and achieves 76% on GPQA Diamond. The model is released under the NVIDIA Open Model License and supports commercial use.
Open Source LLM Platforms
Three tools dominate the space for running, fine-tuning, and serving open-weight models: Ollama, Unsloth, and vLLM.
Ollama
Ollama is the fastest way to run open-weight LLMs. A single command like ollama run llama4:maverick downloads and launches the model, handling quantization and hardware detection automatically.
It supports a wide range of model families, exposes an OpenAI-compatible API endpoint, and is the go-to tool for developers who want to prototype quickly without managing infrastructure.
Read the Thunder Compute guide to running Ollama.
Unsloth
Unsloth is an open-source fine-tuning library that rewrites attention and backpropagation kernels in Triton to deliver faster training with lower VRAM consumption.
It supports supervised fine-tuning, QLoRA, and LoRA workflows on models from Llama 4 to Qwen to DeepSeek, and integrates with Hugging Face Transformers for dataset loading and model export. The Thunder Compute Unsloth guide covers installation and first fine-tuning runs in detail.
vLLM
vLLM is the production inference engine of choice for serving open-weight models at scale. It implements PagedAttention for efficient KV cache management, continuous batching to maximize GPU utilization, and an OpenAI-compatible server API.
vLLM is the standard choice when a model needs to handle multiple concurrent users or sustained throughput. It supports FP8 and GPTQ quantization, making it practical for deploying large models like Nemotron Ultra 253B on a single 8x H100 node.
Run and Fine-Tune the Best Open Source LLMs on Thunder Compute
GLM-5.2, DeepSeek V4, and Mistral Large 3 require multi-GPU setups for full-precision inference. Even quantized variants of these models benefit from A100 or H100-class GPUs to deliver acceptable throughput.
For a full breakdown of which GPU fits which model size and budget, see the Thunder Compute guide to the best GPU for LLM work.
Thunder Compute provides on-demand access to A100 and H100 GPUs billed by the minute, with pre-configured templates for both Ollama and Unsloth.
Spin up an Ollama instance to run Llama 4 in minutes, or launch an Unsloth environment to fine-tune DeepSeek-R1 on your own dataset without configuring CUDA, PyTorch, or dependency conflicts from scratch.
For fine-tuning specifics, the supervised fine-tuning guide walks through dataset preparation, training configuration, and evaluation.
Last Thoughts on Open Source LLMs
Kimi K3, GLM-5.2, and DeepSeek V4 all reached frontier-level benchmark scores in 2025 and 2026, often on a fraction of the compute budget of their closed-source counterparts. Thunder Compute's on-demand A100 and H100 instances let you run or fine-tune any of them without a long-term infrastructure commitment.
FAQ
Is ChatGPT an LLM?
Yes. ChatGPT is a conversational interface built on GPT-4o and related models developed by OpenAI. ChatGPT is the product; the GPT models are the LLMs.
Is DeepSeek open source?
DeepSeek publishes weights and inference code for R1, V3, and V3 0324 under the MIT License. However, training datasets and the full training pipeline are not released, so it does not meet the OSI's Open Source AI Definition.
What is the difference between open source and open weight LLMs?
A fully open source model publishes weights, training data, architecture code, and training pipeline under a permissive license. An open weight model only releases the weights. Most models marketed as open source are actually open weight.
What is the best open source LLM for coding?
Kimi K3 is the strongest all-around open model for coding as of July 2026, topping the Frontend Code Arena and scoring 76.8% on SWE-Bench Verified. GLM-5.2 still leads SWE-bench Pro (62.1%) and Terminal-Bench 2.1 (81.0%). Because K3's 2.8T-parameter weights need a large GPU cluster to self-host, GLM-5.2 or the Kimi K2 line are more practical on modest hardware.
What GPU do I need to run large open-weight models?
GLM-5.2 and DeepSeek V4 require multi-GPU setups with A100 or H100-class GPUs. Smaller distilled variants like DeepSeek-R1 1.5B to 70B can run on consumer GPUs. Thunder Compute offers on-demand A100 and H100 instances billed by the minute.
What platform should I use to run open-weight LLMs locally?
Ollama is the fastest option for local prototyping. Unsloth is the go-to for fine-tuning with lower VRAM usage. vLLM handles production serving at scale with continuous batching and an OpenAI-compatible API.
What is Kimi K3?
Kimi K3 is Moonshot AI's 2.8T-parameter open-weight Mixture-of-Experts model, released July 16, 2026, with full weights following on July 27 under a Modified MIT license. It activates 16 of 896 experts per token (roughly 50B active), supports a 1M-token context window, and adds native multimodal input. It ranks near the frontier on coding and reasoning benchmarks but, at 2.8T parameters, needs a large multi-GPU cluster to self-host rather than a single GPU.
What is GLM-5.2?
GLM-5.2 is Z.ai's 744B MoE open-weight model released in June 2026 under an MIT license. It leads all open-weight models on SWE-bench Pro (62.1%) and Terminal-Bench 2.1 (81.0%), and scores 91.2% on GPQA Diamond. It does not support image understanding and requires significant multi-GPU infrastructure for self-hosted inference.