Two open-weight families, thirteen models between them, and one question most comparisons skip: which pair should you actually be comparing?
The model finished downloading at 1:40 AM. Twenty-two gigabytes, pulled over a hotel wifi connection, because someone on a forum said this was the one.
It loaded. It answered the first question in about forty seconds. Then it started thinking, and thinking, and it was still thinking when the output limit cut it off mid-sentence, having produced eleven hundred tokens of reasoning and zero tokens of answer.
That is the actual experience of picking a local model in 2026, and it is not a benchmark problem. It is a matchup problem.
Quick pick by GPU
| Your VRAM | Run this | Why | Full comparison |
|---|---|---|---|
| 8 GB | Qwen 3.5 9B (4-bit), or Gemma 4 E4B for audio | Best capability per gigabyte, with room left for context | Gemma 4 12B vs Qwen 3.5 9B |
| 12 GB | Qwen 3.5 9B for tool calling, Gemma 4 12B for images and audio | The 12B fits at about 7 to 8GB at 4-bit but leaves less room for context | Gemma 4 12B vs Qwen 3.5 9B |
| 16 GB | Gemma 4 12B (multimodal, multilingual) or Qwen 3.5 9B (agents) | The closest call in either family | Gemma 4 12B vs Qwen 3.5 9B |
| 24 GB | Qwen 3.8 27B for coding, Gemma 4 26B A4B for speed | Both fit at 4-bit; Qwen 3.8 is the stronger coder, the Gemma MoE the faster one | Gemma 4 vs Qwen 3.8 |
| 32 GB+ | Gemma 4 31B with real context room, or Qwen 3.6 35B A3B | Dense quality against a light MoE | Gemma 4 31B vs Qwen 3.6 27B |
The detail behind every row is in Which to pick by hardware below.
If you only care about the newest Qwen at the 24GB tier, our Gemma 4 vs Qwen 3.8 comparison covers that matchup on its own.
Now here is why most comparison articles get this wrong before they start.
The full size lineup, side by side
Both families landed within eight weeks of each other in 2026. Both are Apache 2.0. Both are multimodal. That is where the similarity ends, because they are shaped completely differently.
Gemma 4 (Google DeepMind; E2B, E4B, 26B A4B and 31B released 31 March 2026, the 12B on 3 June 2026; Apache 2.0; 140+ languages):
| Model | Parameters | Context | Approx VRAM at 4-bit | Notes |
|---|---|---|---|---|
| E2B | ~2.3B effective | 128K | ~4GB | Phone and browser class, native audio |
| E4B | ~4.5B effective | 128K | ~5 to 6GB | Laptop class, native audio and video |
| 12B | 11.95B unified | 256K | ~7 to 8GB | Encoder-free, handles text, image and audio in one decoder |
| 26B A4B | 25.2B total, 3.8B active | 256K | ~14 to 18GB | Mixture of Experts, the efficiency play |
| 31B | 30.7B dense | 256K | ~18 to 20GB | Quality peak of the family |
Qwen 3.5 (Alibaba, rolled out across February and March 2026, Apache 2.0, 201 languages):
| Model | Parameters | Context | Approx VRAM at 4-bit | Notes |
|---|---|---|---|---|
| 0.8B | 0.8B | 262K | ~1 to 2GB | Edge and embedded |
| 2B | 2B | 262K | ~2 to 3GB | Phone class |
| 4B | 4B | 262K | ~4 to 5GB | Laptop class |
| 9B | 9B | 262K | ~6 to 8GB | Best value mid-tier |
| 27B | 27B dense | 262K | ~16 to 20GB | Strongest dense open coder in the family |
| 35B A3B | 35B total, 3B active | 262K | ~20 to 24GB | MoE, runs surprisingly small |
| 122B A10B | 122B total, 10B active | 262K | Workstation class | Agentic benchmark leader |
| 397B A17B | 397B total, 17B active | 262K | Multi-GPU only | Flagship |
VRAM figures are rules of thumb at 4-bit quantization. Add headroom for context and KV cache, and add a lot of headroom if you plan to use anything close to that 262K window.
Two things jump out. Qwen's smallest model is a third the size of Gemma's smallest. And every single Qwen 3.5 model ships the same 262,144-token context, extensible to roughly a million with YaRN, while Gemma splits at 128K for the small models and 256K for everything above.
Qwen gives you more rungs on the ladder. Gemma gives you fewer rungs but puts more on each one.
Which pairing should you actually compare?
This is the part that trips people up, and it is why "Gemma 4 vs Qwen 3.5" is close to a meaningless question until you make it specific.
Parameter counts do not line up between these families. Gemma's E2B and E4B use per-layer embeddings, so the "E" stands for effective, meaning the parameters actually loaded into VRAM during computation. Comparing a 4.5B effective model to a 4B dense model by the number on the label tells you nothing useful.
Worse, Gemma's 26B A4B is a Mixture of Experts that activates only 3.8B parameters per token. It occupies the memory footprint of a 26B model and the compute cost of a 4B one. Qwen's 35B A3B does the same trick with 3B active. So a "26B vs 35B" comparison is really a comparison of two sparse models with similar active sizes and different total footprints.
Here is the mapping that actually holds up:
- Gemma 4 E2B against Qwen 3.5 2B. Phone and edge class.
- Gemma 4 E4B against Qwen 3.5 4B. Laptop class. Gemma adds native audio and video here.
- Gemma 4 12B against Qwen 3.5 9B. The closest and most useful matchup in the entire comparison.
- Gemma 4 26B A4B against Qwen 3.5 or 3.6 35B A3B. Sparse against sparse.
- Gemma 4 31B against Qwen 3.5 27B. Dense against dense, and the one everyone benchmarks.
Above that line, Qwen runs alone. There is no Gemma 4 answer to the 122B or the 397B. And there is no Gemma 4 27B at all, whatever a search box suggests; the 27B size was Gemma 3's.
Head to head by size tier
Small tier: E2B and E4B against Qwen 2B and 4B
Gemma wins this tier on capability per gigabyte, and it is not especially close on multimodal work. The E2B and E4B ship native audio input, including speech recognition, and handle video at roughly a frame per second for up to a minute. Qwen's small models are multimodal too, but audio is not their story.
Qwen wins on context. A 2B model with a 262K window is unusual and useful for chewing through long documents on hardware that has no business doing so.
If you are building something that listens, take Gemma. If you are building something that reads a lot, take Qwen.
Gemma 4 E4B vs Qwen 3.5 9B: the laptop question
This isn't a like-for-like pairing, but it's the one many laptop owners actually face, because both fit on an 8GB GPU at 4-bit. E4B (about 4.5B effective, about 5 to 6GB) is the lighter, faster model with native audio and video. Qwen 3.5 9B (about 6 to 8GB) is the stronger model for tool calling and knowledge, with a 262K context against E4B's 128K.
| You want | Pick |
|---|---|
| An agent that calls tools reliably | Qwen 3.5 9B |
| Speech or video input | Gemma 4 E4B |
| The most memory left for context on 8GB | Gemma 4 E4B |
| Long documents | Qwen 3.5 9B |
If you have 12GB or more, the fairer fight is Gemma 4 12B against Qwen 3.5 9B, covered next.
Mid tier: Gemma 4 12B against Qwen 3.5 9B
The most interesting fight, and the one most people running a 16GB machine will actually have.
Gemma's 12B is architecturally strange in a good way. It is encoder-free and unified, meaning it drops the separate vision and audio encoders and projects raw image patches and audio directly into the decoder. Third-party comparisons put it ahead on practical coding benchmarks and multilingual reasoning.
Qwen's 9B takes knowledge, science, and agentic tool-use benchmarks by clear margins, with the tool-use gap being one of the widest in the whole comparison. It also leaves you a little more headroom, since 9B at 4-bit sits comfortably under 7GB while the 12B wants closer to 8.
Neither has a published Arena Elo at this tier, so anyone quoting you one is guessing.
Large tier: Gemma 4 31B against Qwen 3.5 27B
Both fit on a single 24GB card. Both claim the other's territory. The published numbers disagree depending on who ran them, which is the most honest thing anyone can tell you about this matchup.
Google's card and third-party runs put Gemma 4 31B at 89.2 on AIME 2026, 80.0 on LiveCodeBench v6, 84.3 on GPQA Diamond, 85.2 on MMLU Pro, and 66.4 on MRCR v2 for long-context retrieval. That last number is the one that matters most: Gemma 3 27B, the previous top of the family, scored 13.5 on the same test. The 128K window on Gemma 3 was mostly decorative. The 256K window on Gemma 4 actually retrieves.
Qwen 3.5 27B posts 72.4 on SWE-bench Verified, which ties GPT-5 mini and is a serious number for a fully open dense model at that size. One independent evaluation of the BF16 checkpoints found Gemma 4 31B ahead on most benchmarks with MMLU Pro and GPQA Diamond as the exceptions. Another leaderboard has Qwen well ahead on coding, driven almost entirely by SWE-Rebench.
Both cannot be right, which tells you something.
We put the dense large-tier fight against Qwen's newer 27B in our Gemma 4 31B vs Qwen 3.6 27B comparison.
Qwen 3.5 vs 3.6 vs 3.8: which Qwen should you compare against Gemma 4?
Qwen 3.5 was the current line when this comparison first went up in August. It isn't the newest Qwen any more, and what matters is which newer releases you can actually run.
| Qwen line | Released | Open models you can run | Context | What it means against Gemma 4 |
|---|---|---|---|---|
| Qwen 3.5 | Feb to Mar 2026 | 0.8B, 2B, 4B, 9B, 27B, 35B A3B, 122B A10B, 397B A17B (Apache 2.0) | 262K | The only Qwen with small and mid sizes. Compare it with Gemma 4 E2B, E4B and 12B. |
| Qwen 3.6 | Apr 2026 | 27B dense, 35B A3B (on Ollama as qwen3.6:27b and qwen3.6:35b-a3b) | 262K | A newer rival for Gemma 4 31B and 26B A4B |
| Qwen 3.7 | May to Jun 2026 | None. Max and Plus are API-only | n/a | Can't be run locally |
| Qwen 3.8 | Aug 2026 | 27B dense (14 Aug, Apache 2.0, about 18GB as qwen3.8:27b); Flash-Next, a large MoE with about 6B active per token; a 2.4T-A95B model | 262K | Qwen3.8-27B is the strongest local rival for Gemma 4 31B. The others are workstation or data-center class |
Is Qwen 3.8 better than Qwen 3.6? At 27B, on Alibaba's published coding benchmarks, yes: 61.7 against 53.5 on SWE-bench Pro and 73.0 against 63.4 on Terminal Bench 2.1, for the same roughly 18GB download. The catch is tool calling through Ollama's /v1 endpoint, where some agent integrations hang on Qwen 3.8; read our Qwen3.8 27B tool-calling fix first.
Qwen 3.5 vs 3.8? They only overlap at 27B. Below that, Qwen 3.5 is still the newest Qwen you can run: there is no Qwen 3.8 0.8B, 4B or 9B. So on 8 to 16GB, "upgrading to 3.8" isn't an option yet; stay on Qwen 3.5.
What that changes in the matchups:
- Small and mid tiers: nothing. E2B, E4B and 12B are still best compared with Qwen 3.5's 2B, 4B and 9B.
- Sparse tier: Gemma 4 26B A4B can be compared with Qwen 3.6 35B A3B as well as the 3.5 version. For coding agents, also look at Ornith 1.5 35B, built on the same Qwen 3.5 MoE architecture.
- Dense large tier: Gemma 4 31B's real rival is Qwen 3.8 27B or Qwen 3.6 27B, not Qwen 3.5 27B. Same memory footprint, newer model. Full detail: Gemma 4 vs Qwen 3.8.
What about older models and successors? Gemma 3 topped out at 27B with a 128K window that scored 13.5 on MRCR v2 long-context retrieval, against 66.4 for Gemma 4 31B, so for new builds Gemma 4 replaces it. Qwen 3 is likewise replaced by 3.5 and newer. As of 2 November 2026 Google has announced no Gemma 5; the newest Gemma release is EmbeddingGemma 2 (6 October), an embedding model.
Gemma vs Qwen for coding
Qwen, at every Qwen version you can fit on a 24GB card next to Gemma 4 31B. The coding numbers scattered through this page all point the same way once you line them up.
| Matchup | Coding result (Gemma vs Qwen) | Better for coding |
|---|---|---|
| Gemma 4 31B vs Qwen 3.8 27B | LiveCodeBench v6: 80.0 vs 90.3 (vendor cards). Independent 12-task suite: 6/12 vs 12/12 | Qwen 3.8 27B |
| Gemma 4 31B vs Qwen 3.6 27B | LiveCodeBench v6: 80.0 vs 83.9 (vendor cards). Same independent suite: 6/12 vs 8/12 | Qwen 3.6 27B |
| Gemma 4 31B vs Qwen 3.5 27B | Qwen posts 72.4 on SWE-bench Verified. Google publishes no SWE-bench score for the 31B, and third-party leaderboards disagree | Leans Qwen 3.5 27B |
| Gemma 4 12B vs Qwen 3.5 9B | Published comparisons disagree at this size | Too close to call |
Gemma 4 vs Qwen 3.6 for coding. Qwen 3.6 27B beats Gemma 4 31B on code, but it is a generation behind Qwen 3.8 27B, which is the same 18GB download and leads 3.6 on every coding benchmark Alibaba publishes (61.7 against 53.5 on SWE-bench Pro, 73.0 against 63.4 on Terminal Bench 2.1). Unless your agent framework hits the Qwen 3.8 /v1 tool-calling hang, start new coding builds on 3.8. The full breakdown is in our Gemma 4 31B vs Qwen 3.6 27B comparison.
Where Gemma still earns a place in a coding setup is strict tool-call formatting. In the same independent test, Gemma 4 31B went 90 for 90 on single tool calls and 30 for 30 on multi-step runs. If your coding agent breaks on malformed calls rather than bad code, that record matters more than the coding gap. Which local models handle tool calls at all is in our Ollama tool-calling reference.
What the benchmarks say across the range, and where they stop being useful
Here is what you see once you read enough of these tables. Static benchmarks favor Qwen on more individual rows. Human preference testing favors Gemma at the top end. Gemma 4 31B reached third on Arena's text leaderboard at roughly 1452 Elo, with the 26B MoE at sixth on around 1441.
That 26B result deserves a second look. It reaches roughly 97% of the dense 31B's quality while activating 3.8B parameters per token instead of 30.7B. Eight times less compute per step, sixth place on a leaderboard full of models with far more active parameters.
And here is what most guides leave out about running either family at home. Quantization does not just cost you accuracy, it costs you completions. One test of an INT4 Qwen 3.5 quant with reasoning enabled found it truncated around 70% of AIME 2025 answers by hitting the 32K output ceiling, against roughly 30% for the full-precision model. The weights barely lost accuracy. The model just started thinking too much to finish.
That is the failure mode you will actually hit at 1:40 AM. Not wrong answers. Unfinished ones.
If you are choosing a model to sit behind an agent rather than a chat window, that distinction matters more than any leaderboard row, and our guide to picking the right model for the task goes into which failure modes actually break agent loops.
Which to pick by hardware
Skip everything above and start here if you already know your VRAM.
8GB. Qwen 3.5 9B at 4-bit, or Gemma 4 E4B if you need audio. The 9B is the best capability-per-gigabyte model in either family and it leaves you room to actually use the context window. Gemma's 12B will technically load and will then fight you for memory.
16GB. This is the real decision point. Gemma 4 12B if your work is multimodal or multilingual. Qwen 3.5 9B if your work is agentic tool calling, with the leftover memory spent on longer context. The 26B MoE is also reachable here at aggressive quantization, though you will feel it.
24GB. The finalists are Gemma 4 26B A4B and a Qwen dense 27B. The Gemma MoE is faster and frugal, roughly 14 to 18GB at 4-bit, and multimodal. The Qwen dense 27B is the stronger coder, roughly 16 to 20GB, and at this size you can pick Qwen 3.5, 3.6, or 3.8. Gemma 4 31B fits at 18 to 20GB but leaves you little room for context.
32GB and up. Gemma 4 31B dense at 4-bit with real room for context (8-bit needs about 35GB), or Qwen 3.5 or 3.6 35B A3B, which runs far lighter than its total parameter count suggests because only 3B activate per token. Above 48GB, Qwen's 122B A10B is the agentic benchmark leader and Gemma has no answer.
A practical note for anyone running these behind an always-on agent rather than an interactive session: memory ceiling is not your constraint, sustained throughput is. We covered the difference in our writeup on running a local LLM agent on consumer hardware, and if you are wiring a local model into a self-hosted agent stack, the hardware requirements for local models with OpenClaw covers the parts that break first.
Every Gemma vs Qwen comparison we have tested
- Gemma 4 vs Qwen 3.8: the newest Qwen against the Gemma 4 family, with the detailed Qwen 3.8 numbers.
- Gemma 4 12B vs Qwen 3.5 9B: the 16GB decision, benchmark by benchmark.
- Gemma 4 31B vs Qwen 3.6 27B: the dense top end on a 24GB card, including the independent coding and tool-call suite.
The closest matchup deserves its own page
If you are on a 16GB machine, the Gemma 4 12B against Qwen 3.5 9B decision is the only one that matters to you, and it is close enough that a paragraph does not settle it.
Both sit under Apache 2.0. Context windows are nearly identical at 256K and 262K. Both run in Ollama, LM Studio, llama.cpp, and vLLM. The split comes down to Gemma's unified multimodal decoder against Qwen's tool-calling and knowledge scores, plus a gigabyte or so of memory headroom.
We ran that one properly, benchmark by benchmark, in the full Gemma 4 12B vs Qwen 3.5 9B comparison. If your machine has 16GB, read that instead of this. If you have a 24GB card and want the dense top end, the Gemma 4 31B vs Qwen 3.6 27B comparison is the one to read.
One closing thought. A year ago the interesting question about open models was whether they could compete with closed ones. That question is settled. The interesting question now is which of the eight or thirteen models a single vendor ships in a single quarter is the right one for your specific machine, and no leaderboard is ever going to answer that for you. Download two. Run your own workload through both. The one that finishes its sentences wins.
Frequently Asked Questions
What is the difference between Gemma 4 and Qwen 3.5?
Gemma 4 is Google DeepMind's open-weight family: five sizes from E2B to a 31B dense flagship (there is no Gemma 4 27B), 140+ languages and native audio on the smallest models. Qwen 3.5 is Alibaba's family: eight sizes from 0.8B to a 397B MoE flagship, 201 languages and a uniform 262K context. Both are Apache 2.0.
Is Gemma 4 or Qwen 3.8 better?
For coding, Qwen 3.8 27B: it leads Gemma 4 31B on LiveCodeBench v6 (90.3 against 80.0) and scored 12 of 12 against 6 of 12 on an independent coding suite. For strict tool-call formatting and long-context retrieval, Gemma 4 31B. Below 24GB, Qwen 3.8 has no model, so compare Gemma 4 with Qwen 3.5.
Should I use Qwen 3.5, 3.6 or 3.8?
On 8 to 16GB, Qwen 3.5: neither 3.6 nor 3.8 ships an open model below 27B. On 24GB, Qwen 3.8 27B, unless your agent hits its Ollama /v1 tool-calling hang, in which case Qwen 3.6 27B. Qwen 3.7 is API-only.
Is there a Qwen 3.8 0.8B, 4B or 9B?
No. The smallest open Qwen 3.8 model is the 27B. For small hardware, use Qwen 3.5 0.8B, 2B, 4B or 9B.
Gemma 4 E4B or Qwen 3.5 9B on 8GB?
Qwen 3.5 9B for tool calling, knowledge and long documents. Gemma 4 E4B for speech or video input, or if you want more memory left for context.
Which is better for coding, Gemma or Qwen?
Qwen, at every size you can fit on a 24GB card. Gemma 4 31B is still worth a look for coding agents that break on malformed tool calls, because it went 90 for 90 on single calls in an independent test.
How do I run Gemma 4 or Qwen locally?
Both run in Ollama, LM Studio, llama.cpp, vLLM and MLX, with GGUF quantizations on Hugging Face. Pull the model that fits your VRAM at 4-bit, then cap your context length rather than accepting the 256K default, because KV cache at full context exhausts memory on consumer hardware long before the weights do.
Is there a Gemma 5?
Not as of 2 November 2026. Google's latest Gemma release is EmbeddingGemma 2 (6 October 2026), an embedding model; the newest chat models are still Gemma 4, with the 12B added on 3 June 2026.
Are these models reliable enough for a production agent?
For narrow, well-scoped tasks, yes. The risk is behaviour under quantization: reasoning-enabled quants can burn through output limits before answering, which looks like a hang. Test with your real prompts at your real quantization, set an output ceiling, and put an approval step in front of anything destructive.
