Two open-weight families, thirteen models between them, and one question nobody answers properly: which pair should you actually be comparing?
The model finished downloading at 1:40 AM. Twenty-two gigabytes, pulled over a hotel wifi connection, because someone on a forum said this was the one.
It loaded. It answered the first question in about forty seconds. Then it started thinking, and thinking, and it was still thinking when the output limit cut it off mid-sentence, having produced eleven hundred tokens of reasoning and zero tokens of answer.
That is the actual experience of picking a local model in 2026, and it is not a benchmark problem. It is a matchup problem.
Now here is why most comparison articles get this wrong before they start.
The full size lineup, side by side
Both families landed within eight weeks of each other in 2026. Both are Apache 2.0. Both are multimodal. That is where the similarity ends, because they are shaped completely differently.
Gemma 4 (Google DeepMind, April 2, 2026, Apache 2.0, 140+ languages):
| Model | Parameters | Context | Approx VRAM at 4-bit | Notes |
|---|---|---|---|---|
| E2B | ~2.3B effective | 128K | ~4GB | Phone and browser class, native audio |
| E4B | ~4.5B effective | 128K | ~5 to 6GB | Laptop class, native audio and video |
| 12B | 12B unified | 256K | ~9 to 11GB | Encoder-free, handles text, image and audio in one decoder |
| 26B A4B | 25.2B total, 3.8B active | 256K | ~14 to 18GB | Mixture of Experts, the efficiency play |
| 31B | 30.7B dense | 256K | ~20 to 24GB | Quality peak of the family |
Qwen 3.5 (Alibaba, rolled out across February and March 2026, Apache 2.0, 201 languages):
| Model | Parameters | Context | Approx VRAM at 4-bit | Notes |
|---|---|---|---|---|
| 0.8B | 0.8B | 262K | ~1 to 2GB | Edge and embedded |
| 2B | 2B | 262K | ~2 to 3GB | Phone class |
| 4B | 4B | 262K | ~4 to 5GB | Laptop class |
| 9B | 9B | 262K | ~6 to 8GB | Best value mid-tier |
| 27B | 27B dense | 262K | ~16 to 20GB | Strongest dense open coder in the family |
| 35B A3B | 35B total, 3B active | 262K | ~18 to 22GB | MoE, runs surprisingly small |
| 122B A10B | 122B total, 10B active | 262K | Workstation class | Agentic benchmark leader |
| 397B A17B | 397B total, 17B active | 262K | Multi-GPU only | Flagship |
VRAM figures are rules of thumb at 4-bit quantization. Add headroom for context and KV cache, and add a lot of headroom if you plan to use anything close to that 262K window.
Two things jump out. Qwen's smallest model is a third the size of Gemma's smallest. And every single Qwen 3.5 model ships the same 262,144-token context, extensible to roughly a million with YaRN, while Gemma splits at 128K for the small models and 256K for everything above.
Qwen gives you more rungs on the ladder. Gemma gives you fewer rungs but puts more on each one.
Which pairing should you actually compare?
This is the part that trips people up, and it is why "Gemma 4 vs Qwen 3.5" is close to a meaningless question until you make it specific.
Parameter counts do not line up between these families. Gemma's E2B and E4B use per-layer embeddings, so the "E" stands for effective, meaning the parameters actually loaded into VRAM during computation. Comparing a 4.5B effective model to a 4B dense model by the number on the label tells you nothing useful.
Worse, Gemma's 26B A4B is a Mixture of Experts that activates only 3.8B parameters per token. It occupies the memory footprint of a 26B model and the compute cost of a 4B one. Qwen's 35B A3B does the same trick with 3B active. So a "26B vs 35B" comparison is really a comparison of two sparse models with similar active sizes and different total footprints.
Here is the mapping that actually holds up:
- Gemma 4 E2B against Qwen 3.5 2B. Phone and edge class.
- Gemma 4 E4B against Qwen 3.5 4B. Laptop class. Gemma adds native audio and video here.
- Gemma 4 12B against Qwen 3.5 9B. The closest and most useful matchup in the entire comparison.
- Gemma 4 26B A4B against Qwen 3.5 35B A3B. Sparse against sparse.
- Gemma 4 31B against Qwen 3.5 27B. Dense against dense, and the one everyone benchmarks.
Above that line, Qwen runs alone. There is no Gemma 4 answer to the 122B or the 397B.
Head to head by size tier
Small tier: E2B and E4B against Qwen 2B and 4B
Gemma wins this tier on capability per gigabyte, and it is not especially close on multimodal work. The E2B and E4B ship native audio input, including speech recognition, and handle video at roughly a frame per second for up to a minute. Qwen's small models are multimodal too, but audio is not their story.
Qwen wins on context. A 2B model with a 262K window is unusual and genuinely useful for chewing through long documents on hardware that has no business doing so.
If you are building something that listens, take Gemma. If you are building something that reads a lot, take Qwen.
Mid tier: Gemma 4 12B against Qwen 3.5 9B
The most interesting fight, and the one most people running a 16GB machine will actually have.
Gemma's 12B is architecturally strange in a good way. It is encoder-free and unified, meaning it drops the separate vision and audio encoders and projects raw image patches and audio directly into the decoder. Third-party comparisons put it ahead on practical coding benchmarks and multilingual reasoning.
Qwen's 9B takes knowledge, science, and agentic tool-use benchmarks by clear margins, with the tool-use gap being one of the widest in the whole comparison. It also leaves you more headroom, since 9B at 4-bit sits comfortably under 8GB while the 12B wants closer to 11.
Neither has a published Arena Elo at this tier, so anyone quoting you one is guessing.
Large tier: Gemma 4 31B against Qwen 3.5 27B
Both fit on a single 24GB card. Both claim the other's territory. The published numbers disagree depending on who ran them, which is the most honest thing anyone can tell you about this matchup.
Google's card and third-party runs put Gemma 4 31B at 89.2 on AIME 2026, 80.0 on LiveCodeBench v6, 84.3 on GPQA Diamond, 85.2 on MMLU Pro, and 66.4 on MRCR v2 for long-context retrieval. That last number is the one that matters most: Gemma 3's 31B predecessor scored 13.5 on the same test. The 128K window on Gemma 3 was mostly decorative. The 256K window on Gemma 4 actually retrieves.
Qwen 3.5 27B posts 72.4 on SWE-bench Verified, which ties GPT-5 mini and is a serious number for a fully open dense model at that size. One independent evaluation of the BF16 checkpoints found Gemma 4 31B ahead on most benchmarks with MMLU Pro and GPQA Diamond as the exceptions. Another leaderboard has Qwen well ahead on coding, driven almost entirely by SWE-Rebench.
Both cannot be right, which tells you something.
What the benchmarks say across the range, and where they stop being useful
Here is the pattern once you read enough of these tables. Static benchmarks favor Qwen on more individual rows. Human preference testing favors Gemma at the top end. Gemma 4 31B reached third on Arena's text leaderboard at roughly 1452 Elo, with the 26B MoE at sixth on around 1441.
That 26B result deserves a second look. It reaches roughly 97% of the dense 31B's quality while activating 3.8B parameters per token instead of 30.7B. Eight times less compute per step, sixth place on a leaderboard full of models with far more active parameters.
And here is what nobody tells you about running either family at home. Quantization does not just cost you accuracy, it costs you completions. One test of an INT4 Qwen 3.5 quant with reasoning enabled found it truncated around 70% of AIME 2025 answers by hitting the 32K output ceiling, against roughly 30% for the full-precision model. The weights barely lost accuracy. The model just started thinking too much to finish.
That is the failure mode you will actually hit at 1:40 AM. Not wrong answers. Unfinished ones.
If you are choosing a model to sit behind an agent rather than a chat window, that distinction matters more than any leaderboard row, and our guide to picking the right model for the task goes into which failure modes actually break agent loops.
Quick aside on where this fits. We build a no-code agent platform, and we are bring-your-own-key across 28 providers with no markup on inference, which means we have no financial reason to push you toward one model over another. If you want to point an agent at whichever of these two you pick, our provider and BYOK setup takes about a minute and the free plan does not need a card.
Which to pick by hardware
Skip everything above and start here if you already know your VRAM.
8GB. Qwen 3.5 9B at 4-bit, or Gemma 4 E4B if you need audio. The 9B is the best capability-per-gigabyte model in either family and it leaves you room to actually use the context window. Gemma's 12B will technically load and will then fight you for memory.
16GB. This is the real decision point. Gemma 4 12B if your work is multimodal or multilingual. Qwen 3.5 9B if your work is agentic tool calling, with the leftover memory spent on longer context. The 26B MoE is also reachable here at aggressive quantization, though you will feel it.
24GB. The finalists are Gemma 4 26B A4B and Qwen 3.5 27B. The Gemma MoE is faster and frugal, roughly 14 to 18GB at 4-bit, and multimodal. The Qwen dense 27B is the stronger coder, roughly 16 to 20GB. Gemma 4 31B fits at 20 to 24GB but leaves you nothing for context.
32GB and up. Gemma 4 31B dense unquantized-ish, or Qwen 3.5 35B A3B, which runs far lighter than its total parameter count suggests because only 3B activate per token. Above 48GB, Qwen's 122B A10B is the agentic benchmark leader and Gemma has no answer.
A practical note for anyone running these behind an always-on agent rather than an interactive session: memory ceiling is not your constraint, sustained throughput is. We covered the difference in our writeup on running a local LLM agent on consumer hardware, and if you are wiring a local model into a self-hosted agent stack, the hardware requirements for local models with OpenClaw covers the parts that break first.
The closest matchup deserves its own page
If you are on a 16GB machine, the Gemma 4 12B against Qwen 3.5 9B decision is the only one that matters to you, and it is genuinely close enough that a paragraph does not settle it.
Both sit under Apache 2.0. Context windows are nearly identical at 256K and 262K. Both run in Ollama, LM Studio, llama.cpp, and vLLM. The split comes down to Gemma's unified multimodal decoder against Qwen's tool-calling and knowledge scores, plus about 3GB of memory headroom.
We ran that one properly, benchmark by benchmark, in the full Gemma 4 12B vs Qwen 3.5 9B comparison. If your machine has 16GB, read that instead of this.
If any of this resonated, start free with BetterClaw. One agent, every core feature, no credit card, and bring your own key so you pay your provider directly with nothing added on top. Pro is $49 a month for five agents, or $39 on annual. Full pricing is here. Your first deploy takes about a minute, and the model choice stays yours.
One closing thought. A year ago the interesting question about open models was whether they could compete with closed ones. That question is settled. The interesting question now is which of the eight or thirteen models a single vendor ships in a single quarter is the right one for your specific machine, and no leaderboard is ever going to answer that for you. Download two. Run your own workload through both. The one that finishes its sentences wins.
Frequently Asked Questions
What is the difference between Gemma 4 and Qwen 3.5?
Gemma 4 is Google DeepMind's open-weight family released April 2, 2026, with five sizes from E2B up to a 31B dense flagship, 140+ languages, and native audio on the smallest models. Qwen 3.5 is Alibaba's family released across February and March 2026, with eight sizes from 0.8B to a 397B Mixture of Experts flagship, 201 languages, and a uniform 262K context window. Both are Apache 2.0 licensed with no commercial restrictions.
How does Gemma 4 31B compare to Qwen 3.5 27B?
They are the closest large-tier matchup and both fit on a single 24GB GPU. Gemma 4 31B leads on multimodal, multilingual, and long-context retrieval, scoring 66.4 on MRCR v2 against 13.5 for its Gemma 3 predecessor. Qwen 3.5 27B leads on coding, posting 72.4 on SWE-bench Verified. Independent evaluations disagree on the overall winner, which is a reasonable signal that your own workload should decide it.
How do I run Gemma 4 or Qwen 3.5 locally?
Both families are supported in Ollama, LM Studio, llama.cpp, vLLM, and MLX, with GGUF quantizations published on Hugging Face. Pull the model that fits your VRAM at 4-bit, then cap your context length rather than accepting the 256K default, because KV cache allocation at full context will exhaust memory on consumer hardware long before the weights do.
Is Gemma 4 or Qwen 3.5 cheaper to run?
Both are free to download under Apache 2.0, so the cost is entirely hardware and electricity. The efficiency winners are the sparse models: Gemma 4 26B A4B activates 3.8B parameters per token and reaches roughly 97% of the dense 31B's quality, and Qwen 3.5 35B A3B activates only 3B. If you are comparing against hosted APIs instead, remember that any managed agent platform charging a markup on inference is a recurring cost these models do not have.
Are open models like these reliable enough to put behind a production agent?
For narrow, well-scoped tasks, yes, and thousands of teams already do it. The risk is not accuracy, it is behaviour under quantization: reasoning-enabled quants can burn through output token limits before producing an answer, which looks like a hang rather than an error. Test with your real prompts at your real quantization level, set an output ceiling, and put an approval step in front of anything destructive before you let it run unattended.
