Gemma 4 31B vs Qwen 3.8 27B: Which Local Model Wins for AI Agents?

Updated September 2026: this comparison now covers Qwen 3.8 27B, which replaced Qwen 3.6 27B as the newest dense Qwen at this size.

Both fit in 24GB at 4-bit. Both are Apache 2.0. One of them nails strict tool schemas and the other writes code that actually runs. Here's which is which.

Skip the GPU cluster.

Bring your own Gemma or Qwen endpoint, get the visual builder and unlimited skills, deploy in 60 seconds. Free forever, not a trial. Start free → No credit card · No Docker · No config files

If you searched for "Gemma 4 27B" or "Qwen 4 27B," you are about to be told that neither of them exists. That is the short version. The useful version is what to run instead, and that comes down to two dense models that both fit on a single 24GB consumer GPU at 4-bit: Google's Gemma 4 31B and Alibaba's Qwen3.8-27B.

They are genuinely different animals under the hood. One is a conventional dense transformer that posts Google's best long-context retrieval numbers and, in independent testing, never missed a tool call. The other mixes linear attention layers into a dense model built to keep long sequences cheap, and it finished every coding task in the same test where Gemma finished half.

This post covers what actually separates them for agent work: tool calling, code generation, speed, VRAM headroom, and which size you should be comparing in the first place.

What changed in October 2026

The local test results on this page are dated history and stay as they were. Here is what has moved around them, checked on 13 October 2026.

• Still no Gemma 4 27B. Google's Gemma release notes list one release since June: EmbeddingGemma 2 (740M) on 6 October, an embedding model rather than a chat model. Gemma 4 31B is still the dense Gemma to compare with a 27B.
• Still no Qwen 4. Alibaba has not released Qwen 4 weights or given a date. Qwen3.8-27B remains the newest dense Qwen at this size.
• Hosted small models got cheaper than ever. If you're weighing a local 27B against an API, Claude Haiku 5.5 (7 October) lists at $0.10/$0.50 per million tokens for prompts up to 100K tokens ($0.50/$2.50 for the whole request above that). GLM-5.3 Flash is $0.15/$0.50. DeepSeek V4.1 Flash is $0.30/$1.20 at peak (01:00–04:00 and 06:00–10:00 UTC on weekdays) and half that off-peak.
• The mid-tier moved too. Claude Sonnet 5.5 and GPT-6.1 Sol both list at $2/$10. Claude Sonnet 4.6 is still on sale at $3/$15 but is no longer the newest or the cheapest Sonnet. Claude Opus 5.5 is $4/$20. All prices are from each provider's own pricing page.

None of the hosted models above went through the local tests on this page, so there are no benchmark comparisons with Gemma or Qwen here.

First, There Is No Gemma 4 27B

If you searched for "Gemma 4 27B," you are not alone, and you are not going to find it. Gemma 3 topped out at 27B. Gemma 4 does not have a 27B.

The Gemma 4 family ships five sizes: E2B, E4B, 12B, 26B A4B, and 31B. The two that sit near 27B are very different:

• Gemma 4 31B is dense, with 30.7B parameters. Every parameter works on every token. This is the one this post compares, because it is the like-for-like match for a dense 27B.
• Gemma 4 26B A4B is a mixture-of-experts model with 25.2B total parameters and only 3.8B active per token. It is faster and weaker, and comparing it to a dense 27B is comparing two different kinds of model. Our Gemma 4 vs Qwen 3.5 size-by-size comparison pairs it with Qwen's 35B-A3B instead.

Google has added to the family since launch without adding a 27B. Multi-token prediction drafters for speculative decoding arrived in May 2026, and the unified 12B, which handles text, image and audio in one encoder-free decoder, landed in June 2026. There is still no Gemma 4.1 and still no 27B.

Is There a Qwen 4 27B?

Not yet. As of 13 October 2026, Qwen 4 has not been released and Alibaba has not given a date.

What does exist is Qwen3.8-Flash-Next, published in August 2026, which Alibaba describes as an experimental preview of the architecture that will underpin Qwen 4. It is a 125B mixture-of-experts model with about 6B active parameters, so it is not a drop-in for a 27B dense model and it will not fit on your 24GB card. It is there so the community can look at the architecture early, the same role Qwen3-Next played ahead of Qwen 3.5.

The newest dense Qwen you can actually run at this size today is Qwen3.8-27B, released August 14, 2026. When Qwen 4 ships at this size, we will update this comparison.

Which Sizes Actually Exist (October 2026)

What You Are Actually Comparing

Before the benchmarks, a quick grounding.

Google Gemma 4 31B is the dense flagship of Google DeepMind's Gemma 4 family, released April 2026. It has 30.7B parameters, a 256K context window, configurable thinking modes, and native function calling. Input is text and images, and it can analyze video as a sequence of frames. It does not take audio: in Gemma 4, native audio is limited to the E2B, E4B, and 12B models. Ollama's gemma4:31b download is 20GB.

Alibaba Qwen3.8-27B is the dense model in the Qwen 3.8 family, released August 14, 2026. It keeps the hybrid layout of Qwen 3.6: 64 layers built from 16 repeating blocks, each of which is three Gated DeltaNet linear-attention layers followed by one gated attention layer. Context window is 262K tokens natively, extensible to roughly 1M, though Ollama serves it at 256K. Text, image, and video input. Thinking is on by default with configurable depth (low, medium, xhigh, with xhigh as the default), so you can dial reasoning cost per request. Ollama's qwen3.8:27b download is 18GB. Apache 2.0.

Still on Qwen 3.6 27B? It is the same size on disk (qwen3.6:27b is also 18GB) and the same architecture family, just a generation behind. Alibaba's own card shows 3.8 ahead of 3.6 on every agentic benchmark it publishes. The one real reason to stay put is that some agent frameworks hang on Qwen 3.8 through Ollama's /v1 endpoint. Our Qwen3.8 27B tool-calling fix covers the workaround. Otherwise, for new builds, start on 3.8.

Both models are Apache 2.0. Both are genuinely useful for building real AI agents. Neither requires a cloud API call. On tighter hardware, see our Gemma 4 12B vs Qwen 3.5 9B comparison for the smaller siblings.

Tool Calling: Gemma Wins This One

Here's the thing about local models and agents. Most developers compare them on MMLU or HumanEval and then wonder why their agent hallucinates a JSON field name at 2 AM.

We used to say Qwen won this category. Independent testing says otherwise, and we are correcting it.

An independent test on a single RTX 4090, running all three models at Q4_K_M under llama.cpp with a 64K context, scored strict tool calling like this:

Gemma 4 31B was perfect on both. Qwen 3.8 matched it on single calls and dropped two of thirty multi-step runs. The interesting result is Qwen 3.6, which missed eleven single calls out of ninety. If tool-call reliability is why you are still on 3.6, that is an argument for moving, not staying.

Google's own number backs the Gemma result up: 76.9% on Tau2, an agentic tool-use benchmark, against 16.2% for Gemma 3 27B. It handles Google-style function declarations well, unsurprisingly given its lineage, and its error recovery when a tool returns an unexpected response is often cleaner.

Bottom line: Gemma 4 31B is the safer pick for strict schema compliance. Test your own tool schemas before you commit either way, because a 15-tool schema with nested objects is not what any of these benchmarks measure.

Coding: Where Qwen Still Wins Decisively

Qwen 3.8 27B generates cleaner code, and the gap is not subtle.

In the same 24GB test, on a twelve-task coding suite scored pass@1:

Qwen 3.8 completed every task. Gemma finished half.

Alibaba's own model card points the same direction. These are vendor-reported numbers, so treat the size of the gap with some suspicion, but the generation-over-generation jump is large and consistent:

The one benchmark both vendors publish is LiveCodeBench v6: Qwen 3.8 27B at 90.3 against Gemma 4 31B at 80.0. Google does not publish a SWE-bench score for the 31B, so anyone showing you a head-to-head SWE-bench table for these two is comparing numbers from different harnesses. We previously ran such a comparison in this post and have removed it.

More importantly for agent builders: Qwen hallucinates library names less. When your agent is writing a Python script to process a file, you want the imports to be real. Qwen 3.8 27B is conservative here. It reaches for standard library solutions before inventing a convenience wrapper that does not exist.

The one area where Gemma 4 pulls ahead: it tends to write more readable, commented code. Qwen optimizes for correctness, Gemma optimizes for something that reads like a senior developer wrote it. Depending on whether your agent explains its own output, that might matter.

Multimodal: Closer Than You Have Been Told

This is where most comparisons of these two get it wrong.

Gemma 4's headline audio support does not reach the 31B. Google's model card lists the 31B as text and image input, with video handled as a sequence of frames. Native audio lives on the smaller E2B, E4B, and 12B models.

Qwen 3.8 27B takes text, images, and video. No audio either.

So at this weight class, neither model will transcribe a voicemail for you, and both can look at screenshots and video. If your agent genuinely needs audio in a single model call, the answer is not a bigger model. It is Gemma 4 12B, which handles text, image, and audio in one encoder-free decoder and runs in about 8GB. Our Gemma 4 12B vs Qwen 3.5 9B comparison covers it.

If your agent is text-and-image, which covers the majority of real-world agent tasks, multimodal is not the deciding factor between these two.

Context Window: Both Big, Both Capped by Your VRAM

262K native context on Qwen 3.8 27B, extensible to roughly 1M. Gemma 4 31B ships with 256K. On paper that is close to a tie. In practice, on a 24GB card, neither number is the one that matters.

Gemma 4 31B has the stronger published retrieval result. Google reports 66.4% on MRCR v2 (8 needles at 128K), against 13.5% for Gemma 3 27B. If your agent has to find specific facts buried deep in a long document, that is the number to care about.

Qwen 3.8 27B's advantage is cost per token of context. Its Gated DeltaNet layers use linear attention, so memory and compute grow more gently as the sequence gets longer. For agents running persistent memory loops or ingesting large documents mid-conversation, that keeps long sessions cheaper to serve. Watch the thinking mode, though: reasoning tokens are generated into the conversation too, and at the default xhigh depth on long tasks they add up fast. Dropping to medium or low for routine turns is the single easiest latency win on this model.

The real ceiling is the KV cache. In the independent 24GB test, the highest stable context across all three models was 64K, not the advertised 256K or 262K. Plan your agent around that number, not the spec sheet.

Speed and VRAM: The Numbers That Decide It

Measured on one RTX 4090 (24,564 MiB usable, display disabled), llama.cpp, Q4_K_M, flash attention on, 64K context:

Two things to take from this.

First, the speed gap is real but small: Qwen is about 9% faster on decode. That is not the reason to choose it.

Second, the headroom gap is the reason. Gemma 4 31B could not hold 64K context with an F16 KV cache on a 24GB card at all. It needed the KV cache dropped to Q8_0 to fit, and even then it left only 2.55 GiB free. Qwen left 4.20 GiB with full F16 precision. Once you add a vision projector, a desktop display, speculative decoding, or any concurrency, that difference is what determines whether your agent runs or out-of-memories at 3 AM.

These are numbers from one independent test at one quantization on one inference server. Your results will move with quant level, KV precision, and whether you are on llama.cpp, vLLM or Ollama. Treat them as the shape of the difference, not as guarantees.

Two more notes. Qwen 3.8 ships an MTP (multi-token prediction) draft option that can speed decoding, but outputs differ from the baseline, so validate per task before switching it on. Gemma 4 has MTP drafters across the lineup too, with Google claiming up to a 3x speedup on the right serving stack.

A third-party intelligence index also places Qwen 3.8 27B well ahead of Gemma 4 31B overall, 34 against an estimated 19, with hosted output speed of roughly 41 tok/s against 35. Gemma is the clear winner on time to first token in those hosted measurements, at about 1 second against 4, which reflects Qwen thinking by default.

Speed by Size

If you are not locked to this weight class, here is the ladder. Sizes are the Ollama Q4 downloads.

The 12B and 26B A4B decode speeds are relative, not measured on the same rig as the two headline models. We have not seen a clean independent number for either at Q4 on a consumer card, so we are not going to invent one.

Which Gemma 4 Should You Compare With Qwen 3.8 27B?

A lot of the confusion around this matchup is people comparing across weight classes without realising it.

If you land on the sparse comparison and your agent mostly writes code, Ornith 1.5 35B is worth a look too: same 35B-A3B shape as Qwen 3.6 35B-A3B, and its model card reports 79.0 against Qwen 3.6's 73.4 on SWE-bench Verified.

The Agent Decision Matrix

Not every agent task maps cleanly to one model. Here is the honest breakdown:

Pick Qwen 3.8 27B if: your agent writes or runs code, it drives a browser or uses computer-use tools, or you need the most VRAM headroom you can get on a 24GB card.

Pick Gemma 4 31B if: strict tool-schema compliance matters more to you than anything else, you need the best long-document retrieval (66.4% on MRCR v2), you want the lowest time to first token, or your team is already in Google's function-calling ecosystem.

Stay on Qwen 3.6 27B if: it is already in production and your framework hits the Qwen 3.8 /v1 tool-calling hang. That is the only good reason left.

Pick neither if: your agent needs audio input. Drop to Gemma 4 12B.

Where they are roughly equal: instruction following on straightforward tasks, image and video understanding, RAG pipeline performance, general reasoning, and multilingual tasks.

If your agent does both heavy tool calling and code execution, Qwen 3.8 is the better all-rounder. Two missed multi-step runs out of thirty is a retry-loop problem. Six coding tasks out of twelve is an agent that does not work.

What Nobody Tells You About Running 27B-Class Models for Agents

The benchmarks show one thing. Production shows another.

The biggest practical difference between running Gemma 4 31B and Qwen 3.8 27B in an agent loop is not the model. It is the system prompt size, the tool schema serialization format, and whether your agent framework handles partial tool-call responses gracefully.

Thinking mode adds latency on both. Qwen 3.8 thinks at xhigh by default, and Gemma 4 has a configurable thinking mode of its own. If you leave deep reasoning on for every turn, you will see 2-5x slower responses on simple tasks. You need a routing layer that decides when thinking is worth the overhead and when fast mode is fine. That is not complicated, but it is work that does not show up in any benchmark. Our write-up on how thinking mode behaves in agents covers the routing pattern.

Vision input requires the right inference server. Not every Ollama setup or llama.cpp build exposes image and video input cleanly for every model. If you are deploying either model for multimodal agents, verify your serving layer handles those input types before you build your agent around them.

Both models benefit from well-structured system prompts. Vague instructions produce worse results than tight, specific role definitions with explicit tool-use guidelines. That is not a model-specific insight. It is just agent development hygiene.

For reference on how to build skills and tool integrations that work cleanly across local models, the skills that actually reduce token usage breakdown is worth reading alongside this comparison. The same patterns that prevent token bloat in hosted models matter even more when you are working with a 64K practical context at 49 tokens per second locally.

Connecting Local Models to Real Agent Workflows

Here is a question worth asking: why are you running local in the first place?

If the answer is data privacy, both models give you that. No API call leaves your network.

If the answer is cost, the math is more complex than it looks. Local inference on 27B-class models is not free. A single 4090 at current electricity prices, amortized over three years, costs roughly $0.08 to $0.12 per hour of inference. At typical agent task volumes, hosted BYOK models on platforms with zero inference markup often come out cheaper than the electricity bill. The honest hardware math and the full total cost of ownership comparison are both worth a read before you buy a GPU.

If the answer is latency, local wins cleanly. Sub-100ms first-token latency on a good setup beats any hosted API.

If the answer is compliance or air-gap requirements, local is the only answer.

For teams who want the flexibility of local models in development but the reliability of managed infrastructure in production, BetterClaw supports 28+ LLM providers, including self-hosted endpoints. You can build and test against your local Gemma or Qwen instance, then point your production agents at a managed provider when you need reliability guarantees. The agent logic does not change. Just the endpoint. Free plan available, no credit card.

The Honest Verdict

Qwen 3.8 27B is the better all-around agent model as of October 2026. It matched Gemma on single tool calls, swept a twelve-task coding suite that Gemma half-finished, leads on every agentic benchmark Alibaba publishes, decodes about 9% faster, and leaves 4.2 GiB of headroom on a 24GB card where Gemma leaves 2.55 GiB and has to compromise its KV cache to get there.

Gemma 4 31B is the right choice when strict schema compliance is the whole job. Ninety out of ninety and thirty out of thirty is a perfect record that Qwen did not quite match, and if your agent's failure mode is a malformed tool call rather than a broken script, that record is worth more than the coding gap. It also owns long-document retrieval and time to first token.

Qwen 3.6 27B is now a legacy choice. It is behind 3.8 on everything published and it missed eleven of ninety single tool calls in independent testing. Move unless the /v1 hang is blocking you.

Neither of the two current models is a wrong choice. The fact that you can run either on a single consumer GPU, for free, with an Apache 2.0 license, is something that would have sounded like science fiction in 2022.

The more interesting constraint is not which model. It is what you build around it. Tool schemas, system prompt quality, context management, and memory architecture matter more than the model choice for most real agent tasks. A well-engineered agent on Gemma 4 31B will outperform a sloppily built one on anything.

Pick the model that fits your workload. Then invest your engineering time in the agent architecture. That is where the real performance gap gets made.

If you want to connect either of these models to a full agent stack without building the infrastructure yourself, give BetterClaw a try. Bring your own inference endpoint, use the visual builder, and get unlimited skills out of the box. Free plan, no credit card, first deploy in 60 seconds. We handle the agent infrastructure so you can focus on what the agent actually does.

Frequently Asked Questions

Is there a Gemma 4 27B?

No. Gemma 4 ships at E2B, E4B, 12B, 26B A4B, and 31B. The 27B size belonged to Gemma 3. If you want a dense Gemma 4 to compare against a 27B model, use Gemma 4 31B. If you want the faster option, the 26B A4B is a mixture-of-experts model with 3.8B active parameters per token. Google has since added multi-token prediction drafters across the lineup and a unified 12B with text, image and audio input, but still no 27B and no Gemma 4.1.

Is there a Qwen 4 27B?

No. Qwen 4 is unreleased as of 13 October 2026 and Alibaba has not announced a date. Qwen3.8-Flash-Next, published in August 2026, is an experimental preview of the architecture Qwen 4 will be built on, but it is a 125B mixture-of-experts model with about 6B active parameters, not a dense 27B, and it will not run on a 24GB card. The current dense 27B is Qwen3.8-27B, on Ollama as qwen3.8:27b at 18GB.

What is the difference between Gemma 4 31B and Qwen 3.8 27B for AI agents?

Both are dense, Apache 2.0 models that run on a 24GB GPU at 4-bit. Qwen 3.8 27B is stronger on code generation and agentic tasks, scoring 90.3 to Gemma's 80.0 on LiveCodeBench v6 and completing 12 of 12 coding tasks in independent testing where Gemma completed 6. Gemma 4 31B is perfect on strict tool calling in the same test and has the stronger long-context retrieval result (66.4% on MRCR v2). Neither takes audio input; in Gemma 4 that is limited to the E2B, E4B, and 12B models.

Which is better for tool calling, Gemma 4 31B or Qwen 3.8 27B?

In independent testing on a 24GB card at Q4, Gemma 4 31B was perfect on both strict single tool calls (90/90) and multi-step runs (30/30). Qwen 3.8 27B matched it on single calls at 90/90 but missed 2 of 30 multi-step runs. Qwen 3.8 wins clearly when the agent also writes and runs code. Test your own tool schemas before choosing, because neither benchmark reflects a large nested schema in your own framework.

Should I use Qwen 3.8 27B instead of Qwen 3.6 27B?

For new builds, yes. Qwen3.8-27B is ahead of 3.6 on every agentic benchmark Alibaba published, including SWE-bench Pro (61.7 vs 53.5), LiveCodeBench v6 (90.3 vs 83.9), Terminal Bench 2.1 (73.0 vs 63.4), OSWorld (84.3 vs 63.9), WebArena (64.8 vs 48.8) and AndroidWorld (81.9 vs 70.3). It is also the same 18GB download on Ollama. In independent testing 3.6 missed 11 of 90 single tool calls where 3.8 missed none. Stay on 3.6 only if your framework hits the Qwen 3.8 /v1 tool-calling hang, and see our Qwen3.8 27B tool-calling fix for the workaround.

Gemma 4 12B vs Qwen 3.8 27B: which should I run?

They are different weight classes, so pick by VRAM first. Gemma 4 12B runs in about 8GB, fits a 12GB card, and takes audio input. Qwen 3.8 27B needs about 18GB and is much stronger on code and agentic tasks. If you have 24GB, run Qwen 3.8 27B. If you have 12GB or you need audio in a single model call, run Gemma 4 12B.

How do I run Gemma 4 31B or Qwen 3.8 27B locally for AI agents?

Both models run locally via Ollama, llama.cpp, or vLLM. On Ollama, pull gemma4:31b (20GB) or qwen3.8:27b (18GB). For Q4 quantized inference, a single 24GB GPU such as an NVIDIA 4090 or A5000 is sufficient for either model if you cap the context length. Expect 64K to be the practical ceiling on 24GB, not the advertised 256K. Both have thinking modes that need configuring so they only activate when a task needs them.

How much VRAM do Gemma 4 31B and Qwen 3.8 27B need at 64K context?

On a 24GB card at Q4_K_M, Qwen 3.8 27B peaked at 20,266 MiB with a full F16 KV cache, leaving 4.20 GiB free. Gemma 4 31B could not fit 64K with an F16 KV cache at all; dropping to a Q8_0 KV cache brought it to 21,956 MiB with 2.55 GiB free. If you plan to run a vision projector, a desktop display, or any concurrency alongside the model, that headroom difference matters more than the speed difference.

Is it worth running a 27B-class local model versus using a hosted API for agents?

It depends on your requirements. Local 27B-class models provide data privacy, no per-token API cost, and low latency. However, the total cost of ownership including hardware amortization, electricity, and maintenance is often higher than using a BYOK hosted provider with zero inference markup. For compliance or air-gap requirements, local is the only option. For most teams, a hybrid approach, developing locally and deploying via a managed provider, balances cost and reliability best.

Are Gemma 4 31B and Qwen 3.8 27B reliable enough for production agent workflows?

Both models are production-grade at this scale. The more relevant reliability questions are around your serving infrastructure, not the models themselves. Inference server uptime, quantization quality, context window management, and tool schema handling matter more than the base model for production reliability. In agent deployments specifically, the quality of your system prompt, tool definitions, and memory architecture will have a larger impact on reliability than the model choice between these two.

Test local, deploy anywhere.

Build against your local Gemma or Qwen endpoint, then point production at any of 28+ providers. Same agent logic. Free forever, not a trial. Start free →