Two open-weight families, thirteen models between them, and one question nobody answers properly: which pair should you actually be comparing?
The model finished downloading at 1:40 AM. Twenty-two gigabytes, pulled over a hotel wifi connection, because someone on a forum said this was the one.
It loaded. It answered the first question in about forty seconds. Then it started thinking, and thinking, and it was still thinking when the output limit cut it off mid-sentence, having produced eleven hundred tokens of reasoning and zero tokens of answer.
That is the actual experience of picking a local model in 2026, and it is not a benchmark problem. It is a matchup problem.
Now here is why most comparison articles get this wrong before they start.
The full size lineup, side by side
Both families landed within eight weeks of each other in 2026. Both are Apache 2.0. Both are multimodal. That is where the similarity ends, because they are shaped completely differently.
Gemma 4 (Google DeepMind, April 2, 2026 for E2B, E4B, 26B A4B and 31B, with the 12B added June 3, 2026, Apache 2.0, 140+ languages):
| Model | Parameters | Context | Approx VRAM at 4-bit | Notes |
|---|---|---|---|---|
| E2B | ~2.3B effective | 128K | ~4GB | Phone and browser class, native audio |
| E4B | ~4.5B effective | 128K | ~5 to 6GB | Laptop class, native audio and video |
| 12B | 11.95B unified | 256K | ~7 to 8GB | Encoder-free, handles text, image and audio in one decoder |
| 26B A4B | 25.2B total, 3.8B active | 256K | ~14 to 18GB | Mixture of Experts, the efficiency play |
| 31B | 30.7B dense | 256K | ~18 to 20GB | Quality peak of the family |
Qwen 3.5 (Alibaba, rolled out across February and March 2026, Apache 2.0, 201 languages):
| Model | Parameters | Context | Approx VRAM at 4-bit | Notes |
|---|---|---|---|---|
| 0.8B | 0.8B | 262K | ~1 to 2GB | Edge and embedded |
| 2B | 2B | 262K | ~2 to 3GB | Phone class |
| 4B | 4B | 262K | ~4 to 5GB | Laptop class |
| 9B | 9B | 262K | ~6 to 8GB | Best value mid-tier |
| 27B | 27B dense | 262K | ~16 to 20GB | Strongest dense open coder in the family |
| 35B A3B | 35B total, 3B active | 262K | ~20 to 24GB | MoE, runs surprisingly small |
| 122B A10B | 122B total, 10B active | 262K | Workstation class | Agentic benchmark leader |
| 397B A17B | 397B total, 17B active | 262K | Multi-GPU only | Flagship |
VRAM figures are rules of thumb at 4-bit quantization. Add headroom for context and KV cache, and add a lot of headroom if you plan to use anything close to that 262K window.
Two things jump out. Qwen's smallest model is a third the size of Gemma's smallest. And every single Qwen 3.5 model ships the same 262,144-token context, extensible to roughly a million with YaRN, while Gemma splits at 128K for the small models and 256K for everything above.
Qwen gives you more rungs on the ladder. Gemma gives you fewer rungs but puts more on each one.
Which pairing should you actually compare?
This is the part that trips people up, and it is why "Gemma 4 vs Qwen 3.5" is close to a meaningless question until you make it specific.
Parameter counts do not line up between these families. Gemma's E2B and E4B use per-layer embeddings, so the "E" stands for effective, meaning the parameters actually loaded into VRAM during computation. Comparing a 4.5B effective model to a 4B dense model by the number on the label tells you nothing useful.
Worse, Gemma's 26B A4B is a Mixture of Experts that activates only 3.8B parameters per token. It occupies the memory footprint of a 26B model and the compute cost of a 4B one. Qwen's 35B A3B does the same trick with 3B active. So a "26B vs 35B" comparison is really a comparison of two sparse models with similar active sizes and different total footprints.
Here is the mapping that actually holds up:
- Gemma 4 E2B against Qwen 3.5 2B. Phone and edge class.
- Gemma 4 E4B against Qwen 3.5 4B. Laptop class. Gemma adds native audio and video here.
- Gemma 4 12B against Qwen 3.5 9B. The closest and most useful matchup in the entire comparison.
- Gemma 4 26B A4B against Qwen 3.5 35B A3B. Sparse against sparse.
- Gemma 4 31B against Qwen 3.5 27B. Dense against dense, and the one everyone benchmarks.
Above that line, Qwen runs alone. There is no Gemma 4 answer to the 122B or the 397B. And there is no Gemma 4 27B at all, whatever a search box suggests; the 27B size was Gemma 3's.
Head to head by size tier
Small tier: E2B and E4B against Qwen 2B and 4B
Gemma wins this tier on capability per gigabyte, and it is not especially close on multimodal work. The E2B and E4B ship native audio input, including speech recognition, and handle video at roughly a frame per second for up to a minute. Qwen's small models are multimodal too, but audio is not their story.
Qwen wins on context. A 2B model with a 262K window is unusual and genuinely useful for chewing through long documents on hardware that has no business doing so.
If you are building something that listens, take Gemma. If you are building something that reads a lot, take Qwen.
Mid tier: Gemma 4 12B against Qwen 3.5 9B
The most interesting fight, and the one most people running a 16GB machine will actually have.
Gemma's 12B is architecturally strange in a good way. It is encoder-free and unified, meaning it drops the separate vision and audio encoders and projects raw image patches and audio directly into the decoder. Third-party comparisons put it ahead on practical coding benchmarks and multilingual reasoning.
Qwen's 9B takes knowledge, science, and agentic tool-use benchmarks by clear margins, with the tool-use gap being one of the widest in the whole comparison. It also leaves you a little more headroom, since 9B at 4-bit sits comfortably under 7GB while the 12B wants closer to 8.
Neither has a published Arena Elo at this tier, so anyone quoting you one is guessing.
Large tier: Gemma 4 31B against Qwen 3.5 27B
Both fit on a single 24GB card. Both claim the other's territory. The published numbers disagree depending on who ran them, which is the most honest thing anyone can tell you about this matchup.
Google's card and third-party runs put Gemma 4 31B at 89.2 on AIME 2026, 80.0 on LiveCodeBench v6, 84.3 on GPQA Diamond, 85.2 on MMLU Pro, and 66.4 on MRCR v2 for long-context retrieval. That last number is the one that matters most: Gemma 3 27B, the previous top of the family, scored 13.5 on the same test. The 128K window on Gemma 3 was mostly decorative. The 256K window on Gemma 4 actually retrieves.
Qwen 3.5 27B posts 72.4 on SWE-bench Verified, which ties GPT-5 mini and is a serious number for a fully open dense model at that size. One independent evaluation of the BF16 checkpoints found Gemma 4 31B ahead on most benchmarks with MMLU Pro and GPQA Diamond as the exceptions. Another leaderboard has Qwen well ahead on coding, driven almost entirely by SWE-Rebench.
Both cannot be right, which tells you something.
We put the dense large-tier fight against Qwen's newer 27B in our Gemma 4 31B vs Qwen 3.6 27B comparison.
Where Qwen 3.6 and Qwen 3.8 fit
Qwen 3.5 was the current line when this comparison went up in August. It is not the newest Qwen any more, and it matters exactly which newer releases you can actually run.
Qwen 3.6 (April 2026) shipped open weights at two sizes: a 35B-A3B mixture-of-experts model and a 27B dense model released April 22. Both take text, image, and video, both have a 262K native context, and both are on Ollama as qwen3.6:35b-a3b and qwen3.6:27b. Alibaba's own card for Qwen3.6-27B puts it ahead of Gemma 4 31B on SWE-bench Verified (77.2 against 52.0), GPQA Diamond (87.8 against 84.3), LiveCodeBench v6 (83.9 against 80.0), and AIME 2026 (94.1 against 89.2). Those are vendor-reported numbers, and Google's card shows the 31B's strength on long-context retrieval, which Alibaba's table does not include.
Qwen 3.7 (Max and Plus, May and June 2026) is API-only. No open weights, nothing to pull in Ollama.
Qwen 3.8 is the current open-weight line. Qwen3.8-27B landed on August 14, 2026: dense, Apache 2.0, 262K native context, text, image, and video input, with thinking on by default and adjustable reasoning depth. It is on Ollama as qwen3.8:27b at about 18GB. The only other 3.8 open release is a 2.4T-parameter Max-class model that no home setup will run. If you plan to put it behind an agent, read our Qwen3.8 27B tool-calling fix first, because some agent integrations hang on it through Ollama's /v1 endpoint.
What that changes in the matchups above:
- Small and mid tiers: nothing. Neither Qwen 3.6 nor 3.8 ships an open model below 27B, so E2B, E4B, and 12B are still best compared against Qwen 3.5's 2B, 4B, and 9B.
- Sparse tier: Gemma 4 26B A4B can now be compared against Qwen 3.6 35B A3B as well as the 3.5 version.
- Dense large tier: Gemma 4 31B's real rival is now Qwen 3.6 27B or Qwen 3.8 27B, not Qwen 3.5 27B. Same memory footprint, newer model.
What the benchmarks say across the range, and where they stop being useful
Here is the pattern once you read enough of these tables. Static benchmarks favor Qwen on more individual rows. Human preference testing favors Gemma at the top end. Gemma 4 31B reached third on Arena's text leaderboard at roughly 1452 Elo, with the 26B MoE at sixth on around 1441.
That 26B result deserves a second look. It reaches roughly 97% of the dense 31B's quality while activating 3.8B parameters per token instead of 30.7B. Eight times less compute per step, sixth place on a leaderboard full of models with far more active parameters.
And here is what nobody tells you about running either family at home. Quantization does not just cost you accuracy, it costs you completions. One test of an INT4 Qwen 3.5 quant with reasoning enabled found it truncated around 70% of AIME 2025 answers by hitting the 32K output ceiling, against roughly 30% for the full-precision model. The weights barely lost accuracy. The model just started thinking too much to finish.
That is the failure mode you will actually hit at 1:40 AM. Not wrong answers. Unfinished ones.
If you are choosing a model to sit behind an agent rather than a chat window, that distinction matters more than any leaderboard row, and our guide to picking the right model for the task goes into which failure modes actually break agent loops.
Quick aside on where this fits. We build a no-code agent platform, and we are bring-your-own-key across 28 providers with no markup on inference, which means we have no financial reason to push you toward one model over another. If you want to point an agent at whichever of these two you pick, our provider and BYOK setup takes about a minute and the free plan does not need a card.
Which to pick by hardware
Skip everything above and start here if you already know your VRAM.
8GB. Qwen 3.5 9B at 4-bit, or Gemma 4 E4B if you need audio. The 9B is the best capability-per-gigabyte model in either family and it leaves you room to actually use the context window. Gemma's 12B will technically load and will then fight you for memory.
16GB. This is the real decision point. Gemma 4 12B if your work is multimodal or multilingual. Qwen 3.5 9B if your work is agentic tool calling, with the leftover memory spent on longer context. The 26B MoE is also reachable here at aggressive quantization, though you will feel it.
24GB. The finalists are Gemma 4 26B A4B and a Qwen dense 27B. The Gemma MoE is faster and frugal, roughly 14 to 18GB at 4-bit, and multimodal. The Qwen dense 27B is the stronger coder, roughly 16 to 20GB, and at this size you can pick Qwen 3.5, 3.6, or 3.8. Gemma 4 31B fits at 18 to 20GB but leaves you little room for context.
32GB and up. Gemma 4 31B dense at 4-bit with real room for context (8-bit needs about 35GB), or Qwen 3.5 or 3.6 35B A3B, which runs far lighter than its total parameter count suggests because only 3B activate per token. Above 48GB, Qwen's 122B A10B is the agentic benchmark leader and Gemma has no answer.
A practical note for anyone running these behind an always-on agent rather than an interactive session: memory ceiling is not your constraint, sustained throughput is. We covered the difference in our writeup on running a local LLM agent on consumer hardware, and if you are wiring a local model into a self-hosted agent stack, the hardware requirements for local models with OpenClaw covers the parts that break first.
The closest matchup deserves its own page
If you are on a 16GB machine, the Gemma 4 12B against Qwen 3.5 9B decision is the only one that matters to you, and it is genuinely close enough that a paragraph does not settle it.
Both sit under Apache 2.0. Context windows are nearly identical at 256K and 262K. Both run in Ollama, LM Studio, llama.cpp, and vLLM. The split comes down to Gemma's unified multimodal decoder against Qwen's tool-calling and knowledge scores, plus a gigabyte or so of memory headroom.
We ran that one properly, benchmark by benchmark, in the full Gemma 4 12B vs Qwen 3.5 9B comparison. If your machine has 16GB, read that instead of this. If you have a 24GB card and want the dense top end, the Gemma 4 31B vs Qwen 3.6 27B comparison is the one to read.
If any of this resonated, start free with BetterClaw. One agent, every core feature, no credit card, and bring your own key so you pay your provider directly with nothing added on top. Pro is $49 a month for five agents, or $39 on annual. Full pricing is here. Your first deploy takes about a minute, and the model choice stays yours.
One closing thought. A year ago the interesting question about open models was whether they could compete with closed ones. That question is settled. The interesting question now is which of the eight or thirteen models a single vendor ships in a single quarter is the right one for your specific machine, and no leaderboard is ever going to answer that for you. Download two. Run your own workload through both. The one that finishes its sentences wins.
Frequently Asked Questions
What is the difference between Gemma 4 and Qwen 3.5?
Gemma 4 is Google DeepMind's open-weight family, released April 2, 2026 with the 12B added on June 3, and it has five sizes from E2B up to a 31B dense flagship (there is no Gemma 4 27B), 140+ languages, and native audio on the smallest models. Qwen 3.5 is Alibaba's family released across February and March 2026, with eight sizes from 0.8B to a 397B Mixture of Experts flagship, 201 languages, and a uniform 262K context window. Both are Apache 2.0 licensed with no commercial restrictions.
Should I use Qwen 3.6 or Qwen 3.8 instead of Qwen 3.5?
Only if you have the memory for a 27B-class model. Qwen 3.6 ships open weights at 27B dense and 35B A3B, and Qwen 3.8's open-weight local model is Qwen3.8-27B, released August 14, 2026. Both are on Ollama. Below 27B, Qwen 3.5 is still the newest Qwen you can run, so on 8 to 16GB machines the Gemma 4 against Qwen 3.5 comparison still holds. Qwen 3.7 is API-only and cannot be run locally.
How does Gemma 4 31B compare to Qwen 3.5 27B?
They are the closest large-tier matchup and both fit on a single 24GB GPU. Gemma 4 31B leads on multimodal, multilingual, and long-context retrieval, scoring 66.4 on MRCR v2 against 13.5 for its Gemma 3 predecessor. Qwen 3.5 27B leads on coding, posting 72.4 on SWE-bench Verified. Independent evaluations disagree on the overall winner, which is a reasonable signal that your own workload should decide it.
How do I run Gemma 4 or Qwen 3.5 locally?
Both families are supported in Ollama, LM Studio, llama.cpp, vLLM, and MLX, with GGUF quantizations published on Hugging Face. Pull the model that fits your VRAM at 4-bit, then cap your context length rather than accepting the 256K default, because KV cache allocation at full context will exhaust memory on consumer hardware long before the weights do.
Is Gemma 4 or Qwen 3.5 cheaper to run?
Both are free to download under Apache 2.0, so the cost is entirely hardware and electricity. The efficiency winners are the sparse models: Gemma 4 26B A4B activates 3.8B parameters per token and reaches roughly 97% of the dense 31B's quality, and Qwen 3.5 35B A3B activates only 3B. If you are comparing against hosted APIs instead, remember that any managed agent platform charging a markup on inference is a recurring cost these models do not have.
Are open models like these reliable enough to put behind a production agent?
For narrow, well-scoped tasks, yes, and thousands of teams already do it. The risk is not accuracy, it is behaviour under quantization: reasoning-enabled quants can burn through output token limits before producing an answer, which looks like a hang rather than an error. Test with your real prompts at your real quantization level, set an output ceiling, and put an approval step in front of anything destructive before you let it run unattended.
