Ornith-1.5-35B-A3B for AI Agents: What It Is, What It Needs, How It Performs

A 35B model that only wakes up 3B parameters per token, fits on one 24GB card at Q4, and posts a 79 on SWE-bench Verified. Here's the hardware math, the exact Ollama, OpenClaw and Hermes configs, and where it actually belongs in a routing stack.

The r/hermesagent thread hit 192 upvotes with "MIND BLOWING" in the title, which is usually the signal to be skeptical. So I pulled the GGUF, read the model card twice, and ran it against the same agent loop I use for every local model. Some of the hype holds. Some of it is people confusing the 9B and the 35B. Here's the sorted version.

What Ornith-1.5 actually is

Ornith-1.5-35B-A3B is the mid-size mixture-of-experts member of the Ornith-1.5 family from the Ornith team (published under DeepReinforce for the 1.0 release, now under the ornith-ai org on Hugging Face). The A3B in the name is the whole story: about 35B total parameters, roughly 3B activated per token. You pay disk and memory for 35B and compute for 3B.

Under the hood it's the Qwen 3.5 MoE architecture: a hybrid of GatedDeltaNet linear-attention layers and full-attention layers, 256 routed experts plus one shared, 256K native context (extendable toward 1M with YaRN), a vision tower, and a thinking block that opens every answer by default. MIT licensed, no regional restrictions.

What makes it different from the Qwen it was built on is the training. Ornith-1.0 added continued pretraining and RL on top of Qwen 3.5 and Gemma 4. Ornith-1.5 widens the self-improvement loop: the model generates its own training tasks, builds its own scaffolds, and learns from the rollouts, rather than training on a fixed human-curated set. It's a coding-agent model first and everything else second.

Terminal-Bench 2.1 (run through Claude Code): 68.5. SWE-bench Verified: 79.0. SWE-bench Pro: 59.6. Qwen3.6-35B-A3B, the closest architectural peer, scores 49.2, 73.4, and 49.5 on the same rows.

Those are the model card's numbers, averaged over five runs, with git history stripped and network disabled during evaluation to block reward hacking. Also from the card: the family pulled 3.5 million GGUF downloads last month, which is not nothing for a model most people hadn't heard of in July.

Hardware requirements, by quant

This is the question everyone actually has, so here are the file sizes straight from the GGUF repo, plus what you need around them.

Two things the table hides. First, context eats memory fast: the card's reference vLLM setup uses two 80GB GPUs specifically to leave headroom for 256K. On a 24GB card at Q4 you'll run 16K to 32K comfortably and should not expect 256K. Second, because only 3B parameters activate per token, decode speed looks like a 3B model, not a 35B one. Community numbers on an Apple Silicon machine put the comparable Qwen3.6-35B MoE at roughly 12 tokens per second at 4-bit, versus about 8 for a dense 26B, which is the MoE dividend.

Minimum I'd actually recommend: 24GB VRAM or a 32GB Apple Silicon machine for Q4_K_M. Below that, this is the wrong model and the 9B is the right one; the Gemma 4 12B vs Qwen 3.5 9B post covers that tier.

How to run it on Ollama

The GGUF repo publishes an Ollama pull directly:

ollama run hf.co/ornith-ai/Ornith-1.5-35B-A3B-GGUF:Q4_K_M

Drop the tag to get the repo default, or swap Q4_K_M for any quant in the table. Ollama's qwen3.5 architecture support handles the hybrid MoE layout, so this works on a current Ollama without a custom Modelfile for chat.

Tool calling needs one check before you trust it. Ornith emits <tool_call> blocks that a server-side parser turns into OpenAI-style tool_calls; on vLLM that's --tool-call-parser qwen3_xml, and the card notes it adjusted the Qwen chat template for its own benchmarks. Ollama uses the template embedded in the GGUF, which is usually fine for this architecture, but verify it with a single tools request before wiring it into an agent. If the model returns raw <tool_call> text inside content instead of a structured tool_calls field, you have a template problem, not a model problem. Our Ollama tool-calling support reference has the test call and the Modelfile fix.

Sampling that the card recommends for general use: temperature 0.6, top_p 0.95, top_k 20. It's a reasoning model, so expect a thinking block first; for short agent turns where you don't want it, /nothink in the system prompt or think: false in the API cuts it, at some cost to multi-step quality.

Ornith vs Gemma 4 12B vs Qwen 3.5 9B

The searches landing on this site pair Ornith against Gemma 4 12B and Qwen 3.5 9B. That's the 9B Ornith's weight class, not the 35B's, so here's the honest version of the table: the published numbers side by side, with a note on where they came from.

Sources matter here. The Ornith-1.5-35B row is from its own card. The Ornith-1.0-9B, Gemma 4 12B, and Qwen 3.5 9B rows are from the Ornith-1.0-9B card, where the Ornith team ran all three through the same evaluation setup. That's a vendor-run comparison, and Google and Alibaba didn't publish those two benchmarks for those two models, so treat the gap as directional. It's a large gap even with a generous discount.

The practical read: if you have 8GB, Ornith 9B replaces Qwen 3.5 9B for coding and matches it on VRAM. If you have 24GB, the 35B is a different tier, and the only real local competitor is Qwen3.6-35B-A3B, which it beats on every published row. Gemma 4 12B's edge is native audio and a slightly more polished multimodal decoder, which isn't what a coding agent needs.

Setting it up in OpenClaw and Hermes

Both frameworks talk to any OpenAI-compatible endpoint, and the model card ships the exact configs. Serve the model first, either with Ollama (port 11434, /v1 path) or llama.cpp:

llama-server -hf ornith-ai/Ornith-1.5-35B-A3B-GGUF:Q4_K_M --port 8080 -c 32768

OpenClaw, one command, from the card

openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "ornith-ai/Ornith-1.5-35B-A3B-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

For Ollama instead of llama.cpp, point --custom-base-url at http://127.0.0.1:11434/v1 and use the Ollama model name. If the agent starts but tool calls come back as text, see the template note above; the local model not working decision tree walks the rest.

Hermes, three config lines

hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default ornith-ai/Ornith-1.5-35B-A3B-GGUF:Q4_K_M

Or the environment-variable form the card also lists: OPENAI_BASE_URL, OPENAI_API_KEY=EMPTY, and MODEL.

One setting that matters on both: context. A 35B-A3B at Q4 on a 24GB card does not have room for the 256K the model supports. Set the server's context to what fits (32K is a sane start) and let the framework's context management handle compaction, rather than letting the server silently truncate.

Where it fits in a routing stack

Here's what nobody tells you about local coding models: the win isn't replacing your cloud model. It's making the cloud model optional for the tasks that don't need it.

Ornith-1.5-35B is the model I'd route to for local coding work: refactors, test writing, code review, terminal tasks inside a repo, anything where the code shouldn't leave the machine or the volume is high enough that $10 per million output tokens hurts. On those tasks it's within striking distance of frontier models on the benchmarks and costs electricity.

It is not the model for web research, long-context reasoning over a quarter of Slack, or anything that needs a 1M window on consumer hardware. Route those to a cloud model. And for the fifty-a-day triage tasks (classification, drafting, summaries), a cheap cloud model like DeepSeek V4 Flash at $0.18 per million output is faster and doesn't tie up your GPU.

The routing rule ends up simple: code in a repo goes local, everything else goes to the cheapest cloud model that clears the bar, and a frontier model is reserved for what measurably needs it. That's the same logic as the OpenClaw model routing guide, with one more slot on the local side.

If you'd rather set that up as per-agent model selection than as config files, that's how BetterClaw works: bring your own keys for the cloud models, point an agent at a local endpoint for the ones that should stay home, and track what each agent costs. Free plan, no card, one agent with 100 credits a month.

The honest verdict

Ornith-1.5-35B-A3B is the best local coding-agent model I've run on a single 24GB card, and it isn't close. The MoE dividend is real: 35B quality at 3B speed. The reasoning block is worth keeping on for multi-step work and worth turning off for one-shot tool calls.

What it isn't is a general assistant. Its lineage is Qwen, its training is coding scaffolds, and its benchmark lead is on exactly those. Use it for what it was built for, route around it for the rest, and the "MIND BLOWING" thread starts to look accurate rather than excitable.

If any of this resonated, give BetterClaw a try. Free plan with 1 agent and 100 credits a month, bring your own API keys, no inference markup. Pro is $49 a month for 5 agents. Local endpoints and 28-plus cloud providers on the same agent, with cost tracking on each. Start free or see full pricing.

Frequently Asked Questions

What is Ornith-1.5-35B-A3B?

It's an open-weight, MIT-licensed mixture-of-experts model for coding agents from the Ornith team (DeepReinforce), built on the Qwen 3.5 MoE architecture with about 35B total parameters and roughly 3B active per token. It has 256K native context, vision input, a default thinking block, and tool calling, and posts 79.0 on SWE-bench Verified and 68.5 on Terminal-Bench 2.1 when run through Claude Code per its model card.

How does Ornith-1.5-35B-A3B compare to Qwen3.6-35B-A3B and Gemma 4?

On the model card's published rows it beats Qwen3.6-35B-A3B on every coding and agentic benchmark (79.0 vs 73.4 SWE-bench Verified, 67.8 vs 52.5 Terminal-Bench Terminus-2) at the same size and speed. Against Gemma 4 31B the gap is wider still on agentic coding. The Gemma 4 12B and Qwen 3.5 9B comparisons belong to Ornith's 9B, which the Ornith-1.0-9B card shows leading both.

How do I run Ornith-1.5-35B-A3B on Ollama with tool calling?

Pull it with ollama run hf.co/ornith-ai/Ornith-1.5-35B-A3B-GGUF:Q4_K_M, then send one test request with a tools array before wiring it into an agent. If the response has a structured tool_calls field you're set; if the call appears as raw <tool_call> text in content, the embedded chat template needs a Modelfile override. Use temperature 0.6, top_p 0.95, top_k 20 for general work.

Is Ornith-1.5-35B-A3B worth running locally instead of a cloud API?

For high-volume coding inside a repo, or code that can't leave the machine, yes: at Q4 it runs on a single 24GB card or 32GB Mac, decodes at 3B-model speeds, and costs electricity rather than $10 per million output tokens. For web research, very long context on consumer hardware, or cheap triage tasks, a cloud model is faster and often cheaper once you count the GPU's time.

Is Ornith-1.5-35B-A3B reliable enough for unsupervised agent work?

As reliable as any open-weight model this size, which means: good on structured tool calls, strong on terminal tasks, and still a model that will confidently do the wrong thing if the agent framework lets it. Run it behind the same controls as a cloud model, action approvals on anything that writes to production, a sandboxed working directory, and a cost or time cap per task. The model card's evaluations disabled network access for a reason.