[{"data":1,"prerenderedAt":1001},["ShallowReactive",2],{"blog-post-best-coding-model-12gb-vram":3,"related-posts-best-coding-model-12gb-vram":978},{"id":4,"title":5,"author":6,"body":11,"category":952,"date":953,"description":954,"extension":955,"featured":956,"hideToc":956,"image":957,"imageAlt":958,"imageHeight":959,"imageWidth":960,"itemList":961,"lang":961,"meta":962,"navigation":963,"noindex":956,"path":964,"readingTime":965,"redirected":956,"relatedSlugs":961,"seo":966,"seoTitle":5,"stem":967,"subtitle":961,"tags":968,"updatedDate":953,"__hash__":977},"blog/blog/best-coding-model-12gb-vram.md","Best Coding Model for 12GB VRAM: Ollama Picks That Fit",{"name":7,"role":8,"avatar":9,"bio":10},"Shabnam Katoch","Growth Head","/img/avatars/shabnam-profile.jpeg","Shabnam leads growth at BetterClaw and has spent the past three years building organic distribution for AI-native products. She writes about no-code agent builders, content strategy, and what it actually takes to grow without an ads budget.",{"type":12,"value":13,"toc":929},"minimark",[14,18,38,174,177,180,183,189,192,201,206,209,212,215,225,232,318,321,324,331,338,344,348,351,357,387,394,399,414,417,424,428,442,445,448,458,462,476,484,491,494,498,512,517,526,557,564,570,574,582,585,641,650,654,657,660,663,688,703,706,713,716,721,728,732,735,815,821,825,828,835,850,854,858,870,874,883,887,896,900,903,907,918,922,925],[15,16,17],"p",{},"A 12 GB card runs good coding models now. What it can't survive is Ollama's default 4K context, so here is what fits, with the memory worked out and the commands to set it.",[19,20,22],"callout",{"type":21},"quick-fix",[15,23,24,28,29,33,34,37],{},[25,26,27],"strong",{},"Best coding model for 12GB VRAM:"," Gemma 4 12B at Q4_K_M (",[30,31,32],"code",{},"gemma4:12b",", 8.0 GB) is the safe default. The one to test first is Bonsai 2 27B (",[30,35,36],{},"bonsai-2:27b",", 8.3 GB), a ternary Qwen 3.8 27B whose maker reports near full-precision coding scores. Qwen 3.5 9B suits long agent sessions; Qwen 2.5 Coder 14B is for autocomplete.",[39,40,41,66],"table",{},[42,43,44],"thead",{},[45,46,47,51,54,57,60,63],"tr",{},[48,49,50],"th",{},"Pick",[48,52,53],{},"Ollama tag",[48,55,56],{},"Download",[48,58,59],{},"Context we set",[48,61,62],{},"Memory at that context (calculated)",[48,64,65],{},"Best for",[67,68,69,91,112,134,155],"tbody",{},[45,70,71,75,79,82,85,88],{},[72,73,74],"td",{},"Gemma 4 12B",[72,76,77],{},[30,78,32],{},[72,80,81],{},"8.02 GB",[72,83,84],{},"65536",[72,86,87],{},"about 9.4 GB",[72,89,90],{},"Safe all-round coding pick",[45,92,93,96,100,103,106,109],{},[72,94,95],{},"Bonsai 2 27B",[72,97,98],{},[30,99,36],{},[72,101,102],{},"8.26 GB",[72,104,105],{},"32768, or 65536 with a q8_0 cache",[72,107,108],{},"about 10.6 GB",[72,110,111],{},"Strongest code scores that fit",[45,113,114,117,122,125,128,131],{},[72,115,116],{},"Qwen 3.5 9B",[72,118,119],{},[30,120,121],{},"qwen3.5:9b",[72,123,124],{},"6.55 GB",[72,126,127],{},"65536, up to 131072",[72,129,130],{},"about 8.8 GB",[72,132,133],{},"Long tool-calling sessions",[45,135,136,139,144,147,150,152],{},[72,137,138],{},"Qwen 2.5 Coder 14B",[72,140,141],{},[30,142,143],{},"qwen2.5-coder:14b",[72,145,146],{},"8.99 GB",[72,148,149],{},"8192",[72,151,108],{},[72,153,154],{},"Editor autocomplete",[45,156,157,160,163,166,168,171],{},[72,158,159],{},"Qwen 3.6 35B-A3B",[72,161,162],{},"llama.cpp, experts in RAM",[72,164,165],{},"22.4 GB file",[72,167,84],{},[72,169,170],{},"12 GB card plus 32 GB of RAM",[72,172,173],{},"The step up",[15,175,176],{},"We found no published tokens-per-second figure for the first four under Ollama on a named 12 GB card, so the table shows calculated memory. The last row has one, below.",[15,178,179],{},"Picture this: you pull a 12B coding model, point OpenCode at it, and ask it to fix a failing test that touches three files.",[15,181,182],{},"It reads the first file, loses track of the second, and edits the wrong function.",[15,184,185],{},[186,187,188],"em",{},"Is the model really this bad?",[15,190,191],{},"Usually not. On any card under 24 GiB, Ollama's default context is 4K tokens, and your agent ran out of room halfway through the job.",[15,193,194,195,200],{},"We checked every tag and file size on Ollama and Hugging Face on 10 November 2026, and worked out the memory from each model's config. Whether a local agent is worth it at all is a separate question, and our guide to ",[196,197,199],"a",{"href":198},"/blog/local-llm-agent-consumer-hardware-2026","running a local LLM agent on consumer hardware"," is blunt about it.",[202,203,205],"h2",{"id":204},"why-12-gb-is-the-awkward-tier-for-local-coding","Why 12 GB is the awkward tier for local coding",[15,207,208],{},"The models that lead local coding are 27B to 31B. At Q4_K_M, Qwen 3.8 27B is a 17.7 GB download and Gemma 4 31B is 20.4 GB, so neither loads on a 12 GB card.",[15,210,211],{},"What fits is the 9B to 14B class, plus one 2-bit 27B. And here's the thing: at that size the weights aren't what decides the winner. The context cache is.",[15,213,214],{},"Every token in the context stores a key and a value in every attention layer that keeps a full cache:",[216,217,223],"pre",{"className":218,"code":220,"language":221,"meta":222},[219],"language-text","KV cache (bytes) = 2 x attention layers x KV heads x head size x bytes per value x tokens\n","text","",[30,224,220],{"__ignoreMap":222},[15,226,227,228,231],{},"The 2 is one key plus one value, and bytes per value is 2 at Ollama's default ",[30,229,230],{},"f16"," cache. Here's the working from each model's config file on Hugging Face:",[39,233,234,253],{},[42,235,236],{},[45,237,238,241,244,247,250],{},[48,239,240],{},"Model",[48,242,243],{},"Layers that keep a full cache",[48,245,246],{},"KV heads x head size",[48,248,249],{},"Per token",[48,251,252],{},"64K tokens",[67,254,255,271,287,302],{},[45,256,257,259,262,265,268],{},[72,258,138],{},[72,260,261],{},"48 of 48",[72,263,264],{},"8 x 128",[72,266,267],{},"2 x 48 x 8 x 128 x 2 = 192 KiB",[72,269,270],{},"12.9 GB",[45,272,273,275,278,281,284],{},[72,274,95],{},[72,276,277],{},"16 of 64 (the rest linear, about 0.15 GB fixed)",[72,279,280],{},"4 x 256",[72,282,283],{},"2 x 16 x 4 x 256 x 2 = 64 KiB",[72,285,286],{},"4.3 GB",[45,288,289,291,294,296,299],{},[72,290,116],{},[72,292,293],{},"8 of 32 (the rest linear, about 0.05 GB fixed)",[72,295,280],{},[72,297,298],{},"2 x 8 x 4 x 256 x 2 = 32 KiB",[72,300,301],{},"2.1 GB",[45,303,304,306,309,312,315],{},[72,305,74],{},[72,307,308],{},"8 of 48 (40 sliding layers keep 1,024 tokens, about 0.34 GB fixed)",[72,310,311],{},"1 x 512",[72,313,314],{},"2 x 8 x 1 x 512 x 2 = 16 KiB",[72,316,317],{},"1.1 GB",[15,319,320],{},"Same card, a twelve-fold difference in what context costs.",[15,322,323],{},"For the budget: a 12 GB card has 12 GiB, about 12.9 GB. We plan on 11 GB for weights plus cache and leave about 1.9 GB for the driver, compute buffers and your desktop.",[15,325,326,327,330],{},"That margin is our planning choice, not a spec, so confirm with ",[30,328,329],{},"ollama ps",": the PROCESSOR column should say 100% GPU. If you're still choosing a card, price out what four more gigabytes buys before you settle on 12.",[15,332,333],{},[334,335],"img",{"alt":336,"src":337},"VRAM budget on a 12 GB card: model weights plus context cache plus headroom, with the KV cache formula and the per-token cost of each model","/img/blog/best-coding-model-12gb-vram-vram-budget.jpg",[339,340,341],"blockquote",{},[15,342,343],{},"On a 12 GB card, the context cache, not the parameter count, decides which coding model wins.",[202,345,347],{"id":346},"picking-the-best-coding-model-for-12gb-vram-model-by-model","Picking the best coding model for 12GB VRAM, model by model",[15,349,350],{},"Ollama's context docs say the default is 4K on cards under 24 GiB, and that agents and coding tools need at least 64000 tokens. Set it per model with a two-line Modelfile:",[216,352,355],{"className":353,"code":354,"language":221,"meta":222},[219],"FROM gemma4:12b\nPARAMETER num_ctx 65536\n",[30,356,354],{"__ignoreMap":222},[216,358,362],{"className":359,"code":360,"language":361,"meta":222,"style":222},"language-bash shiki shiki-themes github-light","ollama create gemma4-12b-64k -f Modelfile\n","bash",[30,363,364],{"__ignoreMap":222},[365,366,369,373,377,380,384],"span",{"class":367,"line":368},"line",1,[365,370,372],{"class":371},"s7eDp","ollama",[365,374,376],{"class":375},"sYBdl"," create",[365,378,379],{"class":375}," gemma4-12b-64k",[365,381,383],{"class":382},"sYu0t"," -f",[365,385,386],{"class":375}," Modelfile\n",[15,388,389,390,393],{},"Or set it for every model the server loads with ",[30,391,392],{},"OLLAMA_CONTEXT_LENGTH=65536 ollama serve",".",[395,396,398],"h3",{"id":397},"gemma-4-12b-the-safe-default","Gemma 4 12B: the safe default",[216,400,402],{"className":359,"code":401,"language":361,"meta":222,"style":222},"ollama pull gemma4:12b\n",[30,403,404],{"__ignoreMap":222},[365,405,406,408,411],{"class":367,"line":368},[365,407,372],{"class":371},[365,409,410],{"class":375}," pull",[365,412,413],{"class":375}," gemma4:12b\n",[15,415,416],{},"The default tag is Q4_K_M, 8.02 GB including a 0.47 GB draft layer for faster generation. Google's card lists 72.0% on LiveCodeBench v6, 69.0% on Tau2, native function calling and a 256K window, and Ollama tags it for tools, thinking, vision and audio.",[15,418,419,420,423],{},"At 65536 it needs about 9.4 GB by our working, and 131072 still fits at about 10.5 GB. ",[30,421,422],{},"gemma4:12b-it-qat"," (7.15 GB) gives more headroom; the Q8 build (12.84 GB) doesn't fit.",[395,425,427],{"id":426},"bonsai-2-27b-the-27b-that-fits","Bonsai 2 27B: the 27B that fits",[216,429,431],{"className":359,"code":430,"language":361,"meta":222,"style":222},"ollama pull bonsai-2:27b\n",[30,432,433],{"__ignoreMap":222},[365,434,435,437,439],{"class":367,"line":368},[365,436,372],{"class":371},[365,438,410],{"class":375},[365,440,441],{"class":375}," bonsai-2:27b\n",[15,443,444],{},"Prism ML's Bonsai 2 27B is Qwen 3.8 27B rebuilt with ternary weights (each one is minus one, zero or plus one), released three weeks ago under Apache 2.0. Ollama's tag is 8.26 GB, including a 0.63 GB vision file.",[15,446,447],{},"Prism reports an average of 84.78 across 14 thinking-mode benchmarks against 86.32 for full-precision Qwen 3.8 27B, with LiveCodeBench at 90.07 and BFCL v3 tool calling at 74.92. Those are the maker's own numbers, and the model thinks at its highest effort by default, which eats context.",[15,449,450,451,453,454,457],{},"At 64 KiB per token, plan on 32768 at ",[30,452,230],{},", or 65536 with the ",[30,455,456],{},"q8_0"," cache shown below (about 10.6 GB either way). If you test one model this week, test this one against Gemma on your own repo.",[395,459,461],{"id":460},"qwen-35-9b-or-its-mimo-fine-tune-the-long-session-pick","Qwen 3.5 9B, or its MiMo fine-tune: the long-session pick",[216,463,465],{"className":359,"code":464,"language":361,"meta":222,"style":222},"ollama pull qwen3.5:9b\n",[30,466,467],{"__ignoreMap":222},[365,468,469,471,473],{"class":367,"line":368},[365,470,372],{"class":371},[365,472,410],{"class":375},[365,474,475],{"class":375}," qwen3.5:9b\n",[15,477,478,479,483],{},"Our ",[196,480,482],{"href":481},"/blog/gemma-4-vs-qwen-3-8","Gemma 4 vs Qwen 3.8 comparison"," covers both families in depth. This one is 6.55 GB, 0.92 GB of it the vision projector. Qwen reports 65.6 on LiveCodeBench v6 and 79.1 on TAU2-Bench, and its card advises at least 128K of context to preserve thinking: about 10.9 GB by our working, tight but inside the plan.",[15,485,486,487,490],{},"Xiaomi's MiMo V2.6 9B (",[30,488,489],{},"mimo-v2.6:9b",", same size, MIT licence) is a fine-tune of it for code and agents. Xiaomi reports 61.1 on SWE-bench Verified against 60.0, and 37.1 on Terminal Bench 2.1 against 27.0, and calls it a research checkpoint.",[15,492,493],{},"Every score on this page is self-reported on different setups, so read them as direction.",[395,495,497],{"id":496},"qwen-25-coder-14b-autocomplete-not-agents","Qwen 2.5 Coder 14B: autocomplete, not agents",[216,499,501],{"className":359,"code":500,"language":361,"meta":222,"style":222},"ollama pull qwen2.5-coder:14b\n",[30,502,503],{"__ignoreMap":222},[365,504,505,507,509],{"class":367,"line":368},[365,506,372],{"class":371},[365,508,410],{"class":375},[365,510,511],{"class":375}," qwen2.5-coder:14b\n",[15,513,514,515,393],{},"It's 8.99 GB, Ollama lists a 32K window, and the model dates from November 2024. At 192 KiB per token, about 10K tokens fit at ",[30,516,230],{},[15,518,519,520,522,523,525],{},"Ollama's FAQ says a ",[30,521,456],{}," cache uses about half the memory of ",[30,524,230],{}," with very little quality loss. It needs Flash Attention and applies to every model the server runs:",[216,527,529],{"className":359,"code":528,"language":361,"meta":222,"style":222},"OLLAMA_FLASH_ATTENTION=1 OLLAMA_KV_CACHE_TYPE=q8_0 ollama serve\n",[30,530,531],{"__ignoreMap":222},[365,532,533,537,541,544,547,549,551,554],{"class":367,"line":368},[365,534,536],{"class":535},"sgsFI","OLLAMA_FLASH_ATTENTION",[365,538,540],{"class":539},"sD7c4","=",[365,542,543],{"class":375},"1",[365,545,546],{"class":535}," OLLAMA_KV_CACHE_TYPE",[365,548,540],{"class":539},[365,550,456],{"class":375},[365,552,553],{"class":371}," ollama",[365,555,556],{"class":375}," serve\n",[15,558,559,560,563],{},"That gets this model to about 20K, still short of 64K. Its real job is fill-in-the-middle: its Ollama template handles a prompt plus a suffix, which is what editor autocomplete sends. Give it ",[30,561,562],{},"num_ctx 8192",", and skip Q5_K_M (10.51 GB).",[15,565,566],{},[334,567],{"alt":568,"src":569},"Model-fit table for 12GB VRAM: Ollama tag, download size, cache per token, context that fits and best use for each coding model","/img/blog/best-coding-model-12gb-vram-model-fit.jpg",[202,571,573],{"id":572},"is-the-best-coding-model-for-12gb-vram-good-enough-for-coding-agents","Is the best coding model for 12GB VRAM good enough for coding agents?",[15,575,576,577,581],{},"For one file or one bug, yes. Every model here carries Ollama's tools tag; whether one calls your tools well is a separate question, and our list of ",[196,578,580],{"href":579},"/blog/ollama-models-tool-calling-support","which Ollama models support tool calling"," covers how Ollama decides.",[15,583,584],{},"The bigger filter is context. Ollama's one-line launchers for coding agents each ask for a big window:",[39,586,587,600],{},[42,588,589],{},[45,590,591,594,597],{},[48,592,593],{},"Agent",[48,595,596],{},"Launch command",[48,598,599],{},"Context Ollama's docs ask for",[67,601,602,615,628],{},[45,603,604,607,612],{},[72,605,606],{},"Claude Code",[72,608,609],{},[30,610,611],{},"ollama launch claude",[72,613,614],{},"64K or more for larger repos",[45,616,617,620,625],{},[72,618,619],{},"OpenCode",[72,621,622],{},[30,623,624],{},"ollama launch opencode",[72,626,627],{},"64K or more, required",[45,629,630,633,638],{},[72,631,632],{},"Codex CLI",[72,634,635],{},[30,636,637],{},"ollama launch codex",[72,639,640],{},"at least 64K",[15,642,643,644,646,647,649],{},"Gemma 4 12B and the 9B models clear 64K easily, Bonsai 2 27B clears it with a ",[30,645,456],{}," cache, and Qwen 2.5 Coder 14B can't. When an agent starts dropping tool calls, check the CONTEXT column in ",[30,648,329],{}," before you blame the model.",[202,651,653],{"id":652},"using-system-ram-to-go-bigger","Using system RAM to go bigger",[15,655,656],{},"With 32 GB of system RAM, a mixture-of-experts model can use it: only a few experts run per token, so they can sit in RAM while attention and the cache stay on the card.",[15,658,659],{},"The best fit for 12 GB plus 32 GB is Qwen 3.6 35B-A3B: 35B parameters in total, 3B active per token, Apache 2.0. Qwen's card reports 73.4 on SWE-bench Verified and 51.5 on Terminal-Bench 2.0, and only 10 of its 40 layers keep a full cache, so 64K costs about 1.3 GB.",[15,661,662],{},"With llama.cpp you choose where the experts live:",[216,664,666],{"className":359,"code":665,"language":361,"meta":222,"style":222},"llama-server -hf unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q4_K_XL --cpu-moe -c 65536\n",[30,667,668],{"__ignoreMap":222},[365,669,670,673,676,679,682,685],{"class":367,"line":368},[365,671,672],{"class":371},"llama-server",[365,674,675],{"class":382}," -hf",[365,677,678],{"class":375}," unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q4_K_XL",[365,680,681],{"class":382}," --cpu-moe",[365,683,684],{"class":382}," -c",[365,686,687],{"class":382}," 65536\n",[15,689,690,693,694,697,698,702],{},[30,691,692],{},"--cpu-moe"," keeps every expert on the CPU. If VRAM is left over, swap it for ",[30,695,696],{},"--n-cpu-moe N",", which keeps only the experts of the first N layers on the CPU, and lower N until the card is nearly full. If you'd rather click than type flags, our ",[196,699,701],{"href":700},"/blog/ollama-vs-lm-studio","Ollama vs LM Studio comparison"," covers the trade-offs.",[15,704,705],{},"Speed is the part people ask about. The Strata project, an open-source local engine, publishes 59.7 tokens per second of decode for this model at UD-Q4_K_XL on an RTX 3060 12 GB with most experts on the CPU, and plans for 32 GB of RAM. That's Strata's own engine and measurement; we haven't reproduced it.",[15,707,708,709,712],{},"Recursive's Flint (",[30,710,711],{},"flint:40b",", 22.86 GB, Apache 2.0) is the newest model in the same shape: 39.8B total, about 2.5B active, built for coding agents, with Recursive reporting 74.2% on SWE-bench Verified. We found no published speed for it on a 12 GB card.",[15,714,715],{},"What about Qwen 3.8 Flash Next? Strata's docs say its Coder build runs on a 32 GB PC with a 12 to 16 GB card only by reading most experts from the SSD, which is much slower. With 64 GB of RAM, Strata's older RTX 5070 12 GB measurement shows 55 tokens per second on a short chat.",[339,717,718],{},[15,719,720],{},"With 32 GB of RAM, run Qwen 3.6 35B-A3B or Flint. With 64 GB, Flash Next's Coder build is worth a weekend.",[15,722,723,724],{},"Once one of these runs, the next question is how an agent that lives somewhere else reaches it. That's what BetterClaw Desktop does: it connects the models on your computer, through Ollama, LM Studio, llama.cpp or any OpenAI-compatible address, to your BetterClaw agents in the cloud, with no open ports and no tunnels. It shows each model's context size and how well it handles agent tools before you assign it, on Apple Silicon Macs, 64-bit Windows 10 and 11, and Linux. ",[196,725,727],{"href":726},"/downloads","Get BetterClaw Desktop",[202,729,731],{"id":730},"what-does-not-fit-on-12-gb","What does not fit on 12 GB",[15,733,734],{},"None of these loads fully on a 12 GB card at Q4:",[39,736,737,750],{},[42,738,739],{},[45,740,741,743,745,747],{},[48,742,240],{},[48,744,53],{},[48,746,56],{},[48,748,749],{},"What happens on 12 GB",[67,751,752,768,784,800],{},[45,753,754,757,762,765],{},[72,755,756],{},"Gemma 4 26B A4B (MoE)",[72,758,759],{},[30,760,761],{},"gemma4:26b-a4b-it-qat",[72,763,764],{},"15.63 GB",[72,766,767],{},"Runs only with part of it in system RAM",[45,769,770,773,778,781],{},[72,771,772],{},"Qwen 3.8 27B",[72,774,775],{},[30,776,777],{},"qwen3.8:27b",[72,779,780],{},"17.74 GB",[72,782,783],{},"Doesn't fit; Bonsai 2 27B is the way in",[45,785,786,789,794,797],{},[72,787,788],{},"Qwen 3.6 27B coding",[72,790,791],{},[30,792,793],{},"qwen3.6:27b-coding",[72,795,796],{},"17.77 GB",[72,798,799],{},"Doesn't fit",[45,801,802,805,810,813],{},[72,803,804],{},"Gemma 4 31B",[72,806,807],{},[30,808,809],{},"gemma4:31b",[72,811,812],{},"20.39 GB",[72,814,799],{},[15,816,817],{},[334,818],{"alt":819,"src":820},"Calculated VRAM-fit chart: total memory for each coding model at 8K, 32K, 64K and 128K context against an 11 GB plan on a 12 GB card","/img/blog/best-coding-model-12gb-vram-vram-fit-chart.jpg",[202,822,824],{"id":823},"twelve-gigabytes-is-enough-if-you-spend-it-on-context","Twelve gigabytes is enough, if you spend it on context",[15,826,827],{},"The old advice for a 12 GB card was to load the biggest coder that fits. That advice now quietly breaks agents, because a model with no room for context forgets the file it just read.",[15,829,830,831,834],{},"Pick the model that leaves space to think, and set ",[30,832,833],{},"num_ctx"," yourself. The card you own is probably fine. The default setting isn't.",[19,836,838,843],{"type":837},"cta",[15,839,840],{},[25,841,842],{},"Not sure local models fit your team's work?",[15,844,845,846],{},"If you're weighing a new GPU against cloud models for a team, our AI automation audit starts with a 30-minute call about how your team works today. Within 5 business days you get a written report: a workflow map, a proposed agent setup, ROI estimates and an implementation timeline. ",[196,847,849],{"href":848},"/ai-automation-audit","Book an AI automation audit",[202,851,853],{"id":852},"frequently-asked-questions","Frequently Asked Questions",[395,855,857],{"id":856},"what-is-the-best-coding-model-for-12gb-vram","What is the best coding model for 12GB VRAM?",[15,859,860,861,863,864,866,867,869],{},"Gemma 4 12B at Q4_K_M (",[30,862,32],{},", 8.02 GB) is the safe pick, and Bonsai 2 27B (",[30,865,36],{},", 8.26 GB) is the one to test, since its maker reports near full-precision Qwen 3.8 27B coding scores. Both fit a useful context on a 12 GB card by our calculation. Set ",[30,868,833],{}," yourself, because Ollama defaults to 4K on cards under 24 GiB.",[395,871,873],{"id":872},"can-i-run-a-14b-model-on-12gb","Can I run a 14B model on 12GB?",[15,875,876,877,879,880,882],{},"Yes, but with little context. Qwen 2.5 Coder 14B at Q4_K_M is 8.99 GB, and its cache costs 192 KiB per token, so about 10K tokens fit at the default ",[30,878,230],{}," cache, or about 20K with a ",[30,881,456],{}," cache. That works for autocomplete, not for coding agents that want 64K.",[395,884,886],{"id":885},"can-i-run-a-27b-model-on-12gb","Can I run a 27B model on 12GB?",[15,888,889,890,892,893,895],{},"Only a 2-bit one. Qwen 3.8 27B at Q4_K_M is 17.74 GB, but Bonsai 2 27B, a ternary build of the same model, is 8.26 GB on Ollama. By our working it fits a 32K context at ",[30,891,230],{},", or 64K with a ",[30,894,456],{}," cache.",[395,897,899],{"id":898},"does-gemma-4-12b-fit-on-12gb","Does Gemma 4 12B fit on 12GB?",[15,901,902],{},"Yes. The default Q4_K_M build is 8.02 GB, and by our working it needs about 9.4 GB at a 64K context and about 10.5 GB at 128K. The QAT build is 7.15 GB, while the Q8 build at 12.84 GB doesn't fit.",[395,904,906],{"id":905},"what-context-length-can-i-use","What context length can I use?",[15,908,909,910,913,914,917],{},"Ollama uses 4K by default on cards under 24 GiB, and its docs recommend at least 64000 for agents and coding tools. Set ",[30,911,912],{},"PARAMETER num_ctx 65536"," in a Modelfile or start the server with ",[30,915,916],{},"OLLAMA_CONTEXT_LENGTH=65536",". Gemma 4 12B and Qwen 3.5 9B reach about 128K on 12 GB on paper; Qwen 2.5 Coder 14B tops out near 10K to 20K.",[395,919,921],{"id":920},"is-an-rtx-3060-12gb-enough-for-local-coding-agents","Is an RTX 3060 12GB enough for local coding agents?",[15,923,924],{},"For single-file fixes and short tasks, yes: Gemma 4 12B and Qwen 3.5 9B both fit with a 64K context. With 32 GB of RAM it can also run Qwen 3.6 35B-A3B with experts on the CPU, which the Strata project measured at 59.7 tokens per second on a 3060.",[926,927,928],"style",{},"html pre.shiki code .s7eDp, html code.shiki .s7eDp{--shiki-default:#6F42C1}html pre.shiki code .sYBdl, html code.shiki .sYBdl{--shiki-default:#032F62}html pre.shiki code .sYu0t, html code.shiki .sYu0t{--shiki-default:#005CC5}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html pre.shiki code .sgsFI, html code.shiki .sgsFI{--shiki-default:#24292E}html pre.shiki code .sD7c4, html code.shiki .sD7c4{--shiki-default:#D73A49}",{"title":222,"searchDepth":930,"depth":930,"links":931},2,[932,933,940,941,942,943,944],{"id":204,"depth":930,"text":205},{"id":346,"depth":930,"text":347,"children":934},[935,937,938,939],{"id":397,"depth":936,"text":398},3,{"id":426,"depth":936,"text":427},{"id":460,"depth":936,"text":461},{"id":496,"depth":936,"text":497},{"id":572,"depth":930,"text":573},{"id":652,"depth":930,"text":653},{"id":730,"depth":930,"text":731},{"id":823,"depth":930,"text":824},{"id":852,"depth":930,"text":853,"children":945},[946,947,948,949,950,951],{"id":856,"depth":936,"text":857},{"id":872,"depth":936,"text":873},{"id":885,"depth":936,"text":886},{"id":898,"depth":936,"text":899},{"id":905,"depth":936,"text":906},{"id":920,"depth":936,"text":921},"Hardware","2026-11-12","Best coding model for 12GB VRAM in 2026: Gemma 4 12B, Bonsai 2 27B, Qwen 3.5 9B and Qwen 2.5 Coder 14B on fit, context and tools, with Ollama commands.","md",false,"/img/blog/best-coding-model-12gb-vram.jpg","Best coding model for 12GB VRAM cover: a GPU card outline with a 12 GB memory bar split into model weights, context cache and headroom",512,1024,null,{},true,"/blog/best-coding-model-12gb-vram","10 min read",{"title":5,"description":954},"blog/best-coding-model-12gb-vram",[969,970,971,972,973,974,975,976],"best coding model for 12gb vram","best local coding model 12gb","ollama models for 12gb vram","rtx 3060 12gb local llm","gemma 4 12b vram","bonsai 2 27b","qwen 3.5 9b ollama","ollama num_ctx","KoaI8TlVFkA10_PR07eYHqUXJWbobtvbL_SbR-4eNnw",[979,987,993],{"stem":980,"title":981,"description":982,"date":953,"image":983,"category":984,"redirected":956,"author":985,"readingTime":986},"blog/deepseek-v4-1-flash-vs-glm-5-3-flash","DeepSeek V4.1 Flash vs GLM 5.3 Flash: Price per Task, Speed and Which to Use for Agents","DeepSeek V4.1 Flash vs GLM 5.3 Flash compared on price per task, speed, tool calling and context, plus how each stacks up against Opus 5.5 and Kimi K3.","/img/blog/deepseek-v4-1-flash-vs-glm-5-3-flash.jpg","Comparison",{"name":7,"role":8,"avatar":9,"bio":10},"11 min read",{"stem":988,"title":989,"description":990,"date":953,"image":991,"category":984,"redirected":956,"author":992,"readingTime":986},"blog/opencode-go-vs-openrouter-vs-deepseek-api","OpenCode Go vs OpenRouter vs DeepSeek API: Which $10 Lasts Longest for Agents","OpenCode Go vs OpenRouter vs the DeepSeek API for $10: what each gives you, models, limits, data retention and which lasts longest for real agent work.","/img/blog/opencode-go-vs-openrouter-vs-deepseek-api.jpg",{"name":7,"role":8,"avatar":9,"bio":10},{"stem":994,"title":995,"description":996,"date":997,"image":998,"category":984,"redirected":956,"author":999,"readingTime":1000},"blog/qwen-4-vs-qwen-3-8","Qwen 4: Release Date, Expected Sizes and What Changes from Qwen 3.8","Qwen 4 is not out yet. What Qwen has confirmed, what the leaked model ids suggest, the expected sizes including 27B, and what to run until it ships.","2026-11-11","/img/blog/qwen-4-vs-qwen-3-8.jpg",{"name":7,"role":8,"avatar":9,"bio":10},"9 min read",1794489111864]