Mistral Large 4 Hardware Requirements: Can You Run a 1T Model Locally?

Memory at each quantisation, the machines that fit, what each build costs today, and when the API is the smarter buy.

Picture this. It's 2 AM, a 378 GB download just finished, and your 256 GB workstation starts swapping the moment the model loads.

Should have checked the numbers first.

That's the trap with Mistral Large 4. Mistral released the open weights, nicknamed "Le Chonk", on 29 October 2026 and pitched them at self-hosters. At any precision you'd want to use, it's bigger than any single GPU you can buy.

So we did the homework on Mistral Large 4 hardware requirements: real file sizes, prices checked on 3 November 2026, and only published speed numbers. New to local agents? Start with our guide to running a local LLM agent on consumer hardware.

What Mistral Large 4 actually is (and why its size hurts)

Mistral Large 4 is Mistral's biggest model to date: a mixture-of-experts (MoE) model with 1.05T total parameters (the card says "1 trillion"), 128 routed experts and a 1M-token context. Per token, only 4 experts plus 1 shared expert fire: 49B active parameters, or 52B with embeddings and output layers.

MoE is what makes this expensive: every expert has to sit in memory, waiting for its turn.

The Mistral Ecosystem Open-Weight License is a modified MIT licence: use, fine-tune and sell freely, but a Model-as-a-Service business needs a separate commercial licence. Large 3 was Apache 2.0, so read the terms first.

Mistral aims it at coding and cyber defence, and self-reports 41% on Terminal-Bench 4.0 and 51% on the Artificial Analysis Cyber Index.

Here's the weird part: Mistral never released BF16 weights. The BF16 GGUFs are, in ggml-org's words, a "lossless upcast of the FP8 release." Twice the size, zero extra quality.

Mistral Large 4 hardware requirements: the memory you need at each quant

Start with one rule. Each parameter takes its bit width in memory, and 8 bits make a byte. So the weights need roughly parameters × bits ÷ 8 bytes.

At 4 bits, that's 1.05T × 4 ÷ 8 = 525 GB. Simple, right?

Not quite. Real files run larger because quants store scales next to the weights and keep sensitive layers at higher precision. The KV cache, which holds your context, comes on top.

The rule column is our estimate of weights only, from Mistral's 1.05T figure. Downloads are Hugging Face listings checked on 3 November 2026. Image input adds a 3.4 GB vision projector.

Look at the 1-bit row. The rule says 131 GB, but the smallest real file is 233 GB. That gap is deliberate: ggml-org's IQ2_XXS, for example, keeps attention, shared experts and embeddings at Q8_0.

The rule gives you the floor. The download gives you the bill.

So which quant? Unsloth measured how often each one picks the same top token as the full model: 90.6% for UD-IQ1_S, 95.6% for UD-Q2_K_XL, 97.8% for UD-Q4_K_XL and 98.5% for UD-Q5_K_XL.

Our take: 2-bit is the lowest we'd point an agent at for real work. 4-bit is where the loss gets small.

And Ollama? It only lists mistral-large-4:cloud, which runs on Ollama's servers, so there's nothing local to pull. Locally, use llama.cpp with the GGUFs. This is ggml-org's command (untested by us):

llama serve -hf ggml-org/Mistral-Large-4-1T-A52B-GGUF:IQ2_XXS

Which machines meet the Mistral Large 4 hardware requirements (and which can't)

This is where most people get it wrong: they count GPU VRAM and stop. A 1T model needs one huge memory pool, or several GPUs splitting it.

Mac Studio M5 Ultra with 512 GB

The cheapest single box that holds a usable copy: 512 GB of unified memory at 1.2 TB/s, only on the 36-core CPU, 80-core GPU chip. ggml-org says its Q2_K (380.7 GB) and IQ2_XXS (297.7 GB) files fit in 512 GB.

3-bit sits at the edge of Unsloth's 435 to 500 GB guidance, and 4-bit doesn't fit at all. A 256 GB Mac Studio takes only the 1-bit file.

DGX Spark clusters

One Spark has 128 GB at 273 GB/s, so one can't run it. NVIDIA supports linking up to four: Unsloth says its 1-bit file should fit on two, and four (512 GB) reach 2-bit on capacity. We found no published results on a Spark cluster, so treat it as possible, not proven.

For what a Spark is good at, see our DGX Spark alternatives guide.

NVIDIA DGX Station

748 GB of coherent memory, which NVIDIA says supports models up to 1 trillion parameters. The NVFP4 checkpoint (610.0 GB) and the 4-bit GGUFs fit; FP8 doesn't.

The catch: only 252 GB of that is HBM3e at 7.1 TB/s. The other 496 GB is LPDDR5X at 396 GB/s, so most weights sit in the slower pool.

Multi-GPU servers

This is how Mistral and vLLM expect you to serve it. vLLM's recipe validates one 4-GPU GB300 tray (NVFP4), 8× B300 (NVFP4 and FP8), 8× B200 (NVFP4) and two 8× H200 nodes (FP8), plus GB300 in FP8 and AMD MI350X in MXFP4. On 8 GPUs, FP8 needs about 128 GiB of weights per GPU and NVFP4 about 76 GiB, which is why the H200 recipe for FP8 uses two nodes.

What can't run it at all: any single GPU (an RTX 5090 has 32 GB, an RTX PRO 6000 has 96 GB), any 128 GB mini PC, a single DGX Spark, and any Mac Studio below 256 GB.

What each build costs right now (and why waiting won't help)

The Mac price is Apple's own: $6,799 for the 36-core CPU, 80-core GPU Mac Studio plus $15,500 for 512 GB, orderable since 28 October (TweakTown reports 512 GB units shipping from 3 December). The Spark figure is NVIDIA's October list price.

For the DGX Station, Exxact starts at $104,830 today. Our DGX Station price breakdown tracks every OEM listing, from about $92,900 to $159,283.

And prices are moving the wrong way. Hardware Unboxed's tracking, reported on 29 October, found GPU prices up 15% in August, 7% in September and 11% in October: well over 30% in three months.

SK hynix, in its third-quarter results on 28 October, said it expects memory demand to exceed supply through 2030.

So should you wait for prices to drop? On that evidence, we wouldn't bet on it.

Speed: the numbers that exist, and the ones that don't

The only published speeds for Large 4 come from data-centre hardware:

• 8× GB300, NVFP4, DSpark on (vLLM recipe): 3.13 ms per output token for one request, roughly 320 tokens per second, and about 55 per user under load at 8K input.
• DSpark speculative decoding: up to 1.9× throughput (Mistral), and about 2× decode speed at batch size 1 in llama.cpp (Unsloth).

We found no published speeds for a Mac Studio, DGX Station or Spark cluster, and haven't tested one, so we won't guess.

What drives it: every token reads the 52B active parameters from memory, so speed tracks bandwidth. That's 1.2 TB/s on the Mac, 396 GB/s for most of the DGX Station's memory and 273 GB/s per Spark.

Our post on DGX Spark memory bandwidth for AI agents explains why bandwidth, not FLOPs, sets the pace. And Large 4 reasons at "high" effort by default, so agent tasks generate a lot of tokens. A slow box hurts twice.

When the API beats the box

Mistral charges $0.70 per million input tokens, $0.07 per million cached input and $2.10 per million output, per its model docs. Batch processing halves that.

Now run the numbers. To pay back a $22,299 Mac Studio over three years, you'd need to be spending about $620 a month on the API, before electricity.

Say your agent reads 10 million tokens and writes 1 million every day. That costs $9.10 a day, about $273 a month at list price with no caching.

At that rate the Mac takes nearly seven years to pay for itself. And it runs a 2-bit quant while the API serves the full model. A $104,830 DGX Station works out to about $2,912 a month over three years.

For most teams, owning Large 4 is an expensive way to run a smaller copy of it.

Owning still wins when data legally can't leave your building, when the hardware already runs around the clock, or when you fine-tune locally. For everyone else, the API is cheaper.

That's the bet we'd make too. Skip the $20k build. Run Mistral and 28+ other providers on your own keys with zero markup. Start your agent in 60 seconds

Smaller Mistral models that fit the machine you already own

Want Mistral weights at home without a five-figure bill? Our roundup of what you can run locally in 2026 sorts every vendor's models by the memory you have. Within Mistral, look one size down.

• Mistral Small 4: 119B total, 6.5B active, 256K context, Apache 2.0. About 60 GB at 4-bit by the rule (our estimate), in reach of a 96 GB or 128 GB machine.
• Mistral Medium 3.5: dense 128B, 256K context, modified MIT. About 64 GB at 4-bit (our estimate), but it reads all 128B parameters per token, so it runs slower.

For most home setups, our pick is Small 4.

So, can you run a 1T model locally?

Yes, you can. A 1T model on a desk for $22,299 is real now.

But "can" and "should" are different questions. At that price you get a 2-bit copy, an unknown speed, and years of heavy use before you beat the API.

Our honest read: buy the box if your data can't leave the building, or if you love the craft. Otherwise, rent the brain and spend the $22,299 on whatever your agent is meant to build.

The best hardware for a 1T model is often the hardware someone else already paid for.

Frequently Asked Questions

What is Mistral Large 4?

It's Mistral's largest open-weight model, a mixture-of-experts model with 1.05T total parameters and 52B active per token. Open weights arrived on 29 October 2026.

How much memory does Mistral Large 4 need?

About 256 GB of total memory for 1-bit, 310 to 390 GB for 2-bit, about 630 GB for 4-bit and 1.2 TB for 8-bit, per Unsloth. KV cache comes on top.

Mac Studio, DGX Station or DGX Spark: which one runs Mistral Large 4?

The 512 GB Mac Studio M5 Ultra ($22,299) is the cheapest single machine and runs 1-bit and 2-bit quants. The 748 GB DGX Station, from $104,830, holds NVFP4 and 4-bit. One 128 GB DGX Spark can't run it.

How do I run Mistral Large 4 locally?

Download a GGUF that fits your memory and serve it with llama.cpp; ggml-org's command is llama serve -hf ggml-org/Mistral-Large-4-1T-A52B-GGUF:IQ2_XXS. Ollama only offers mistral-large-4:cloud, which runs on Ollama's servers.

Is it cheaper to use the Mistral Large 4 API?

For most teams, yes. An agent reading 10 million and writing 1 million tokens a day costs about $273 a month at list price. A $22,299 Mac Studio needs about $620 a month of API spend to pay back in three years.

What quantisation should I use, and is 2-bit reliable enough for agents?

Use the highest quant your memory allows with room for the KV cache. Unsloth found UD-Q2_K_XL matches the full model's top token 95.6% of the time and UD-Q4_K_XL 97.8%, so test 2-bit on your own tasks first. When accuracy matters most, run the full model on the API.