North Mini Code in NVFP4: ~1.65× Faster, 40% Smaller, Zero Quality Loss on a Single DGX Spark
At Xanu, we spend a lot of time on one question: how do you run genuinely useful models on hardware you actually own? Sovereign, run-it-yourself inference is only interesting if the numbers hold up — if a model is fast enough, small enough, and good enough to live on a box that sits under your desk instead of in someone else’s data center.
So when Cohere released **North Mini Code 1.0** (North Mini Code: Agentic Coding Model for Developers | Cohere) — their first open agentic coding model — we quantized it to **NVFP4** and put it head-to-head against the FP8 reference on a single NVIDIA DGX Spark (GB10). Same model, same recipe, same Spark. The only thing we changed was the quantization format.
The short version: **~1.65× faster single-user decode, ~40% smaller memory footprint, and no measurable quality loss.**
Why North Mini Code, and why the Spark
North Mini Code is a 30B mixture-of-experts model with only 3B active parameters, released under Apache 2.0. That combination matters. The MoE design keeps inference cheap, the Apache license keeps it genuinely yours, and Cohere built it for agentic coding — so it’s tuned to know *when* to call a tool, not just how to autocomplete. That tool-calling instinct turns out to be the difference between a model that’s fun to demo and one you can actually drop into a coding loop.
The DGX Spark / GB10 is a natural home for it. The GB10 has native FP4 support in its tensor cores, which is exactly the capability that’s been underused on this platform. A 30B/3B MoE is small enough to run comfortably on a single Spark with room to spare — and NVFP4 is the format that lets you cash in on the hardware.
What we ran
Everything ran on **a single Spark (tensor parallel 1) under vLLM**, with an FP8 KV cache, and tool calling plus reasoning wired up through the `cohere_command4` parsers. We published both runs on Spark Arena so they’re fully reproducible:
**FP8 Cohere reference:** `CohereLabs/North-Mini-Code-1.0-fp8`
**NVFP4 quant (ours):** `XanuNetworks/North-Mini-Code-1.0-NVFP4`
Same Spark, same vLLM recipe, same parsers — we only swapped the quant.
The results
| Metric | FP8 | NVFP4 | Delta |
| Single user @ 16K context | ~32 tok/s | ~52 tok/s | **~1.65× faster** |
| Two concurrent users (aggregate) | — | ~84 tok/s | scales cleanly |
| Weights in memory | 28 GB | 17 GB | **~40% smaller** |
| HumanEval | baseline | identical | **no measurable loss** |
The headline is the combination, not any single number. A 1.65× decode speedup *and* a 40% smaller footprint *and* no quality regression on HumanEval is the kind of result that changes what you’re willing to run locally. At 16K context — a realistic working window for agentic coding, not a synthetic micro-benchmark — NVFP4 took us from ~32 tok/s to ~52 tok/s. The model also stretches to 256K context if you need it, with throughput tapering off as you scale up, roughly in line with comparable Qwen-class coder models.
The honest part: bandwidth vs. tensor cores
It’s worth being precise about *where* the speedup comes from, because the forum discussion raised a fair point. Decode (token generation) is largely memory-bandwidth bound, and at 4 bits the weights are roughly the same size regardless of which 4-bit format you pick — so a chunk of that decode speedup is something any solid 4-bit quant would deliver, simply by moving less data per token.
NVFP4’s distinctive advantage shows up in **prefill** — prompt processing — where the GB10’s FP4 tensor cores actually do compute in the format, not just store it. We measured prefill for both quants and the NVFP4 path benefits there in a way that’s specific to the hardware, not just to the bit width. For agentic coding workloads, where you’re constantly feeding large files, tool outputs, and context back into the model, prefill efficiency is not a footnote — it’s a big part of the felt latency.
We’d rather tell you that than oversell a single number. The decode win is real and it’s nice; the prefill story is the part that’s genuinely about NVFP4 on this chip.
Reproduce it yourself
We didn’t want this to be a “trust me” benchmark. The full recipe — vLLM config, the `cohere_command4` tool and reasoning parser setup, and the prompt-processing-vs-concurrency logs — lives on both Spark Arena pages, and the recipe has been merged into the sparkrun GitHub registry so you can pull it directly.
**NVFP4 quant (ours):** `XanuNetworks/North-Mini-Code-1.0-NVFP4`
XanuNetworks/North-Mini-Code-1.0-NVFP4 · Hugging Face
If you tried North Mini Code earlier and ran into rough edges with tool calls or code generation (a few people did, especially through external coding agents), the parser configuration in our recipe is often the missing piece. Getting the `cohere_command4` tool and reasoning parsers wired up correctly is what makes the model behave like the agentic coder it was trained to be.
Where this is going
This is the pattern we care about at Xanu: take a strong open model, quantize it well, and prove out the numbers on hardware a single team or a single developer can own. North Mini Code in NVFP4 is a clean example — a capable agentic coding model running fast and small on one Spark, with a recipe anyone can reproduce.
If you run it on your own setup, we’d love to hear what you see — especially on heavier coding workloads than HumanEval. And a genuine nod to the Cohere team for shipping such a solid little agentic model under an open license. It’s exactly the kind of thing that makes local, sovereign inference worth doing.