There’s a particular satisfaction in watching a model that has no business running on the hardware in front of you generate its first coherent token.
We got GLM-5.2 — the full, unpruned model, all 256 experts — serving at 327,000 tokens of context at about 25 tokens per second, across four NVIDIA DGX Spark boxes sitting on a shelf. Not a distilled or pruned-down version. The real thing, sharded across four machines you could carry in a backpack.
Visit the github repo here: https://github.com/XanuNetworks/GLM-5.2-QuantTrio-DCP-4x-DGX-Spark
The full recipe is open source on GitHub. Here’s the short version of the story behind it.
Four small boxes, one big model
A DGX Spark is NVIDIA’s little GB10 machine — 128 GB of unified memory in a workstation-sized box. One is capable. Four wired together start to look like something you’d normally need a rack to build.
GLM-5.2 is far too big to fit on any single Spark. The trick is splitting the model’s weights across all four nodes and sharding its running memory across them too, so no single box has to hold the whole thing. Do that, and four modest boxes can hold a frontier model’s full context.
One recipe, four modes
Four nodes give you a fixed memory budget, and you get to decide how to spend it — on depth (one very long conversation) or width (several conversations at once). We wrapped that choice into a single setting:
- Depth — one session, up to 327K tokens, maximum speed (the default)
- Max context — one session, up to 655K tokens
- Multi-user — 3 agents at once, or 5 with a bit less context each
Same model, same boxes. Flip one variable and the cluster reconfigures from one deep session to five parallel ones.
The bug that nearly cost us a week
Early on, decode was coming in slow — around 15 tokens per second, well below what it should. So we did what everyone does: we suspected the software. The driver? The kernel library? The networking layer? All plausible, all days of work to bisect.
The real cause: one of the four GPUs had silently wedged itself at a quarter of its clock speed. Not crashed, not erroring — just quietly running slow. And in a tightly synchronized cluster, the slowest GPU sets the pace for all four. Three healthy GPUs were sitting around waiting on one sandbagging node, every single token.
No reboot fixed it. The only thing that cleared it was a full cold power cycle — pulling the plug and letting it fully discharge. After that: 15 to 23 tokens per second on the identical setup. Nothing in software had changed.
So the number one line in our troubleshooting guide is now: if your decode is slow, check your GPU clocks before you blame anything else.
Benchmark honestly
Most benchmark tools fill the context with random tokens. It’s convenient, and for a model like this it’s misleading — random garbage makes the model’s speed-up trick misfire and reports throughput lower than reality. Feed it real prompts — actual code and prose — and it runs 10–15% faster, because real output is predictable and structured.
Every number we published is measured on real, coherent prompts. Less flattering than the alternatives. Also true.
What it can do
- ~25 tokens/sec that barely fades as the conversation gets longer — flat all the way out to 300K tokens
- 87/100 on an independent tool-calling benchmark
- Quarter of a million tokens of context, running in a room, not a datacenter
Why bother?
Because the frontier doesn’t have to live exclusively in someone else’s cloud. The gap between “too big to self-host” and “running on gear you own” is closing fast — and a lot of what stands in the way isn’t compute, it’s operational knowledge nobody’s written down yet.
That’s why the whole recipe is public. Have a look and go build something.
Running your own DGX Spark cluster? I’d love to compare notes — get in touch.