0xBakeer

DGX Spark recipes

The serving recipes I published before TandemLLM, one per model, each with the numbers I measured at the time. They stay online as they were and get no more updates. TandemLLM continues the work.

TandemLLM

Qwen3.8-27B FP8 on one Spark

From 7.88 to 58.5 tok/s for one request without touching the weights: speculative decoding and prefix caching, measured side by side in vLLM.

DocsReportGitHub

to

Qwen3.8-27B 4-bit on one Spark

NVFP4, MixedInt4-AutoRound and the FP8 baseline through one benchmark script: 75 tok/s for one request, 246 tok/s over eight requests at once.

DocsReportGitHub

to

Qwen3.8-Flash-Next on one Spark

A 180B model on a 128 GB box. The model only ever reads from its 51B n-gram embedding table, never writes, so the table can live on the NVMe. Flash-Next also runs in TandemLLM since release 0.8.0.

DocsGitHub

to

DeepSeek V4 Flash on one Spark

A 2.54-bit EXL3 quant in 92 GB with a 384k context, thinking, tools and an OpenAI-compatible API, installed with one command. 30 tok/s on prose, 57 on code.

DocsGitHub

to

Ling-3.0-flash on one Spark

Ant Group's 124B mixture of experts in MXFP4 on SGLang, with the Humming MoE backend, an FP8 LM head and DSpark drafting. The two public setups it combines differ in one variable each, so the comparison can be run again.

GitHub

to

DeepSeek-V4.1-Flash on one Spark

510 GB of weights on a box with 128 GB of memory. The experts a workload uses stay in memory, as the checkpoint stores them, and every other expert is read from the NVMe when a token needs it. Version 0.5.0 lists what works and what does not.

GitHubThe article series

to