0xBakeer

Research

One paper so far. It answers one question in speculative decoding: how many tokens to guess each round, when the cost of checking them climbs in steps.

StairCut: sizing speculative draft trees on a measured verification-cost staircase

Khaled Bakeer, 29 September 2026. Preprint on Zenodo, CC BY 4.0. doi:10.5281/zenodo.23045275

Read the PDF The code, v0.2.0-staircut Weights and drafters

What it says

A dense 27B model on one GPU is slow because every step reads all the weights. Speculative decoding helps: drafters guess tokens and the model checks them in one pass. Qwen3.8-27B makes that harder, because 48 of its 64 layers carry a recurrent state that cannot step back after a wrong guess. TandemLLM commits the state of the accepted path in closed form instead.

The cost of a check does not grow smoothly. On this board a check of 16 rows costs 74.4 ms and a check of 8 costs 75.5 ms, at 1k tokens of context. Row 17 opens a new tile in the 4-bit kernel and the cost jumps. StairCut prices every round on that measured staircase and keeps the tree size that pays the most expected tokens per millisecond.

On one request at a time, with the published NVFP4 weights, StairCut writes 49.89 tokens a second. The same drafters at a fixed block write 47.13 and 44.80, and plain greedy decoding writes 13.69. On a teacher-forced bench of 25 workloads it beats the fixed block of 16 on 23 and the block of 8 on 22.

Agreement with plain decoding is reported as measured, not claimed. Of 30 answers, 17 are identical to the end, and 13 take the other token at a near tie, where the two best tokens are at most one unit in the last place apart.

Qwen3.8-27B, one request, tokens a second

  1. StairCut49.89
  2. Fixed block of 8, the narrow drafter alone47.13
  3. Fixed block of 16, the wide drafter alone44.80
  4. Plain greedy decoding13.69
From the StairCut paper: published NVFP4 weights, 50 requests of 256 prompt tokens and 256 new tokens after 3 warm-ups, thinking off, greedy. StairCut writes 3.64 times as many tokens a second as plain decoding on the same weights.

Cite it

The paper measured commit 431ddee. Release v0.2.0-staircut is that code, and the Hugging Face collection holds the weights and drafters it ran on.

BibTeX
@misc{bakeer2026staircut,
  author    = {Bakeer, Khaled},
  title     = {StairCut: sizing speculative draft trees on a
               measured verification-cost staircase},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.23045275},
  url       = {https://doi.org/10.5281/zenodo.23045275}
}

Other measurements