Qwen3.8-27B-DSpark-Agentic

A DSpark speculative-decoding drafter for Qwen/Qwen3.8-27B, trained on real agentic coding/assistant sessions. DSpark drafts a block of tokens conditioned on auxiliary features tapped from the target model's layers, with a Markov head for intra-block token dependency and a confidence head that predicts per-position acceptance. Trained with SpecForge; serves with SGLang.

A drafter never changes model output: every drafted token is verified by the target model. It only changes decode speed, through the acceptance rate on your traffic. This one is aimed at agentic/assistant workloads.

Model specifications

Target model	Qwen/Qwen3.8-27B (evals below ran FP8 and NVFP4 builds of this trunk)
Draft parameters	1,359,284,737 (1.36B), BF16, 62 tensors in one safetensors file
Architecture	5 full-attention transformer layers, hidden 5,120, GQA 40 query / 8 KV heads, head_dim 128, MLP 10,240
Target feature taps	layers 4, 16, 28, 40, 52 (of 64)
Heads	vanilla Markov head, rank 256; confidence head with Markov features
DSpark block size	7 draft tokens (verify width 8 with the target bonus token)
Positional encoding	YaRN (factor 32, original 8,192), max position 262,144
Vocabulary	248,320 (uses the target model's tokenizer; no separate tokenizer shipped)

model.safetensors sha256: 3c4e7d51f6a7492fd3c289e392424ab0fba31dc7b4ab4896da10a5eeb21fcd6f

Training

Trained from scratch (random initialization — no third-party model weights) with the SpecForge harness. Labels are the target model's own temperature-0 regenerations over a prompt mix of real agentic coding/assistant sessions and a public chat prompt blend, covering both thinking and non-thinking modes, training on the last turn of each multi-turn session. Roughly 1,000 cumulative optimizer steps on B200 / RTX PRO 6000-class hardware; this repo is the final checkpoint of that run.

Evaluation results

All numbers below are absolute measurements of this checkpoint.

Acceptance length — SGLang, FP8 target, sampled thinking-mode decoding

Setup: SGLang; target Qwen/Qwen3.8-27B-FP8 on a single RTX PRO 6000 Blackwell (96 GB); this drafter unquantized BF16; DSpark block size 7; temperature 0.6, top-k 20, top-p 0.95; thinking enabled; max_new_tokens=2048. Acceptance length = mean accepted tokens per verification step including the target bonus token (per-request completion_tokens / spec_verify_ct, unweighted mean).

Workload	Prompts	Acceptance length
Real agentic sessions — short chat turns	64	2.88 (per-request 2.09–5.94)
Real agentic sessions — long agentic turns	64	2.92 (per-request 2.03–5.45)
GSM8K	128, concurrency 8	4.61

The GSM8K cell decoded 60,365 output tokens in 87.1 s — 693 tok/s aggregate at concurrency 8 on the single card. The two session buckets produced 81,199 and 115,263 output tokens respectively. The session prompts are held-out real agentic/assistant turns, not used in training.

Acceptance — greedy argmax, NVFP4 GGUF trunk (memra engine)

Setup: memra engine, greedy decoding, single stream, RTX PRO 6000 Blackwell (96 GB); trunk = NVFP4 GGUF build of the same target; DSpark block size 7, with target features harvested under the shifted-label (DSPARK-strategy) convention this drafter was trained on. The speculative stream is gated byte-identical to plain greedy decode on every prompt (verified across 5 repetitions). Tokens per round = 1 + accepted/rounds, read from the server's per-request speculation counters; 256-token completions.

Workload	Prompts	Tokens per round — adaptive verify	Tokens per round — full-block verify
Agentic-session prompts	16	2.412	2.784
Short chat prompts	16	2.228	2.415
GSM8K	12	3.664	4.620

The adaptive column uses the engine's default adaptive verification-window ladder; the full-block column verifies the whole 7-token draft block plus the bonus token every round — the same verification protocol as the SGLang table above.

Single-stream greedy throughput on held-out agentic-session prompts, 128 tokens per prompt: 96.3 tok/s with this drafter vs 65.0 tok/s plain decode on the same engine, card, and prompts (1.48×; per-repetition means 95.4–96.3 vs 63.1–65.1 across 5 runs), with speculative output byte-identical to plain decode on every prompt.

These memra numbers require the shifted-label (DSPARK-strategy) feature harvest in the engine. That harvest mode is landing in an upcoming memra release; it is not part of any tagged memra release yet.

Acceptance is protocol-dependent: sampled thinking-mode decoding on an FP8 trunk (first table) and greedy argmax on an NVFP4 trunk (second table) are different observables. Both are reported as measured; expect your own numbers to depend on trunk quantization, sampling settings, verification width, and traffic shape.

Serving

SGLang (use a version with DSPARK support):

sglang serve \
  --trust-remote-code \
  --model-path Qwen/Qwen3.8-27B-FP8 \
  --tp-size 1 \
  --speculative-algorithm DSPARK \
  --speculative-draft-model-path Avifenesh/Qwen3.8-27B-DSpark-Agentic \
  --speculative-dspark-block-size 7 \
  --speculative-draft-model-quantization unquant \
  --mamba-scheduler-strategy extra_buffer

memra (Rust + CUDA engine for RTX Blackwell) has the DSpark route wired and exactness-gated — spec output byte-identical to plain decode — landing in an upcoming release behind an env-gated flag.

The team behind this drafter serves Qwen3.8-27B in production at inference.tiyuvta.ai.

License

Apache-2.0. The drafter was trained from scratch; no third-party model weights were used for initialization. Training labels were generated by Qwen/Qwen3.8-27B (Apache-2.0); training used the open-source SpecForge harness.

Citations

• DSpark (speculative decoding method): arXiv:2607.05147
• DFlash (block drafting lineage): z-lab/dflash, arXiv:2602.06036
• SpecForge (training harness): sgl-project/SpecForge