agentionai/Signal-3.8-27B-GGUF

Signal 3.8 27B

This is Qwen3.8-27B that gets to the answer faster.

Update 2026-09-13 — use these settings for stable, short reasoning Temperature 0.6, min-p 0.05 (top-p 0.95, top-k 20), KV cache q8_0 or f16, and the MTP draft head on. Without min-p, long answers can run away into repetition on some draws; a 4-bit value cache makes that deterministic. Both are fixed by the settings above, on Signal and on the base model alike.

The tiers in this repository were rebuilt on 2026-09-13. The first release's tiers could, rarely, repeat an answer line when sampling without the draft head; the rebuilt tiers showed no loop or cap in 55 traces under the same conditions. If you downloaded before that date, re-download.

AgentionAI Signal is a minimally invasive fine-tune of Qwen3.8-27B designed for lower generation latency and better token efficiency. On our held-out general-prompt evaluation, Signal produces 57% fewer answer tokens and uses 52% fewer thinking tokens, while matching or improving the measured answer quality of the base model.

The percentages above were measured on the first release. The tiers rebuilt on 2026-09-13 trade a little of that reduction for stability; their re-measurement on the same prompt set is in progress and will replace these numbers.

The result is substantially faster end-to-end generation: on typical chat prompts, Signal can finish in less than half the wall time of the untouched Qwen3.8-27B on the same hardware.

Signal gets there by being more direct rather than by truncating answers. It removes unnecessary preambles, excessive formatting, sign-offs, and explanatory narration while preserving the substance of the response. In thinking mode, it keeps the useful reasoning steps while spending fewer tokens describing the process.

Signal is trained by self-distillation: on Qwen3.8-27B's own answers, generated under an instruction to be direct that the released model no longer needs. No external data and no other model's outputs went into it, which is why it keeps the base model's knowledge and voice intact.

It is a drop-in GGUF replacement for llama.cpp setups that already run Qwen3.8-27B.

What changes, measured

We evaluated Signal against the untouched Qwen3.8-27B Q8_0 using the same server, sampling settings, prompts, and otherwise identical model file. All prompts in these evaluations were held out from tuning.

	base Q8_0	Signal	change
general answers, median tokens	243	104	-57%
answers opening with a preamble ("Sure!", "Great question")	13%	0%	gone
answers with markdown headers	47%	18%	-62%
answers with bold	85%	52%	-39%
coding answers, median tokens	159	142	-11%
coding answers, p90 tokens	1026	914	-11%

Thinking mode, same prompts with reasoning on:

	base Q8_0	Signal	change
reasoning tokens, general prompts, median	153	74	-52%
reasoning tokens, coding prompts, median	225	166	-26%
reasoning tokens, GSM8K, median	119	81	-32%

Quality, exact match on GSM8K:

	base Q8_0	Signal
thinking off, 60 problems	98.3%	98.3%
thinking on, 40 problems	92.5%	95.0%

Shorter is not cheaper: no answer in the 100-prompt style set was cut off early (0 answers ending on a header or a colon, 0 unclosed code blocks). On a separate loop check of 55 thinking-mode traces (math, GSM8K, coding and general prompts, Q5_K_M on mainline llama.cpp with and without the draft head, q8_0 KV cache) no trace looped or hit the token cap.

Faster with speculative decoding

Qwen3.8-27B carries a built-in multi-token-prediction draft head. Signal's answers are more predictable, so the drafter agrees with the model more often:

Draft acceptance and decode speed with --spec-type draft-mtp, both models Q8_0 on the same machine (Strix Halo, Vulkan), 200-token greedy runs for the fixed-draft rows:

prompt / draft length	base acceptance	Signal acceptance	decode speed vs base
prose, draft 3	39%	47%	+10%
prose, draft 4	35%	28%	-9%
structured output (JSON), draft 3	72%	94%	+20%
structured output (JSON), draft 4	66%	87%	+22%
chat prompts, sampled at 0.7, adaptive draft ≤4 (40 prompts)	57%	60%	—

Combined with the shorter answers, a typical chat reply finishes in well under half the wall time of the base model on the same hardware. Because of the specific finetuning there is no degredation over long context length.

Why shorter is not worse

Chain-of-thought length is not mostly a function of problem difficulty. A recent study of reasoning models (Lotfi, Kirichenko, Li and Liu, Quantized Reasoning Models Think They Need to Think Longer, but They Do Not, arXiv:2606.00206) finds that a small set of hesitation and branching tokens ("Wait", "But", "Alternatively", "maybe") drives most of the excess: they are sampled at the positions where the model is least certain, each one opens a new line of reasoning, and in up to half of the failures the model had already reached the right answer before branching away from it. Suppressing those fifty tokens at decode time cut reasoning length by 12 to 23 percent across five models without hurting accuracy.

Signal reaches the same place from the opposite side. The penalty method tells the model what not to say, with a fixed bias on a fixed word list at every step. Signal was taught by example: it learned from the base model's own answers where it went straight to the point, so nothing is banned and nothing is penalised at decode time. The model still branches where a branch does work; it has simply stopped preferring the detour where the direct continuation was as good.

Files

Nine tiers from IQ3_XXS to Q8_0, one folder each. AP = Agention Precision: tier names refer to the base ftype, the per-tensor types are chosen for accuracy per gigabyte using Agention Precision recipes. All built from Signal's BF16 (rebuilt 2026-09-13) using our custom imatrix; KLD and top-1 are against that BF16 on a held-out 2026 corpus and wikitext-2, -c 2048. VRAM is the file size: this is a dense model, nothing offloads.

tier	size	eff. bpw	KLD held-out	top-1	KLD wikitext	what
Q8_0	27.05 GiB	8.26	0.0040	94.7%	0.0046	reference quality; every number below was measured on this file
AP-Q6_K	20.89 GiB	6.57	0.0055	94.5%	0.0075	Q6_K with imatrix
AP-Q5_K_M	18.19 GiB	5.72	0.0081	94.0%	0.0093	Q5_K_M with imatrix
AP-Q4_K_XL	16.35 GiB	5.14	0.0124	93.3%	0.0148	precision tier — matched unsloth's UD-Q4_K_XL on the base model at the same size
AP-Q4_K_M	15.83 GiB	4.98	0.0184	92.3%	0.0218	fast tier — +20% prefill over UD-Q4_K_M on the base model at the same size, some quality traded
AP-IQ4_XS	13.27 GiB	4.17	0.0287	90.9%	0.0327	UD-IQ4_XS per-tensor map with our imatrix — same size
AP-Q3_K_XL	14.05 GiB	4.41	0.0595	88.1%	0.0562	3-bit body, Q6_K output + Q5_K embeddings — the 16 GB slot with headroom
AP-IQ3_S	12.38 GiB	3.89	0.0661	87.9%	0.0655	i-quant 3-bit body, protected head — the value pick under 4-bit
AP-IQ3_XXS	11.39 GiB	3.58	0.1003	85.7%	0.1008	smallest tier, most aggressive; quality traded for a long context in 16 GB

Which one: AP-Q4_K_XL if 16.5 GiB fits, AP-IQ4_XS for the 13 GiB slot, AP-Q4_K_M when prefill speed matters more than the last bit of quality, AP-Q6_K or Q8_0 when memory is no object. For a 16 GB card, the three-bit tiers leave room for a long context plus the MTP draft head or the vision projector: AP-IQ3_XXS (11.4 GiB) fits 100K+ context with extras, AP-IQ3_S (12.4 GiB) is the value pick with a protected head, and AP-Q3_K_XL (14.1 GiB) trades context room for a little more margin.

Running

Thinking on and off both work; the chat template is the original Qwen3.8 template.

Sampling: temperature 0.6, top-p 0.95, top-k 20, min-p 0.05, as in the commands below. The min-p cuts the low-probability tail that reasoning runaways are sampled from: on five prompts that ran away at temperature 0.7 without min-p, this setting answered 14 of 15 draws against 7 of 15, and the base model runs away on the same prompts at the same rate. Use sampling rather than greedy decoding (we saw a single loop at temperature 0), and do not raise the temperature to 1.0: on the smallest tier (AP-IQ3_XXS, 40 level-5 MATH-500 problems) 1.0 doubled the 90th-percentile reasoning length to the token cap and cost 12 points of accuracy against 0.7.

KV cache: keep the cache at q8_0 or f16 for both K and V. A 4-bit value cache (-ctv q4_1 or q4_0) makes long reasoning traces degenerate into a repeated phrase; we reproduced a user's loop report with exactly that setting, and the same prompt is clean at q8_0.

llama.cpp

llama-server -hf agentionai/Signal-3.8-27B-GGUF:AP-Q4_K_XL \
  --jinja -ngl 999 -fa on -c 65536 -ctk q8_0 -ctv q8_0 \
  --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.05

Add the built-in draft head for the throughput above (needs a build with --spec-type draft-mtp):

  --spec-type draft-mtp --spec-draft-n-max 4

Thinking is on by default. To turn it off per request, send "chat_template_kwargs": {"enable_thinking": false} with the chat completion.

vLLM (NVIDIA Blackwell)

A community NVFP4 (W4A16) build for vLLM, MTP included, is at Shockem/Signal-3.8-27B-NVFP4: 2x RTX 5060 Ti 16 GB, TP=2, 200k context, FP8 KV, ~50-70 tok/s, total solution wall time roughly half of the stock model. The BF16 source for your own quants is agentionai/Signal-3.8-27B.

Ollama / LM Studio

Import the GGUF as any Qwen3.8-27B file. Use the sampling settings above; the template is embedded in the file.

🛠️ 7. Method and tooling

Signal is trained by self-distillation on the base model's own answers; no external data. The tiers are built with our own Rust tooling, agention-infer: gguf-pack for byte-exact, content-addressed tier builds and validation, gguf-info for inspection, and a per-tensor distortion solver behind the Agention Precision recipes. Every tier is measured against Signal's own BF16 on a held-out 2026 corpus before it ships.

Vision

Signal keeps Qwen3.8-27B's image input. The vision encoder and projector are untouched by the tune, so the base model's own mmproj-BF16.gguf is at the repository root, 0.87 GiB. Download it alongside any tier:

Support AgentionAI

Signal3.8 is released freely. If it saves you compute or makes Qwen more useful, you can sponsor continued tuning, quantization and benchmarking on GitHub.