steampunque/Qwen3.6-27B-MP-GGUF

Mixed Precision GGUF layer quantization of Qwen3.6-27B by Qwen

Original model: https://huggingface.co/Qwen/Qwen3.6-27B

The hybrid quant employs different quantization levels on a per layer basis to enable both high performance and small file size at the same time. The quants employed are all K to avoid slow CPU or older GPU processing of IQ quants. An extended layer definition E quant Q4_E_H for the model is defined as follows (updated 8/24/2026):

   LAYER_TYPES='[
   ["A","attn","Q","attn_q","K","attn_k","V","attn_v","O","attn_o","S","ssm","F","ffn","G","ffn_g","U","ffn_u","D","ffn_d"],
   ["MAP","VOSD"],
   [0 ,"Q5_K_6666"],[1 ,"Q5_K_6555"],[2 ,"Q4_K_6555"],[3 ,"Q4_K_6555"],
   [4 ,"Q4_K_5555"],[5 ,"Q4_K_5555"],[6 ,"Q4_K_4444"],[7 ,"Q4_K_5554"],
   [8 ,"Q4_K_4444"],[9 ,"Q4_K_4444"],[10,"Q4_K_4444"],[11,"Q4_K_5554"],
   [12,"Q4_K_4444"],[13,"Q4_K_4444"],[14,"Q4_K_4444"],[15,"Q4_K_5554"],
   [16,"Q4_K_4444"],[17,"Q4_K_4444"],[18,"Q4_K_4444"],[19,"Q4_K_5554"],
   [20,"Q4_K_4444"],[21,"Q4_K_4444"],[22,"Q4_K_4444"],[23,"Q4_K_5554"],
   [24,"Q4_K_4444"],[25,"Q4_K_4444"],[26,"Q4_K_4444"],[27,"Q4_K_5554"],
   [28,"Q4_K_5444"],[29,"Q4_K_5444"],[30,"Q4_K_5444"],[31,"Q4_K_5554"],
   [32,"Q4_K_5454"],[33,"Q4_K_5454"],[34,"Q4_K_5454"],[35,"Q4_K_5555"],
   [36,"Q4_K_5454"],[37,"Q4_K_5454"],[38,"Q4_K_5454"],[39,"Q4_K_5555"],
   [40,"Q4_K_5554"],[41,"Q4_K_5554"],[42,"Q4_K_5554"],[43,"Q4_K_5555"],
   [44,"Q4_K_5554"],[45,"Q4_K_5554"],[46,"Q4_K_5554"],[47,"Q4_K_5555"],
   [48,"Q4_K_5554"],[49,"Q4_K_5554"],[50,"Q4_K_5554"],[51,"Q4_K_5555"],
   [52,"Q4_K_5555"],[53,"Q4_K_5555"],[54,"Q4_K_5555"],[55,"Q4_K_6665"],
   [56,"Q4_K_5555"],[57,"Q4_K_5555"],[58,"Q4_K_5555"],[59,"Q5_K_5565"],
   [60,"Q5_K_6665"],[61,"Q5_K_6666"],[62,"Q5_K_6668"],[63,"Q6_K_8666"],
   [64,"Q4_K_6554"]
   ]'
   FLAGS="--token-embedding-type Q4_K --output-tensor-type Q6_K --layer-types-high"

The quant was optimized for approximately Q4_K_M bpw with strong performance across a curated set of reasoning test prompts, scoring solid 100% across the test set with greedy deterministic sampling, exhibiting good common sense. The model exhibits severe overthinking on some prompts but no infinite rep loops were found on any of the prompts. The quant includes layer 64 nextn MTP layer. If not using MTP the loader will give warning messages about unused tensors on layer 64 but the model will run correctly. The quant was updated 8/24/2026 for minium quant across layers at Q4_K to reduce entropy loss and provide noticeable performance improvement compared to the first Q4_E_H quant.

Comparison:

Quant	size	PPL	Comment
Q4_K_M	16.5e9	10.6	Q4_K_M with default embedding and output (no MTP layer)
Q4_E_H	16.7e9	9.8	Hybrid quant with Q4_K embedding Q6_K output (with 0.3B MTP layer)

Usage:

Qwen3.6-27B is a vision capable dense RL model. It can be used together with its multimedia projector layers to process images and text inputs and generate text outputs. The mmproj file is made available in this repository.

Due to the attention scheme used in the model, straightforward speculation approaches cannot be used. Later versions of llama.cpp as of 4/23/2026 attempt to circumvent this problem however efficiency will not be good so use of speculation prior to Qwen3.5 MTP patches is not recommended. As of llama.cpp b9180 MTP support for the model was added to upstream and may be experimented with by using the Q4_E_H_MTP quant.

On a 2x 4070 setup (1 RPC) approx performance is:

Q	QKV	NKV	gen tps
Q4_E_H	F16	100k +	25
Q4_E_H	Q8_0	160k +	23

The model went into an infinite rep loop with with Q8_0 QKV on one test prompt. No instability was found for F16 QKV.

The model appears to be trained to decide itself whether to do a think block or not. When it does a think block it can fall into very heavy overthinking but does come up with accurate answers. Over a small set of eval prompts the model did extremely well. To avoid the overthinking inject think start and think stop tokens first thing after assistant prompt:

THINK_START="<think>\n"
THINK_STOP="\n</think>\n\n"

If the model doesnt feel like doing thinking on a given prompt it will automatically do this. To force the model into a think block inject a bootstrap think start after the assistant prompt:

"<think>\n"

The model was found to be highly capable on reasoning tasks when skipping think block, with zero overthinking, just accurate direct deductions to final solutions. On two test prompts it was necessary to force the model into think mode to get the right answer.

Note: testing shows this model is prone to severe overthinking on some prompts, significantly higher than Qwen 3.5-27B. This version of the model appears to have been fine tuned with stronger forced introspections which make it harder for it to escape the think block if there is even the slightest ambiguity in its reasoning.

VISION:

The quant went 2 for 2 on a couple tough bird ID images prompts with severe overthinking on one of the images where it toggled back and forth between two candidates for a long time.

CODE:

The quant was tested across a small set of code gen prompts and found to be very solid in its ability to generate working programs.

Benchmarks:

A full set of both math and vision benchmarks for the model will eventually be given here: https://huggingface.co/spaces/steampunque/benchlm

Download the file from below:

Link	Type	Size/e9 B	Notes
Qwen3.6-27B.Q4_E_H.gguf	Q4_E_H	16.7e9 B	includes 0.3B MTP layer
Qwen3.6-27B.mmproj.gguf	F16	0.93e9 B	multimedia projector

A discussion thread about the hybrid layer quant approach can be found here on the llama.cpp git repository:

https://github.com/ggml-org/llama.cpp/discussions/13040