soyrsoyr/glm-4.5-Air-NVFP4-MTP

GLM-4.5-Air NVFP4 with MTP

This is a calibrated NVFP4 checkpoint of zai-org/GLM-4.5-Air (revision a24ceef6ce4f3536971efe9b778bdaa1bab18daa), including its MTP decoder. The backbone and MTP decoder projections were calibrated and quantized together in one run. The MTP projection eh_proj and non-linear/router/norm weights remain BF16 where vLLM does not load them as NVFP4 modules.

The checkpoint was tested on an NVIDIA B200 with vLLM 0.28.0 and MTP speculative decoding. A 64-token completion succeeded with an 85.3% draft-token acceptance rate (29 of 34 draft tokens accepted). This is a single serving smoke test, not a quality or throughput benchmark. Quantization used Transformers 5.17.0 and compressed-tensors 0.19.1a20260919.

vllm serve soyrsoyr/glm-4.5-Air-NVFP4-MTP \
  --speculative-config '{"method":"mtp","num_speculative_tokens":1}'