⚑ Qwen 2.5 Coder 1.5B Instruct (GGUF Quantized)

High-efficiency, low-latency quantized GGUF weights of Alibaba Cloud's Qwen2.5-Coder-1.5B-Instruct, optimized for edge execution, fast code autocompletion, and low-overhead draft verification in speculative decoding pipelines.


πŸ“Š Quantization Sizing & VRAM Requirements

File Name Quant Method Bits Per Weight (bpw) Model File Size Minimum Required VRAM Recommended Execution
qwen2.5-coder-1.5b-instruct-q4_k_m.gguf Q4_K_M 4.5 bpw ~1.05 GB 2.2 GB VRAM Ultra-fast local coding / Speculative Draft
qwen2.5-coder-1.5b-instruct-q8_0.gguf Q8_0 8.5 bpw ~1.85 GB 3.1 GB VRAM Near-lossless code generation

πŸš€ How to Run Locally

1. Run with llama.cpp

./llama-cli \
  -m qwen2.5-coder-1.5b-instruct-q4_k_m.gguf \
  -p "def quicksort(arr):" \
  -n 256 \
  -t 8 -ngl 99

2. Accelerate Heavy 70B Models (Speculative Decoding)

This 1.5B model serves as an ideal high-acceptance draft model to double generation speed on larger models (such as Qwen 2.5 32B or Llama 3.3 70B).


πŸ“– Official Hardware Benchmarks & Guides (TechGuidePro)


Research Reference

Maslmany, A. (2026). Speculative Verification Dynamics in Heterogeneous Local LLM Inference. CERN Zenodo. DOI: 10.5281/zenodo.22641931

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for abdulrahmanmaslmany/Qwen2.5-Coder-1.5B-Instruct-GGUF

Finetuned
(244)
this model