MiniCPM-V 4.7 35B-A3B โ€” APEX GGUF

This repository provides official APEX (Adaptive Precision Quantization for MoE) GGUF releases of openbmb/MiniCPM-V-4.7-35B-A3B.

All quantized models feature layer-adaptive precision rules specifically designed to preserve Mixture-of-Experts routing dynamics and critical attention pathways, paired with a dedicated importance matrix (iMatrix) calibrated on a high-fidelity multilingual/code dataset.


Model Architecture Overview

  • Backbone: Hybrid architecture combining Linear Attention (DeltaNet) with Full Self-Attention every 4 layers.
  • Scale: 40 layers, 256 fine-grained experts, top-8 experts routed per token (~3.0B active parameters out of 35B total weights).
  • Vision Tower: High-resolution SigLIP vision encoder coupled with a Window-Attention ViT Merger (vit_merger) and DownsampleMLP.
  • Multimodal Projector: Available as mmproj-minicpm-v-4.7-35b.gguf (BF16, 1.1 GB).

Available APEX Quantization Profiles

File Profile BPW Size Recommended VRAM / RAM Description & Target Use-Case
minicpm-v-4.7-35b-APEX-I-BALANCED.gguf apex-i-balanced 5.85 23.6 GB 24โ€“32 GB Recommended Sweet Spot. Router in F32, shared experts in Q8_0, edges in Q6_K, middle experts in Q5_K via iMatrix. Near-lossless reasoning and vision understanding.
minicpm-v-4.7-35b-APEX-I-QUALITY.gguf apex-i-quality 5.26 21.8 GB 24โ€“32 GB Maximum Fidelity. Q6_K base with Q8_0 shared experts and edge retention. Minimal perplexity degradation vs original BF16.
minicpm-v-4.7-35b-APEX-I-COMPACT.gguf apex-i-compact 3.81 15.4 GB 16โ€“24 GB Balanced Consumer Footprint. Q4_K_M base with protected shared experts. Fits comfortably in 16โ€“24 GB GPUs (RTX 4080/3090/4090).
minicpm-v-4.7-35b-APEX-MINI.gguf apex-mini 3.11 12.5 GB 16 GB Compact Consumer Variant. Q3_K_M base with IQ2_S middle experts for 16GB GPUs or memory-constrained machines.
minicpm-v-4.7-35b-APEX-NANO.gguf apex-nano 2.65 10.7 GB 12 GB Ultra-Compact Edge Variant. Extreme IQ2_XXS compression for low-resource 12GB VRAM environments.
mmproj-minicpm-v-4.7-35b.gguf Multimodal Projector 16.0 1.1 GB N/A Required for image/video input. Full-precision SigLIP vision tower and vit_merger.

Note: To perform vision-language tasks (image understanding, OCR, document analysis), you must pass both the quantized model file and mmproj-minicpm-v-4.7-35b.gguf.


APEX Precision Architecture

Unlike uniform quantization (which applies identical bit-widths across all tensors), APEX dynamically differentiates tensor criticality:

  1. Router Protection (F32): ffn_gate_inp layers remain unquantized in full 32-bit floating point to ensure routing entropy and expert selection are never perturbed.
  2. Shared Expert Elevation (Q8_0): Shared experts (ffn_*_shexp), which activate on every single token regardless of routing, are kept in 8-bit precision.
  3. Edge Attention Guard (Q6_K): Initial and final layers governing syntactic grounding and output formatting are protected in 6-bit K-quants.
  4. iMatrix Adaptive Routing (Q5_K / Q4_K / IQ): Middle layer experts are selectively quantized along the axes of least activation impact derived from second-order statistics.

Citation & Acknowledgements

Downloads last month
4,819
GGUF
Model size
35B params
Architecture
qwen35moe
Hardware compatibility
Log In to add your hardware

We're not able to determine the quantization variants.

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for White-Wolf13/MiniCPM-V-4.7-35B-A3B-APEX-GGUF

Quantized
(78)
this model