Instructions to use White-Wolf13/MiniCPM-V-4.7-35B-A3B-APEX-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use White-Wolf13/MiniCPM-V-4.7-35B-A3B-APEX-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf White-Wolf13/MiniCPM-V-4.7-35B-A3B-APEX-GGUF # Run inference directly in the terminal: llama cli -hf White-Wolf13/MiniCPM-V-4.7-35B-A3B-APEX-GGUF
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf White-Wolf13/MiniCPM-V-4.7-35B-A3B-APEX-GGUF # Run inference directly in the terminal: llama cli -hf White-Wolf13/MiniCPM-V-4.7-35B-A3B-APEX-GGUF
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf White-Wolf13/MiniCPM-V-4.7-35B-A3B-APEX-GGUF # Run inference directly in the terminal: ./build/bin/llama cli -hf White-Wolf13/MiniCPM-V-4.7-35B-A3B-APEX-GGUF
Use Docker
docker model run hf.co/White-Wolf13/MiniCPM-V-4.7-35B-A3B-APEX-GGUF
- Llama new
- LM Studio
- Jan
- vLLM
How to use White-Wolf13/MiniCPM-V-4.7-35B-A3B-APEX-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "White-Wolf13/MiniCPM-V-4.7-35B-A3B-APEX-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "White-Wolf13/MiniCPM-V-4.7-35B-A3B-APEX-GGUF", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/White-Wolf13/MiniCPM-V-4.7-35B-A3B-APEX-GGUF
- Ollama
How to use White-Wolf13/MiniCPM-V-4.7-35B-A3B-APEX-GGUF with Ollama:
ollama run hf.co/White-Wolf13/MiniCPM-V-4.7-35B-A3B-APEX-GGUF
- Unsloth Desktop
- Pi
How to use White-Wolf13/MiniCPM-V-4.7-35B-A3B-APEX-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf White-Wolf13/MiniCPM-V-4.7-35B-A3B-APEX-GGUF
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "White-Wolf13/MiniCPM-V-4.7-35B-A3B-APEX-GGUF" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use White-Wolf13/MiniCPM-V-4.7-35B-A3B-APEX-GGUF with Docker Model Runner:
docker model run hf.co/White-Wolf13/MiniCPM-V-4.7-35B-A3B-APEX-GGUF
- Lemonade
How to use White-Wolf13/MiniCPM-V-4.7-35B-A3B-APEX-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull White-Wolf13/MiniCPM-V-4.7-35B-A3B-APEX-GGUF
Run and chat with the model
lemonade run user.MiniCPM-V-4.7-35B-A3B-APEX-GGUF-{{QUANT_TAG}}List all available models
lemonade list
- Hermes Agent
How to use White-Wolf13/MiniCPM-V-4.7-35B-A3B-APEX-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf White-Wolf13/MiniCPM-V-4.7-35B-A3B-APEX-GGUF
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default White-Wolf13/MiniCPM-V-4.7-35B-A3B-APEX-GGUF
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use White-Wolf13/MiniCPM-V-4.7-35B-A3B-APEX-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf White-Wolf13/MiniCPM-V-4.7-35B-A3B-APEX-GGUF
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "White-Wolf13/MiniCPM-V-4.7-35B-A3B-APEX-GGUF" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
MiniCPM-V 4.7 35B-A3B โ APEX GGUF
This repository provides official APEX (Adaptive Precision Quantization for MoE) GGUF releases of openbmb/MiniCPM-V-4.7-35B-A3B.
All quantized models feature layer-adaptive precision rules specifically designed to preserve Mixture-of-Experts routing dynamics and critical attention pathways, paired with a dedicated importance matrix (iMatrix) calibrated on a high-fidelity multilingual/code dataset.
Model Architecture Overview
- Backbone: Hybrid architecture combining Linear Attention (DeltaNet) with Full Self-Attention every 4 layers.
- Scale: 40 layers, 256 fine-grained experts, top-8 experts routed per token (~3.0B active parameters out of 35B total weights).
- Vision Tower: High-resolution SigLIP vision encoder coupled with a Window-Attention ViT Merger (
vit_merger) and DownsampleMLP. - Multimodal Projector: Available as
mmproj-minicpm-v-4.7-35b.gguf(BF16, 1.1 GB).
Available APEX Quantization Profiles
| File | Profile | BPW | Size | Recommended VRAM / RAM | Description & Target Use-Case |
|---|---|---|---|---|---|
minicpm-v-4.7-35b-APEX-I-BALANCED.gguf |
apex-i-balanced |
5.85 | 23.6 GB | 24โ32 GB | Recommended Sweet Spot. Router in F32, shared experts in Q8_0, edges in Q6_K, middle experts in Q5_K via iMatrix. Near-lossless reasoning and vision understanding. |
minicpm-v-4.7-35b-APEX-I-QUALITY.gguf |
apex-i-quality |
5.26 | 21.8 GB | 24โ32 GB | Maximum Fidelity. Q6_K base with Q8_0 shared experts and edge retention. Minimal perplexity degradation vs original BF16. |
minicpm-v-4.7-35b-APEX-I-COMPACT.gguf |
apex-i-compact |
3.81 | 15.4 GB | 16โ24 GB | Balanced Consumer Footprint. Q4_K_M base with protected shared experts. Fits comfortably in 16โ24 GB GPUs (RTX 4080/3090/4090). |
minicpm-v-4.7-35b-APEX-MINI.gguf |
apex-mini |
3.11 | 12.5 GB | 16 GB | Compact Consumer Variant. Q3_K_M base with IQ2_S middle experts for 16GB GPUs or memory-constrained machines. |
minicpm-v-4.7-35b-APEX-NANO.gguf |
apex-nano |
2.65 | 10.7 GB | 12 GB | Ultra-Compact Edge Variant. Extreme IQ2_XXS compression for low-resource 12GB VRAM environments. |
mmproj-minicpm-v-4.7-35b.gguf |
Multimodal Projector | 16.0 | 1.1 GB | N/A | Required for image/video input. Full-precision SigLIP vision tower and vit_merger. |
Note: To perform vision-language tasks (image understanding, OCR, document analysis), you must pass both the quantized model file and
mmproj-minicpm-v-4.7-35b.gguf.
APEX Precision Architecture
Unlike uniform quantization (which applies identical bit-widths across all tensors), APEX dynamically differentiates tensor criticality:
- Router Protection (F32):
ffn_gate_inplayers remain unquantized in full 32-bit floating point to ensure routing entropy and expert selection are never perturbed. - Shared Expert Elevation (Q8_0): Shared experts (
ffn_*_shexp), which activate on every single token regardless of routing, are kept in 8-bit precision. - Edge Attention Guard (Q6_K): Initial and final layers governing syntactic grounding and output formatting are protected in 6-bit K-quants.
- iMatrix Adaptive Routing (Q5_K / Q4_K / IQ): Middle layer experts are selectively quantized along the axes of least activation impact derived from second-order statistics.
Citation & Acknowledgements
- Original Base Model: OpenBMB / MiniCPM-V-4.7-35B-A3B
- Quantization Framework: mudler/apex-quant
- Runtime Engine: ggml-org/llama.cpp
- Downloads last month
- 4,819
We're not able to determine the quantization variants.
Model tree for White-Wolf13/MiniCPM-V-4.7-35B-A3B-APEX-GGUF
Base model
openbmb/MiniCPM-V-4.7-35B-A3B