clef-MLX-NVFP4
MLX conversion of Cloudflare/clef for mlx-vlm. Clef is a decision model: it returns a probability for every option of every question in one forward pass and does not generate text.
Clef support is not yet in an mlx-vlm release:
pip install "git+https://github.com/Lazarus-931/mlx-vlm.git@feat/clef"
from mlx_vlm import load, predict
model, processor = load("nativ-community/clef-MLX-NVFP4")
result = predict(model, processor, "Please refund my duplicate charge", {
"department": {
"type": "choice",
"instructions": "Which team should handle this ticket?",
"criteria": ["billing", "technical", "sales"],
},
})
print(result["answers"]["department"]["value"])
| Field | Value |
|---|---|
| Source | Cloudflare/clef |
| Source revision | ed3eed331870db2eff4b0db01237128ede8a00ce |
| Quantization | nvfp4, group size 16 |
| mlx-vlm | Lazarus-931/mlx-vlm@c16f81aa |
| Verification | Identical token ids on 11/11 reference records (text, image, video, image+video, two images, max_pixels, fps, num_frames); same answer as Cloudflare's torch reference (bf16) on 25/25 questions; largest probability gap 0.1033 |
- Downloads last month
- 54
Model size
27B params
Tensor type
BF16
·
U32 ·
Hardware compatibility
Log In to add your hardware
4-bit
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("nativ-community/clef-MLX-NVFP4") config = load_config("nativ-community/clef-MLX-NVFP4") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output)