MedGemma 1.5 4B IT GGUF

GGUF conversion and quantization of Google's MedGemma 1.5 4B IT model.

Original Model

  • Model: google/medgemma-1.5-4b-it
  • Architecture: Gemma3ForConditionalGeneration
  • Parameters: ~4B
  • Original format: BF16 SafeTensors
  • Modality: Text + Vision

Original model:

https://huggingface.co/google/medgemma-1.5-4b-it

Files

File Format Approx. Size Purpose
medgemma-1.5-4b-it-f16.gguf F16 ~8.5 GB Main model
medgemma-1.5-4b-it-Q4_K_M.gguf Q4_K_M ~2.4 GB 4-bit quantized model
mmproj-medgemma-1.5-4b-it-f16.gguf F16 ~850 MB Vision projector

Quantization

The original model was downloaded from Hugging Face and converted to GGUF using llama.cpp.

Conversion pipeline:

MedGemma 1.5 4B IT
        |
        v
BF16 SafeTensors
        |
        v
    F16 GGUF
        |
        v
   Q4_K_M GGUF

The original model weights are split into two SafeTensors shards:

model-00001-of-00002.safetensors
model-00002-of-00002.safetensors

These are two parts of the same model, not two separate models.

Q4_K_M

The Q4_K_M file is a 4-bit quantized version of the F16 GGUF model.

Approximate sizes:

  • F16: ~8.5 GB
  • Q4_K_M: ~2.4 GB

Quantization reduces storage and memory requirements while keeping the same underlying model architecture and parameter count.

Multimodal / Vision

MedGemma 1.5 4B IT supports multimodal input.

For vision/image use, the main model should be used together with:

mmproj-medgemma-1.5-4b-it-f16.gguf

The mmproj file contains the multimodal vision projector and is separate from the main language model.

llama.cpp

These files were created using:

https://github.com/ggml-org/llama.cpp

Example Q4 quantization command:

llama-quantize \
    medgemma-1.5-4b-it-f16.gguf \
    medgemma-1.5-4b-it-Q4_K_M.gguf \
    Q4_K_M

Model Usage

Use the appropriate llama.cpp multimodal interface for image input and the main GGUF model for text generation.

The exact command-line options may depend on the version of llama.cpp being used.

Disclaimer

This repository contains a GGUF conversion and quantization of the original MedGemma model. It is not a newly trained or fine-tuned model.

Please read the original model documentation, license, terms of use, intended use, limitations, and safety information:

https://huggingface.co/google/medgemma-1.5-4b-it

This model should not be considered a medical device and should not replace qualified medical professionals.

Credits

Original model:

Google β€” MedGemma 1.5 4B IT

GGUF conversion and quantization:

llama.cpp

Downloads last month
242
GGUF
Model size
4B params
Architecture
gemma3
Hardware compatibility
Log In to add your hardware

4-bit

16-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for mackkkkkilllll/medgemma-1.5-4b-it-GGUF

Quantized
(49)
this model