@danielhanchen on Hugging Face: "Google releases Gemma 4 QAT. ✨ You can now run Gemma 4 at 3x less memory with…"

Join the community of Machine Learners and AI enthusiasts.

posted an update 2 days ago

Post

3730

Google releases Gemma 4 QAT. ✨
You can now run Gemma 4 at 3x less memory with near original performance.

QAT makes it possible to run Gemma 4 26B-A4B on 16GB RAM.

GGUFs: https://huggingface.co/collections/unsloth/gemma-4-qat
QAT Guide: https://unsloth.ai/docs/models/gemma-4/qat

I saw QAT stuff going on for a week or two. I wonder if finetunes going forward will all use QAT as a base?

Or will they do Loras?

I'd wondered for a bit why i see some Lora's where that are almost as big as the base model, vs say SDXL where a Lora is 200Mb vs a 6Gb base.

Either way, I'll be keeping eye eyes peeled for interesting releases everywhere.

In this post