Instructions to use DevelopingDad/Gemma4-12B-it-Fast with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use DevelopingDad/Gemma4-12B-it-Fast with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="DevelopingDad/Gemma4-12B-it-Fast")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("DevelopingDad/Gemma4-12B-it-Fast")
model = AutoModelForMultimodalLM.from_pretrained("DevelopingDad/Gemma4-12B-it-Fast", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use DevelopingDad/Gemma4-12B-it-Fast with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "DevelopingDad/Gemma4-12B-it-Fast"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DevelopingDad/Gemma4-12B-it-Fast",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/DevelopingDad/Gemma4-12B-it-Fast

SGLang

How to use DevelopingDad/Gemma4-12B-it-Fast with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "DevelopingDad/Gemma4-12B-it-Fast" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DevelopingDad/Gemma4-12B-it-Fast",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "DevelopingDad/Gemma4-12B-it-Fast" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DevelopingDad/Gemma4-12B-it-Fast",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use DevelopingDad/Gemma4-12B-it-Fast with Docker Model Runner:
```
docker model run hf.co/DevelopingDad/Gemma4-12B-it-Fast
```

DevelopingDad commited on 16 days ago

Commit

64c6387

verified ·

1 Parent(s): 6aa5b9a

Fix license and checksum links in the rendered model card

Browse files

Use explicit blob URLs for extensionless legal and checksum files, and clarify that packed storage elements do not change the 12B architecture. Target weights, runtime sources, and measured recipe are unchanged.

Files changed (3) hide show

README.md +4 -2
SHA256SUMS +2 -2
serving/README.md +3 -1

README.md CHANGED Viewed

@@ -42,13 +42,15 @@ A calibrated **mixed NVFP4/FP8 export of DevelopingDad's Gemma 4 12B IT Trial62*
 The source contains 11,959,730,224 stored BF16 parameter elements across 48 language layers. Its native configuration permits 262,144 total positions; **this release's tested serving recipe uses 32,768 total prompt-plus-output tokens**.
 Text MLP weights use importance-weighted MSE NVFP4, groups of 16, static FP8 block scales and global divisors. MLP activations use locally dynamic NVFP4. Attention weights use per-channel FP8 and activations use dynamic per-token FP8. K/V scales are calibrated FP8 scalars. Original tied BF16 embedding/output-head and media projections are retained.
 Calibration used 128 English C4 training rows, seed 91027, maximum 1,024 tokens, a 512-row shuffle buffer, and dataset revision `607bd4c8450a42878aa9ddc051a65a055450ef87`. Benchmark/quality prompts and observed answers were excluded from calibration. This was post-training quantization; no new alignment or refusal-removal training was performed. The original Trial62 modification method is not documented in its source repository and is not inferred here.
 The converter handles Gemma's heterogeneous attention dimensions per layer: 40 sliding layers with eight KV heads and eight full layers with one KV head. Gate/up projection observers share their global statistics before packing. All 48 KV initializers, 144 packed MLP weights, and 48 shared gate/up scale pairs passed the export audits. These checks establish export integrity, not broad quality equivalence.
-See [MODIFICATIONS.md](MODIFICATIONS.md), [provenance.json](provenance.json), and [SHA256SUMS](SHA256SUMS).
 ## Credits and attribution
@@ -135,7 +137,7 @@ Media-related components and native protocols are retained, but this release did
 ## License, notices and citation
-The model derives from Google's Apache-2.0 Gemma 4 release. This repository includes [LICENSE](LICENSE), [NOTICE](NOTICE), and [MODIFICATIONS.md](MODIFICATIONS.md). Existing upstream SPDX/copyright headers are retained in the optional runtime source files. C4's separate ODC-BY license and Common Crawl terms apply to the calibration corpus; no corpus text is distributed here.
 Please credit the original Gemma authors, DevelopingDad's Trial62 checkpoint, this conversion, and any assistant/runtime/tooling used in your deployment. Google's original citation is:

 The source contains 11,959,730,224 stored BF16 parameter elements across 48 language layers. Its native configuration permits 262,144 total positions; **this release's tested serving recipe uses 32,768 total prompt-plus-output tokens**.
+The Hub's automatic parameter counter can count packed U8 storage elements rather than logical weights. Each NVFP4 byte stores two logical weights; the smaller automatic count does not mean this is an 8B architecture.
 Text MLP weights use importance-weighted MSE NVFP4, groups of 16, static FP8 block scales and global divisors. MLP activations use locally dynamic NVFP4. Attention weights use per-channel FP8 and activations use dynamic per-token FP8. K/V scales are calibrated FP8 scalars. Original tied BF16 embedding/output-head and media projections are retained.
 Calibration used 128 English C4 training rows, seed 91027, maximum 1,024 tokens, a 512-row shuffle buffer, and dataset revision `607bd4c8450a42878aa9ddc051a65a055450ef87`. Benchmark/quality prompts and observed answers were excluded from calibration. This was post-training quantization; no new alignment or refusal-removal training was performed. The original Trial62 modification method is not documented in its source repository and is not inferred here.
 The converter handles Gemma's heterogeneous attention dimensions per layer: 40 sliding layers with eight KV heads and eight full layers with one KV head. Gate/up projection observers share their global statistics before packing. All 48 KV initializers, 144 packed MLP weights, and 48 shared gate/up scale pairs passed the export audits. These checks establish export integrity, not broad quality equivalence.
+See [MODIFICATIONS.md](MODIFICATIONS.md), [provenance.json](provenance.json), and [SHA256SUMS](https://huggingface.co/DevelopingDad/Gemma4-12B-it-Fast/blob/main/SHA256SUMS).
 ## Credits and attribution
 ## License, notices and citation
+The model derives from Google's Apache-2.0 Gemma 4 release. This repository includes [LICENSE](https://huggingface.co/DevelopingDad/Gemma4-12B-it-Fast/blob/main/LICENSE), [NOTICE](https://huggingface.co/DevelopingDad/Gemma4-12B-it-Fast/blob/main/NOTICE), and [MODIFICATIONS.md](MODIFICATIONS.md). Existing upstream SPDX/copyright headers are retained in the optional runtime source files. C4's separate ODC-BY license and Common Crawl terms apply to the calibration corpus; no corpus text is distributed here.
 Please credit the original Gemma authors, DevelopingDad's Trial62 checkpoint, this conversion, and any assistant/runtime/tooling used in your deployment. Google's original citation is:

SHA256SUMS CHANGED Viewed

@@ -2,7 +2,7 @@
 c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4  LICENSE
 0518adcfc7d7b9f5321fac1f13ebda74a616d6df4e61018404a1a5a26a391e3d  MODIFICATIONS.md
 8340e3a16a607320a17af5aafc54a55f2f525cad61275f2e32fc2104f42f51a2  NOTICE
-44b641e4243f1b382e218b4696df5095935425acf288cbdeb3293f8c18bcb398  README.md
 a7a3fcaac51cc49a86c5103acd1e9c3b639fd8555f1f947d37a7f32e4d608f08  benchmarks/README.md
 0cdbc85335f31d11e7754de8158408eda6fb6cf7d61010af7e8cfad91db93b2f  benchmarks/results.json
 37ee3e82f78e1fcf6254d455d146f6cdb5454d2f90228e38a57f93aeb287619d  benchmarks/workload.json
@@ -12,7 +12,7 @@ f0f7ca9ea682738696f3960e39c427db6e3a4ff315265b13f4799125884e1dee  generation_con
 63536b8ec8456dddf8bbb7a71148bf2ab8499394c60226de2039e8846903a04b  model.safetensors
 6b938e76555b3e9946890770e1abcd442a4718f34041a58e8139dc8ad34545c9  processor_config.json
 56a8815db5692430ba558a90df8c35f0a277436ff4054cebdf0a657b2c75f245  provenance.json
-5a950a1be9ca98ba25ab07a780e703394b708644b050a106d2d2351ca7c70ce7  serving/README.md
 f09777985cdbbf19d137cfffe58979c486caf255321f6b2d791370955c04e11f  serving/artifact-manifest.json
 f9218019c51ea808fa4fe42ec362ea2b41f1300b662de14c1fb1989624a15005  serving/build_profile_template.py
 1432bc7f4c39852ade1f5e4fcc3e73c6eaef440d88b3983a784d4eb39ac27ed1  serving/draft_head_fp8_overlay.py

 c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4  LICENSE
 0518adcfc7d7b9f5321fac1f13ebda74a616d6df4e61018404a1a5a26a391e3d  MODIFICATIONS.md
 8340e3a16a607320a17af5aafc54a55f2f525cad61275f2e32fc2104f42f51a2  NOTICE
+2d6f213d266a9a0ca2f7717ac54db8609d00f018538b74f9053bd43985ea1221  README.md
 a7a3fcaac51cc49a86c5103acd1e9c3b639fd8555f1f947d37a7f32e4d608f08  benchmarks/README.md
 0cdbc85335f31d11e7754de8158408eda6fb6cf7d61010af7e8cfad91db93b2f  benchmarks/results.json
 37ee3e82f78e1fcf6254d455d146f6cdb5454d2f90228e38a57f93aeb287619d  benchmarks/workload.json
 63536b8ec8456dddf8bbb7a71148bf2ab8499394c60226de2039e8846903a04b  model.safetensors
 6b938e76555b3e9946890770e1abcd442a4718f34041a58e8139dc8ad34545c9  processor_config.json
 56a8815db5692430ba558a90df8c35f0a277436ff4054cebdf0a657b2c75f245  provenance.json
+6c4ce8f6936c3a3ff1d0d6de8543957ff0f4a0b8ae669f633b91e31417e3f6b6  serving/README.md
 f09777985cdbbf19d137cfffe58979c486caf255321f6b2d791370955c04e11f  serving/artifact-manifest.json
 f9218019c51ea808fa4fe42ec362ea2b41f1300b662de14c1fb1989624a15005  serving/build_profile_template.py
 1432bc7f4c39852ade1f5e4fcc3e73c6eaef440d88b3983a784d4eb39ac27ed1  serving/draft_head_fp8_overlay.py

serving/README.md CHANGED Viewed

@@ -76,6 +76,8 @@ reported rates. Local deployment orchestration and private endpoints are not
 part of this public package.
 See overlay-manifest.json for upstream-original, tested-source and public-file
-hashes. See ../LICENSE, ../NOTICE and ../MODIFICATIONS.md for attribution.
 The sanitized successful runtime check is in startup-integrity.json; artifact-manifest.json
 binds the exact target files and separately downloaded assistant to their hashes.

 part of this public package.
 See overlay-manifest.json for upstream-original, tested-source and public-file
+hashes. See [LICENSE](https://huggingface.co/DevelopingDad/Gemma4-12B-it-Fast/blob/main/LICENSE),
+[NOTICE](https://huggingface.co/DevelopingDad/Gemma4-12B-it-Fast/blob/main/NOTICE)
+and [MODIFICATIONS.md](../MODIFICATIONS.md) for attribution.
 The sanitized successful runtime check is in startup-integrity.json; artifact-manifest.json
 binds the exact target files and separately downloaded assistant to their hashes.