Instructions to use mncai/hunmin-vlm-0.12 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mncai/hunmin-vlm-0.12 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="mncai/hunmin-vlm-0.12")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("mncai/hunmin-vlm-0.12", dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use mncai/hunmin-vlm-0.12 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "mncai/hunmin-vlm-0.12"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mncai/hunmin-vlm-0.12",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/mncai/hunmin-vlm-0.12

SGLang

How to use mncai/hunmin-vlm-0.12 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "mncai/hunmin-vlm-0.12" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mncai/hunmin-vlm-0.12",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "mncai/hunmin-vlm-0.12" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mncai/hunmin-vlm-0.12",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use mncai/hunmin-vlm-0.12 with Docker Model Runner:
```
docker model run hf.co/mncai/hunmin-vlm-0.12
```

Hunmin-VLM 0.12

A Vision-Language Model built on GLM-4V-9B with a custom projector and LoRA fine-tuning using ORPO (Odds Ratio Preference Optimization).

Model Details

Base Model: THUDM/glm-4v-9b (GLM-4.7-Flash)
Training Method: 4-stage progressive training (projector → LoRA SFT → LoRA ORPO)
Image Tokens: 288 per image
Hardware: 1 node × 8 NVIDIA B200 GPUs
Framework: PyTorch + HuggingFace Transformers ≥5.0.0rc0

Training Stages

This model was trained in 4 progressive stages. Each stage is available as a separate branch.

Branch	Stage	Description	GPUs	Steps	LR
`stage0`	0	Projector warmup (random init)	1	200	3e-4
`stage1`	1	Projector training (init from stage0)	1	1,000	1e-4
`stage2`	2	LoRA SFT, projector frozen	8	500	1e-5
`main`	3	LoRA ORPO, projector frozen, early stopping	8	500	5e-6

Stage Details

Stage 0–1 (Projector): Only the vision-language projector is trained. The LLM backbone and vision encoder are frozen. Output: projector_state.pt (~17MB).
Stage 2 (LoRA SFT): The projector is frozen. LoRA adapters are applied to the LLM for supervised fine-tuning on VLM conversation data. The full merged model is saved (~76GB).
Stage 3 (LoRA ORPO): Same as stage 2, but trained with ORPO preference optimization. Text ORPO pairs are mixed in at 10% ratio. Evaluation every 100 steps with early stopping (patience=5).

Training Data

VLM pairs: mncai/orpo-vlm-pairs-full — multimodal preference pairs
Text pairs: mncai/orpo-text-pairs-full — text-only preference pairs (10% mix in stage 3)

Training Logs (W&B)

Full training metrics are publicly available:

Project: ttagu99/hunmin-vlm-0.12

Run	Stage
`glm47-mm-projector-1node-20260201-083651-stage0`	Projector warmup
`glm47-mm-projector-1node-20260201-083651-stage1`	Projector training
`glm47-mm-projector-1node-20260201-083651-stage2`	LoRA SFT
`glm47-mm-projector-1node-20260201-083651-stage3`	LoRA ORPO (final)

Branch Usage

# Final model (stage3 ORPO)
git clone https://huggingface.co/mncai/hunmin-vlm-0.12

# Specific stage
git clone -b stage2 https://huggingface.co/mncai/hunmin-vlm-0.12

Or with Python:

from huggingface_hub import snapshot_download

# Final model
snapshot_download("mncai/hunmin-vlm-0.12")

# Specific stage
snapshot_download("mncai/hunmin-vlm-0.12", revision="stage1")

Included Training Files

File	Description
`training/train_glm47_mm.py`	Main training script (all 4 stages)
`training/trainjob-glm47-mm-projector-1node.yaml`	Kubeflow TrainJob YAML
`training/smoke_glm47_mm.py`	Smoke test script

License

Please contact mncai for licensing information.

Downloads last month: -; Downloads are not tracked for this model. How to track

Safetensors

Model size

40B params

Tensor type

F32

BF16

Model tree for mncai/hunmin-vlm-0.12

Base model

zai-org/glm-4v-9b

Adapter

(1)

this model