metadata
license: apache-2.0
base_model: unsloth/Qwen3-30B-A3B-Instruct-2507
tags:
- qwen3
- chat
- instruct
OpenPipe/Qwen3-30B-A3B-Instruct-2507
This is a copy of unsloth/Qwen3-30B-A3B-Instruct-2507 with a fixed chat template for SFT training.
Changes from the original
The only change is the chat template. The original Qwen3 chat template adds <think></think> tags inconsistently - only to the last assistant message in a conversation, not to earlier assistant messages in the history.
This causes issues during SFT training because:
- Historical assistant messages:
<|im_start|>assistant\nHi there!<|im_end|> - Target assistant message:
<|im_start|>assistant\n<think>\n\n</think>\n\nI am well!<|im_end|>
The model learns an inconsistent pattern, leading to degraded outputs.
Fixed template
This model uses a consistent chat template that adds <think>\n\n</think>\n\n to ALL assistant messages:
{{- '<|im_start|>assistant\n<think>\n\n</think>\n\n' + content }}
This ensures the model learns a single, consistent pattern during training.
Usage
Use this model as your base model for SFT training with Qwen3-30B-A3B when you need consistent chat formatting.
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("OpenPipe/Qwen3-30B-A3B-Instruct-2507")
model = AutoModelForCausalLM.from_pretrained("OpenPipe/Qwen3-30B-A3B-Instruct-2507")
Model Details
- Base Model: unsloth/Qwen3-30B-A3B-Instruct-2507
- Parameters: 30B (3B active with MoE)
- License: Apache 2.0