File size: 1,724 Bytes
00f1d2a | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 | ---
license: apache-2.0
base_model: unsloth/Qwen3-30B-A3B-Instruct-2507
tags:
- qwen3
- chat
- instruct
---
# OpenPipe/Qwen3-30B-A3B-Instruct-2507
This is a copy of [unsloth/Qwen3-30B-A3B-Instruct-2507](https://huggingface.co/unsloth/Qwen3-30B-A3B-Instruct-2507) with a **fixed chat template** for SFT training.
## Changes from the original
The only change is the chat template. The original Qwen3 chat template adds `<think></think>` tags **inconsistently** - only to the last assistant message in a conversation, not to earlier assistant messages in the history.
This causes issues during SFT training because:
- Historical assistant messages: `<|im_start|>assistant\nHi there!<|im_end|>`
- Target assistant message: `<|im_start|>assistant\n<think>\n\n</think>\n\nI am well!<|im_end|>`
The model learns an inconsistent pattern, leading to degraded outputs.
## Fixed template
This model uses a **consistent** chat template that adds `<think>\n\n</think>\n\n` to ALL assistant messages:
```
{{- '<|im_start|>assistant\n<think>\n\n</think>\n\n' + content }}
```
This ensures the model learns a single, consistent pattern during training.
## Usage
Use this model as your base model for SFT training with Qwen3-30B-A3B when you need consistent chat formatting.
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("OpenPipe/Qwen3-30B-A3B-Instruct-2507")
model = AutoModelForCausalLM.from_pretrained("OpenPipe/Qwen3-30B-A3B-Instruct-2507")
```
## Model Details
- **Base Model**: [unsloth/Qwen3-30B-A3B-Instruct-2507](https://huggingface.co/unsloth/Qwen3-30B-A3B-Instruct-2507)
- **Parameters**: 30B (3B active with MoE)
- **License**: Apache 2.0
|