CyberPuppy v6 — Pinyin LoRA (LoRA-B) | Strengthened Homophone Defense
拼音分支 v6 · Qwen3-8B + LoRA r=64 + 同音字攻擊強化防禦
Companion to v6-bilingual. Doubled rank from r=32 → r=64 for stronger phonetic pattern recognition.
What changed from v5
- LoRA rank: 32 → 64 (doubled)
- LoRA alpha: 64 → 128
- Epochs: 3 → 5 (best at epoch 2)
- Max length: 128 → 192
- Consistency loss: λ=0 → λ=0.5
Performance impact (in v6 dual-LoRA ensemble)
| Metric | v5.1 | v6 |
|---|---|---|
| Pinyin-only dev F1 | 0.7979 | 0.7983 (≈ same) |
| Ensemble HED-COLD | 0.9126 | 0.9317 (+1.91pt) |
| Ensemble TC homo | 0.8496 | 0.8510 (+0.14pt) |
The pinyin LoRA's standalone dev F1 plateaus around 0.80, but the r=64 capacity helps the ensemble especially on systematic homophone perturbations (HED-COLD).
Usage
⚠️ Must be paired with v6-bilingual text LoRA. See the companion repo for full ensemble code.
Training Details
| Parameter | Value |
|---|---|
| Base model | Qwen/Qwen3-8B-Base |
| LoRA rank | 64 |
| LoRA alpha | 128 |
| Training data | 179,186 samples (pinyin-converted v5 bilingual) |
| Epochs | 5 (best at epoch 2, step 9954) |
| Learning rate | 3e-5 |
| Max length | 192 |
| Precision | bf16 |
| Loss | Focal γ=2.5 + uncertainty + consistency λ=0.5 |
| Hardware | 1× NVIDIA RTX 5090 (32GB, 590W OC) |
License
CC BY-NC-SA 4.0.
Citation
See v6-bilingual.
Related
- thc1006/cyberpuppy-v6-bilingual — required companion text LoRA-A
- thc1006/cyberpuppy-v5-pinyin-lora — v5 version (r=32)
Contact
- Author: Hung-Che Tsai (hctsai1006@cs.nctu.edu.tw)
- Takedown: Email above — removed within 7 days
- Downloads last month
- -
Model tree for thc1006/cyberpuppy-v6-pinyin-lora
Base model
Qwen/Qwen3-8B-BaseEvaluation results
- F1 (weighted, pinyin-only dev) on COLDtest set self-reported0.798