tencent

/

Hy-MT2-7B-GGUF

Like 80

Follow

Tencent 12.3k

Hello Hunyuan Team,
I am testing Hy-MT2-7B-GGUF (8_0) for structured subtitle translation using a batch-processing translation UI (via OpenAI-compatible API format). The model shows impressive linguistic quality and excellent prompt processing speed, but it struggles to maintain strict structural formatting under heavy context load (16K context window).
Steps to reproduce / Context:

• Inference: LM Studio (llama.cpp backend) with the official <|hy_begin_of_sentence|> chat template.
• Parameters: Temp 0.7 (also tested down to 0.2), Top_P 0.6, Top_K 20, Rep_Penalty 1.05.
• Input format: Batches of subtitle lines, where the model is instructed to wrap each translated line into indexed technical tags like [TRANSLATE_0]...[/TRANSLATE_0], [TRANSLATE_1]...[/TRANSLATE_1].

Observed behavior:
As the generation progresses, the model suffers from structural degradation:

1. Character/Layout corruption: It randomly mixes Cyrillic and Latin characters within the technical tags (e.g., generating Cyrillic Т instead of Latin T in [TRANSLATE]), which completely breaks downstream regex parsers.

2. Missing closing tags: It frequently skips or forgets to generate closing tags (e.g., [/TRANSLATE_X]) towards the end of a long batch response.

3. Token Leakage and Semantic Collapse via Streaming API (Subtitle Edit use-case)
   I have performed additional testing using Subtitle Edit (which utilizes a line-by-line streaming approach via OpenAI-compatible endpoint), and encountered a critical token-handling regression:

4. Stop Token Leakage: The model completely fails to intercept its own internal chat template boundaries. Instead of stopping the generation, it prints the raw formatting markers directly into the text output, with corrupted/broken special characters (e.g., outputs literal string like: ...как<｜hy_end▁of▁assistant｜>).

5. Semantic Collapse on Idioms: Under this workload, the translation quality severely degrades. The model loses sentence-level context and falls back to chaotic literalism, turning natural expressions into unreadable word-by-word direct translations (e.g., completely misinterpreting context-dependent clauses into broken syntax).

This confirms that the Hy-MT2-7B architecture is highly unstable when interfacing with standard LLM middleware and UI parsing wrappers, leading to immediate context degradation.

It seems the heavy fine-tuning for pure text-to-text translation makes the 7B model "resist" complex non-standard formatting constraints, causing it to lose tracking of XML-like block structures. Lowering the temperature to 0.2 reduces the frequency but does not completely eliminate this tag corruption issue.
Are there any specific prompting strategies or logit bias adjustments recommended to keep the structural tokens strict for the Hy-MT2 series?

Thank you for your hard work on this model!