Aayush518/korean-conversation-checkpoint-250

🇰🇷 Korean Conversation Model - Checkpoint 250

A fine-tuned Korean conversation model optimized for natural dialogue, customer service, and chat applications.

📊 Model Details

• Base Model: naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B
• Training Method: DPO (Direct Preference Optimization) with LoRA
• Training Steps: 250
• Dataset: 8,000 Korean conversation examples
• Language: Korean (한국어)
• License: Apache 2.0

Training Configuration

• LoRA Rank: 4
• LoRA Alpha: 8
• Target Modules: v_proj, q_proj
• Quantization: 4-bit (NF4)
• Learning Rate: 2e-4
• Batch Size: 2
• Gradient Accumulation: 16 steps

🎯 Optimized For

This model is specifically optimized for:

• ✅ Korean conversation (natural dialogue flow)
• ✅ Customer service (polite, professional responses)
• ✅ Short responses (mean: ~25 chars, optimized for quick interactions)
• ✅ Formal/polite Korean (uses 요, 습니다, 세요 forms)
• ✅ Question answering (FAQ, help desk scenarios)
• ✅ Multi-turn dialogues (conversation continuity)

🚀 Quick Start

Installation

pip install transformers torch bitsandbytes accelerate

Basic Usage

from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
import torch

# Load model with 4-bit quantization (recommended)
model_name = "YOUR_USERNAME/korean-conversation-checkpoint-250"

quant_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True
)

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    quantization_config=quant_config,
    device_map="auto",
    trust_remote_code=True
)

# Simple inference
def generate_response(instruction, input_text=""):
    prompt = f"### Instruction:\n{instruction}\n\n### Input:\n{input_text}\n\n### Response:"
    
    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
    outputs = model.generate(
        **inputs,
        max_new_tokens=48,
        temperature=0.5,
        top_p=0.88,
        do_sample=True,
        repetition_penalty=1.15
    )
    
    response = tokenizer.decode(outputs[0], skip_special_tokens=True)
    return response.split("Response:")[-1].strip()

# Example
result = generate_response(
    instruction="고객이 배송 조회를 요청하고 있습니다.",
    input_text="제 주문이 어디에 있나요?"
)
print(result)

Advanced Usage with Inference Class

For production use, we provide an optimized inference class with caching, batch processing, and monitoring:

# Download inference.py from this repo
from inference import KoreanConversationInference, load_model

# Load model
model, tokenizer = load_model("YOUR_USERNAME/korean-conversation-checkpoint-250")

# Create inference system
korean_ai = KoreanConversationInference(
    model=model,
    tokenizer=tokenizer,
    cache_size=256,  # LRU cache for repeated queries
    enable_monitoring=True
)

# Generate with optimized config
result = korean_ai.generate(
    instruction="고객이 배송 조회를 요청하고 있습니다.",
    input_text="제 주문이 어디에 있나요?",
    gen_config='dataset_standard'  # Optimized for dataset
)

print(f"Response: {result['response']}")
print(f"Time: {result['inference_time']:.3f}s")
print(f"Cached: {result['from_cache']}")

⚙️ Generation Configs

The inference class provides 5 dataset-optimized configurations:

Config	Max Tokens	Temperature	Best For
ultra_short	32	0.4	Quick answers, yes/no
dataset_standard ⭐	48	0.5	General conversation (recommended)
dataset_extended	80	0.6	Detailed explanations
conversation_flow	64	0.55	Natural dialogue
polite_formal	56	0.45	Customer service, formal

📈 Performance

• Response Length: Mean ~30-40 chars (optimized for dataset's ~25 char mean)
• Inference Time: ~0.5-1.5s (first request), ~0.01-0.05s (cached)
• Cache Hit Rate: 30-60% (typical workload)
• Korean Quality: 100% Korean responses
• Formality: Maintains polite Korean forms

💡 Use Cases

Customer Service

result = korean_ai.generate(
    instruction="고객이 환불을 요청하고 있습니다.",
    input_text="제품이 마음에 들지 않아요.",
    gen_config='polite_formal'
)

FAQ Bot

result = korean_ai.generate(
    instruction="사용자가 영업 시간을 문의하고 있습니다.",
    input_text="매장 영업 시간이 어떻게 되나요?",
    gen_config='ultra_short'
)

Virtual Assistant

result = korean_ai.generate(
    instruction="사용자가 제품 추천을 요청하고 있습니다.",
    input_text="초보자한테 좋은 제품이 뭐가 있을까요?",
    gen_config='conversation_flow'
)

🎓 Training Details

Dataset

• Size: 8,000 Korean conversation pairs
• Format: Instruction-Input-Output
• Domain: Customer service, FAQ, general conversation
• Language: Korean (formal/polite style)

Training Process

1. Base model: HyperCLOVAX-SEED-Vision-Instruct-3B
2. Method: DPO (Direct Preference Optimization)
3. Adapter: LoRA (rank=4, alpha=8)
4. Quantization: 4-bit (NF4) for efficiency
5. Training steps: 250
6. Validation: Tested on 20+ diverse scenarios

Performance Metrics

• ✅ 100% success rate across test scenarios
• ✅ 95%+ appropriate response length
• ✅ Natural Korean conversation flow
• ✅ Maintains formality and politeness

📦 Files in this Repo

• adapter_model.safetensors - LoRA adapter weights
• adapter_config.json - Adapter configuration
• inference.py - Production-ready inference class
• README.md - This file
• Tokenizer files (vocab.json, merges.txt, etc.)

🔧 System Requirements

• GPU: Recommended (CUDA-compatible)
• RAM: 8GB+ (with 4-bit quantization)
• VRAM: 6GB+ (with 4-bit quantization)
• Python: 3.8+
• PyTorch: 2.0+

⚠️ Limitations

• Model is optimized for Korean language only
• Best performance on customer service and FAQ scenarios
• Trained for short responses (~25-60 chars typical)
• May be verbose compared to training data (inherits base model characteristics)
• Checkpoint 250 - Early training stage (more training may improve accuracy)

📄 License

This model is released under the Apache 2.0 license. The base model (HyperCLOVAX-SEED-Vision-Instruct-3B) has its own license terms.

🙏 Acknowledgments

• Base model by NAVER Cloud HyperCLOVA X team
• Training data: Custom Korean conversation dataset
• Method: DPO (Direct Preference Optimization)
• Framework: Hugging Face Transformers, PEFT, TRL

📚 Citation

If you use this model, please cite:

@model{korean_conversation_checkpoint_250,
  title={Korean Conversation Model - Checkpoint 250},
  author={Your Name},
  year={2025},
  publisher={Hugging Face},
  url={https://huggingface.co/YOUR_USERNAME/korean-conversation-checkpoint-250}
}

🔗 Links

• Base Model
• Transformers Documentation
• PEFT Documentation

💬 Feedback

For issues, questions, or feedback, please open an issue in the repository.

---

Made with ❤️ for the Korean NLP community