COILD: An Indic-Centric Parallel Corpus and Benchmark for Machine Translation Across Indian Languages
Paper • 2609.28826 • Published
How to use COIL-D/translate-it2-hindi-dogri with Transformers:
# Use a pipeline as a high-level helper
# Warning: Pipeline type "translation" is no longer supported in transformers v5.
# You must load the model directly (see below) or downgrade to v4.x with:
# pip install "transformers<5.0.0"
from transformers import pipeline
pipe = pipeline("translation", model="COIL-D/translate-it2-hindi-dogri", trust_remote_code=True) # pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForSeq2SeqLM
model = AutoModelForSeq2SeqLM.from_pretrained("COIL-D/translate-it2-hindi-dogri", trust_remote_code=True, device_map="auto")Bidirectional Hindi ↔ Dogri translation model, fine-tuned from ai4bharat/indictrans2-indic-indic-dist-320M (IndicTrans2, 320M parameters).
| Directions | Hindi → Dogri and Dogri → Hindi |
| Language codes | Hindi hin_Deva, Dogri doi_Deva |
| Format | Hugging Face Transformers (model.safetensors, float32) |
The model uses IndicTrans2's custom code, so trust_remote_code=True is required.
pip install torch transformers IndicTransToolkit
import torch
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
from IndicTransToolkit.processor import IndicProcessor
repo = "COIL-D/translate-it2-hindi-dogri"
tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForSeq2SeqLM.from_pretrained(repo, trust_remote_code=True).eval()
ip = IndicProcessor(inference=True)
def translate(sentences, src_lang, tgt_lang):
batch = ip.preprocess_batch(sentences, src_lang=src_lang, tgt_lang=tgt_lang)
inputs = tokenizer(batch, padding="longest", truncation=True, max_length=256, return_tensors="pt")
with torch.no_grad():
generated = model.generate(**inputs, num_beams=5, max_length=256)
outputs = tokenizer.batch_decode(generated, skip_special_tokens=True)
return ip.postprocess_batch(outputs, lang=tgt_lang)
# Hindi -> Dogri
print(translate(["मुझे आज बाज़ार जाना है और कुछ सब्ज़ियाँ खरीदनी हैं।"], "hin_Deva", "doi_Deva"))
# Dogri -> Hindi
print(translate(["your Dogri input text here"], "doi_Deva", "hin_Deva"))
If you use this model, please cite:
@misc{singh2026coildindiccentricparallelcorpus,
title={COILD: An Indic-Centric Parallel Corpus and Benchmark for Machine Translation Across Indian Languages},
author={Kshetrimayum Boynao Singh and Nitin Kumar Mishra and Palash Pratim Dutta and Atai Waris Khan and Aparna Kaushik and Avinash Kumar and Deeksha and Deepak Kumar and Saroj Kumar Jha and Saloka Sengupta and Anansa Roy and Umalatha Kannoth and Saifulla Samar and Meena Sharma and Manpreet Kaur and Jyoti Sharma and Ashwini Vaidya and Muralikrishna SN and Md Shad Akhtar and Poonam Bansal and Amita Dev and Sanasam Ranbir Singh and Samit Bhattacharya and Tanmoy Chakraborty and Asif Ekbal},
year={2026},
eprint={2609.28826},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2609.28826},
}
Base model
ai4bharat/indictrans2-indic-indic-dist-320M