IndicTrans2 Indic→English, int8, phone package

Offline Indic→English translation for on-device use with ONNX Runtime (Android or desktop). A quantised, repackaged copy of AI4Bharat's IndicTrans2 indic-en distilled 200M — not a new or retrained model. Made for Subtle, a live-subtitle app.

What was done

  1. Started from the community ONNX export hari31416/indictrans2-indic-en-dist-200M-ONNX (MIT).
  2. Dynamic int8 quantisation (per channel) of MatMul and Gather (embedding) weights: 915 MB → 333 MB, same quality, ~2× faster on CPU.
  3. Tokenizer flattened to plain text for fast loading on phones: src_vocab.txt (line = id), src_merges.txt (line = BPE rank), tgt_vocab.txt, plus it2.properties / it2_meta.json (dictionary sizes, special and language-tag ids).

Files

File Size
encoder_model.onnx 121 MB
decoder_model.onnx 111 MB (first decoding step)
decoder_with_past_model.onnx 101 MB (later steps, with KV cache)
src_vocab.txt, src_merges.txt, tgt_vocab.txt, it2.properties, it2_meta.json 6.5 MB

Inference

  1. Pre-process: punctuation normalisation, Devanagari digits → ASCII, nukta letters split (e.g. ड़ → ड + ़), spaces around punctuation, then prefix the language tags: hin_Deva eng_Latn <text> (Tamil: tam_Taml eng_Latn …).
  2. Tokenise: NFKC each space-separated piece, prepend ▁, apply BPE merges by rank, map to ids (unknown → <unk>, ids ≥ src_dict_size → <unk>), append </s> (id 2).
  3. Encoder → greedy decoding from decoder_start_id (2) until </s>: step 0 with decoder_model.onnx, later steps with decoder_with_past_model.onnx feeding each present.* output back as past_key_values.*.
  4. Join target pieces, ▁ → space, light English detokenisation.

A reference implementation (Python, numpy + onnxruntime only) and a Kotlin/JVM port live in the Subtle repository (tools/asr-bench/it2lite.py, it2/IndicTrans2.kt); the Kotlin port reproduces the Python output on 40/40 test sentences.

Evaluation

Hindi→English, FLEURS hi_in dev, scored against the human English translations (en_us), correct Hindi as input.

Translator chrF (50 sent.) Speed (laptop CPU)
Google Cloud Translation (online) 70.3 —
This package (int8, offline) 67.9 ~0.34 s/sentence, 2 threads
IndicTrans2 float (offline) 67.9 ~0.92 s/sentence
Google ML Kit (offline, on phone), 25 sentences 57.4 (vs 66.1 for IndicTrans2 on the same 25) —

Translations are literal: recognition errors in the Hindi input pass through, whereas Google Cloud often repairs them from context.

License and credits

  • Original model: AI4Bharat, IndicTrans2 — MIT license. Please cite their paper when using it: Gala et al., IndicTrans2: Towards High-Quality and Accessible Machine Translation Models for all 22 Scheduled Indian Languages, TMLR 2023.
  • ONNX export: hari31416/indictrans2-indic-en-dist-200M-ONNX — MIT.
  • This package: MIT.
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for ngiftsun/indictrans2-indic-en-int8-mobile

Quantized
(8)
this model

Dataset used to train ngiftsun/indictrans2-indic-en-int8-mobile