Gemma-4 E2B β€” Creole MT Baseline Adapter

LoRA adapter fine-tuned on Kreyol-MT (Robinson et al., NAACL 2024), covering 86 language pairs across 50 Creole and related languages.

Results (dev set, 10 epochs, 8Γ— L40S)

Metric Base Gemma-4 + This adapter Ξ”
Macro BLEU 7.19 18.48 +11.29
Macro chrF 20.90 36.94 +16.04
Corpus BLEU 5.57 18.48 +12.91
Corpus chrF 21.82 38.48 +16.66

Training details

  • Base model: google/gemma-4-E2B
  • Method: LoRA (r=64, Ξ±=128, target=all-linear)
  • Training data: Kreyol-MT public split β€” 86 language pairs, temperature-sampled (T=5, NLLB-style) to 50k examples per epoch
  • Epochs: 10
  • Precision: bf16
  • Hardware: 8Γ— NVIDIA L40S

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

model_id = "google/gemma-4-E2B"
adapter_id = "pranavgupta/gemma4-creole-mt-2026-baseline-adapter"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16).cuda()
model = PeftModel.from_pretrained(model, adapter_id)
model = model.merge_and_unload()

prompt = "Translate this from Haitian Creole into English:\nBonjou, kijan ou rele?\n### Translation:\n"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
out = model.generate(**inputs, max_new_tokens=100, do_sample=False)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Framework versions

  • PEFT 0.19.1
Downloads last month
9
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for pranavgupta/gemma4-creole-mt-2026-baseline-adapter

Adapter
(53)
this model