Boulifa-48M

A 47.8M-parameter character-level Conv-Transformer sequence-to-sequence model for Kabyle (Taqbaylit, kab), trained from scratch to standardize informal, French-keyboard, and Arabizi Kabyle text into canonical Kabyle Latin orthography.

In informal digital communication, Kabyle is commonly written without specialized orthographic characters, using French digraphs (gh, kh, ch), numeral substitutions (3, 7, 5), and unhyphenated clitics. Boulifa-48M resolves these ambiguities simultaneously, reaching 97.39% character accuracy and 85.70% sentence exact match on held-out test pairs under free-running greedy decoding (a 75% reduction in character error rate compared to leaving informal input unmodified).

The Task

Boulifa-48M addresses common informal typing patterns in Kabyle digital communication:

Phenomenon	Example Input	Canonical Output
French digraph gh for ɣ	ighef	iɣef
French digraph kh for x	khedmekh	xedmex
French digraph ch for c	achimi	acimi
Arabizi digit 7 for ḥ	l7adj	lḥadj
Arabizi digit 3 for ɛ or ɣ	3emmi, gh3ir	ɛemmi, ɣir
Arabizi digit 5 for x	5ir	xir
French ou digraph for u	touddart	taddart
Emphatic drop dh for ḍ	thekhedmedh	tḥexedmeḍ
Emphatic drop th for ṭ	thefsouth	tḥefsuṭ
Clitic hyphen omission	dyeffegh	d-yeffeɣ
Preposition contraction	g taddart	deg taddart
Identity (already canonical)	Azul fell-awen	Azul fell-awen

Because substitutions carry contextual ambiguity (e.g. 3 mapping to either ɛ or ɣ depending on morphological context, or th representing an emphatic consonant cluster vs. t + h), standardizing text requires sequence transduction rather than static lookup tables.

Results

Evaluated on 1,000 held-out sentence pairs from the agbalu/KabStandard test split using free-running greedy decoding (num_beams=1):

System	Character Accuracy	Character Error Rate (CER)	Sentence Exact Match
Boulifa-48M	97.39%	2.61%	85.70%
Identity baseline (unmodified input)	89.70%	10.30%	—

• Error Reduction: Achieves a 75% reduction in character error rate over the baseline.
• Error Distribution: Sentence exact match is 85.70%, showing that residual character errors (2.61% CER) are concentrated in the remaining 14.30% of sentences.
• Evaluation Mode: Evaluated under autoregressive free-running generation without teacher forcing.

Qualitative Examples

Outputs generated by Boulifa-48M:

Input:  "achimi ur d-thekhedmedh ara tamazight g l'ecole?"
Output: "acimi ur d-tḥexedmeḍ ara tamaziɣt deg lɛecule?"

Input:  "3emmi l7adj yerza-d 5ir d lbaraka s wuzzal"
Output: "Ɛemmi lḥadj yerza-d xir d lbaraka s wuzzal"

Input:  "thessawledh-d fellanegh zik g thefsouth"
Output: "tḥessawleḍ-d fell-aneɣ zik deg tḥefsuṭ"

Input:  "yennayasen ur ten-idttakken ara degs"
Output: "yennayasen ur ten-id-ttakken ara degs"

Input:  "Azul fell-awen, amek i telliḍ taṣebḥit-a?"
Output: "Azul fell-awen, amek i telliḍ taṣebḥit-a?"

Inputs that are already canonical Latin are preserved without unwanted alteration.

Intended Use

• Input Normalization: Preprocessing informal, social-media, or SMS Kabyle text prior to parsing, semantic retrieval, or downstream language modeling.
• Corpus Curation: Standardizing user-contributed text into canonical orthography for research and training data preparation.
• Out of Scope: Machine translation into non-Berber languages; processing non-Kabyle languages.

Usage

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

REPO = "agbalu/Boulifa-48M"
tokenizer = AutoTokenizer.from_pretrained(REPO, trust_remote_code=True)
model = AutoModelForSeq2SeqLM.from_pretrained(REPO, trust_remote_code=True).eval()

def standardise(text):
    encoded = tokenizer(text, return_tensors="pt")
    generated = model.generate(**encoded, max_length=512, num_beams=1)
    return tokenizer.decode(generated[0], skip_special_tokens=True)

print(standardise("achimi ur d-thekhedmedh ara tamazight g l'ecole?"))
# Output: 'acimi ur d-tḥexedmeḍ ara tamaziɣt deg lɛecule?'

print(standardise("3emmi l7adj yerza-d 5ir d lbaraka s wuzzal"))
# Output: 'Ɛemmi lḥadj yerza-d xir d lbaraka s wuzzal'

print(standardise("Azul fell-awen, amek i telliḍ taṣebḥit-a?"))
# Output: 'Azul fell-awen, amek i telliḍ taṣebḥit-a?'

Architecture

Parameter	Value
Architecture	Character-level Conv-Transformer (BoulifaForSeq2SeqLM)
Total Parameters	47,797,760
Encoder / Decoder Layers	6 layers / 6 layers
Hidden / Feed-Forward Dim	512 / 1,536 (SwiGLU)
Attention Heads	8 heads (head dimension 64)
Positional Embeddings	Rotary Position Embeddings (RoPE) on self-attention
Convolutional Stem	1D depthwise-separable convolution (kernels 3 and 5)
Vocabulary	128 character slots (96 active characters; tied input/output embeddings)
Label Smoothing	$\epsilon = 0.05$

Parameter Breakdown:

• Decoder layers (6×): 26,747,904
• Encoder layers (6×): 20,453,376
• Convolutional stem: 529,920
• Tied embeddings: 65,536
• Layer norms: 1,024

The convolutional stem processes character n-grams before self-attention, capturing common digraph patterns (kh, gh, ch, dj, ou) prior to sequence-level contextualization.

Training Data

Trained on agbalu/KabStandard (497,944 parallel pairs derived from canonical Kabyle text):

• 15% Identity Pairs: Canonical text mapped to itself, ensuring stability on already-standardized input.
• 85% Corrupted Pairs: Synthesized via stochastic sampling across realistic keyboard and Arabizi substitutions:

Corruption Type	Canonical	Informal Replacement	Probability
ɣ digraph	ɣ	gh	0.75
x digraph	x	kh	0.85
c digraph	c	ch	0.75
ğ digraph	ğ	dj	0.80
ḍ emphatic	ḍ	dh	0.75
ṭ emphatic	ṭ	th	0.70
ḥ pharyngeal	ḥ	7	0.25
ɣ Arabizi	ɣ	3	0.08
ɛ pharyngeal	ɛ	3	0.25
x Arabizi	x	5	0.05
u vowel	u	ou	0.45
Clitic hyphen omission	-	or ``	0.50
Preposition contraction	deg	g	0.25

Split Sizes:

• Train: 448,149 pairs
• Validation: 24,897 pairs
• Test: 24,898 pairs

All source targets were normalized using reference normalizer v1.3.0.

Training Recipe

Parameter	Value
Objective	Character cross-entropy with label smoothing ($\epsilon = 0.05$)
Optimizer	AdamW (peak lr 5e-4, $\beta=(0.9, 0.98)$, weight decay 0.01)
Batch Size	64 sequences per step
Gradient Clipping	Maximum norm 1.0
Epochs	2 epochs (converged on validation character accuracy)
Checkpoint	Epoch 2 (boulifa_best.pt, validation character accuracy 99.45%)
Seed	42

Limitations

• Synthetic Training Distribution: While modeled on observed digital typing patterns, real social media text contains idiosyncratic spellings, dialectal vocabulary, and French-Kabyle code-switching outside the corruption distribution.
• Ambiguity in Arabizi Numerals: The numeral 3 represents both ɛ and ɣ. While disambiguated through morphological context, atypical vocabulary may lead to misclassifications.
• Clitic Chains: Long unsegmented clitic sequences remain the most frequent source of sentence-level discrepancy.
• Dialectal Focus: Calibrated for Kabyle Latin; not evaluated on Tashelhit or Central Atlas Tamazight conventions.

Files

File	Size	Description
model.safetensors	~191.3 MB	Model weights (47,797,760 parameters)
config.json	< 5 KB	Architecture configuration and auto_map
configuration_boulifa.py, modeling_boulifa.py	< 30 KB	Transformers modeling and configuration code
tokenizer.json, tokenizer_config.json	< 10 KB	96-character alphabet tokenizer

Reproduction

Trained for 2 epochs on the agbalu/KabStandard training split with AdamW (lr 5e-4, batch size 64). Evaluated with greedy decoding (num_beams=1) over 1,000 sentences from the test split, comparing character error rate and exact match against the identity baseline. All metrics reproduce deterministically from the published checkpoint.

The name

Si Amar ou Saïd Boulifa (1865–1931) was the first grammarian to systematically codify Kabyle Latin orthography. His Recueil de poésies kabyles (1904) set 555 pages of Kabyle in one consistent Latin notation — solving, in the print typography of its day, precisely the mapping problem this model re-solves in the age of keyboard input.

He worked without a committee, without a standard, and against a tradition that had not yet decided whether Kabyle could be written at all. The orthographic rules he established were carried forward through Mammeri, Dallet, and into the INALCO standard that this model was trained to produce.

The naming is homage; it implies no endorsement by anyone.

Citation

@software{agbalu_boulifa_2026,
  title  = {Boulifa-48M: Orthography Standardisation for Informal Kabyle Latin},
  author = {AƔBALU},
  year   = {2026},
  url    = {https://huggingface.co/agbalu/Boulifa-48M},
  note   = {Trained on agbalu/KabStandard; 47,797,760 parameters; 97.39% free-running character accuracy}
}

Licence

• Weights & Code: Apache-2.0.
• Training Data: Derived from agbalu/KabTifinagh.