WhisperLv3-EP-X: Whisper large-v3 fine-tuned for European Portuguese (WhisperX) — pré-Acordo

Whisper large-v3 fine-tuned on 425 h of European Portuguese (EP-425), packaged for WhisperX (CTranslate2, VAD-based segmentation). This is the best EP model in the paper and the recommended one for long-form audio.

From the paper CAMÕES: A Comprehensive Automatic Speech Recognition Benchmark for European Portuguese (INESC-ID / Instituto Superior Técnico). In the paper, this model is Table III → FT → WhisperLv3-X, trained on EP-425.

Results on the CAMÕES benchmark (WER %, from Table III)

Model	RS	BN	T/L	CS	SI	EP Avg.	AAP	BP
This model (WhisperLv3-X, fine-tuned)	7.4	4.7	11.3	11.2	27.9	12.5	24.0	27.2
Baseline: original Whisper large-v3, no fine-tuning	32.4	7.9	15.4	18.3	49.0	24.6	29.9	25.8
Baseline: original Whisper large-v3 + WhisperX, no fine-tuning	16.4	8.2	16.6	15.3	39.3	19.2	29.0	24.6

The baselines are the public openai/whisper-large-v3 used as released (zero-shot rows of Table III), with plain Whisper decoding and with WhisperX (voice-activity detection before transcription), respectively.

EP domains: RS = read speech, BN = broadcast news, T/L = talks/lectures, CS = conversational speech, SI = sociolinguistic interviews. EP Avg. gives each domain equal weight. AAP = African and Asian Portuguese, BP = Brazilian Portuguese. All results use the CAMÕES text normaliser (lower-case, no punctuation, numbers spelled out). Leaderboard and test sets: inesc-id/camoes_asr.

Model details

• Base model: openai/whisper-large-v3 (1.55B parameters, all fine-tuned)
• Training data: EP-425, 425 h of European Portuguese from 14 corpora (Section III-A)
• Fine-tuning: 10 epochs, batch size 64, gradient accumulation 4, learning rate 1e-5, 10% warm-up (Section V)
• Decoding: WhisperX (voice-activity detection + batched faster-whisper), which reduces Whisper hallucinations
• Format: CTranslate2 (model.bin), float16 recommended

Usage

import whisperx

model = whisperx.load_model("inesc-id/WhisperLv3-EP-X", "cuda", compute_type="float16",
                            language="pt", task="transcribe")
audio = whisperx.load_audio("audio.wav")
result = model.transcribe(audio, batch_size=8, language="pt", task="transcribe")
print(" ".join(seg["text"] for seg in result["segments"]))

Output format

Pre-Acordo Ortográfico. All CAMÕES models are trained on transcripts written in the spelling from before the 1990 Orthographic Agreement (pré-Acordo), e.g. projecto, director, actual.

Transcriptions are normalised like the training transcripts: lower-case, no punctuation, and numbers written out in words.

Other CAMÕES models

Model	Paper row (Table III)	Training data	EP Avg.	BP
WhisperLv3-EP-X (this model)	WhisperLv3-X, fine-tuned	EP-425	12.5	27.2
WhisperLv3-FT	WhisperLv3, fine-tuned	EP-425	16.7	28.2
EBranch-w2vBERT2-EP	EBranch-w2vBERT2, trained from scratch	EP-425	14.4	42.4
WhisperLv3-X-PT-All	WhisperLv3-X, PT-All	EP-425 + BP-417 + AAP-8	13.0	18.8

Citation

@inproceedings{camoes,
  title     = {{CAMÕES: A Comprehensive Automatic Speech Recognition Benchmark for European Portuguese}},
  author    = {Carvalho, Carlos and Teixeira, Francisco and Botelho, Catarina and Pompili, Anna and
               Solera-Ure{\~n}a, Rub{\'e}n and Paulo, S{\'e}rgio and Juli{\~a}o, Mariana and Rolland, Thomas and
               Mendon{\c{c}}a, John and Pereira, Diogo and Trancoso, Isabel and Abad, Alberto},
  booktitle = {Proceedings of the IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)},
  year      = {2025}
}