|
|
<!doctype html> |
|
|
<html lang="en"> |
|
|
<head> |
|
|
<meta charset="utf-8" /> |
|
|
<meta name="viewport" content="width=device-width" /> |
|
|
<title>IqraEval.2 Challenge Interspeech 2026</title> |
|
|
<style> |
|
|
:root { |
|
|
--navy-blue: #001f4d; |
|
|
--coral: #ff6f61; |
|
|
--light-gray: #f5f7fa; |
|
|
--text-dark: #222; |
|
|
} |
|
|
body { |
|
|
font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif; |
|
|
background-color: var(--light-gray); |
|
|
color: var(--text-dark); |
|
|
margin: 20px; |
|
|
line-height: 1.6; |
|
|
} |
|
|
h1, h2, h3 { |
|
|
color: var(--navy-blue); |
|
|
font-weight: 700; |
|
|
margin-top: 1.2em; |
|
|
} |
|
|
h1 { |
|
|
text-align: center; |
|
|
font-size: 2.8rem; |
|
|
margin-bottom: 0.3em; |
|
|
} |
|
|
h2 { |
|
|
border-bottom: 3px solid var(--coral); |
|
|
padding-bottom: 0.3em; |
|
|
} |
|
|
h3 { |
|
|
color: var(--coral); |
|
|
margin-top: 1em; |
|
|
} |
|
|
p, ul, pre, ol { |
|
|
max-width: 900px; |
|
|
margin: 0.8em auto; |
|
|
} |
|
|
ul, ol { padding-left: 1.2em; } |
|
|
ul li, ol li { margin: 0.4em 0; } |
|
|
code { |
|
|
background-color: #eef4f8; |
|
|
color: var(--navy-blue); |
|
|
padding: 2px 6px; |
|
|
border-radius: 4px; |
|
|
font-family: Consolas, monospace; |
|
|
font-size: 0.9em; |
|
|
} |
|
|
pre { |
|
|
background-color: #eef4f8; |
|
|
padding: 1em; |
|
|
border-radius: 8px; |
|
|
overflow-x: auto; |
|
|
font-size: 0.95em; |
|
|
} |
|
|
a { |
|
|
color: var(--coral); |
|
|
text-decoration: none; |
|
|
} |
|
|
a:hover { text-decoration: underline; } |
|
|
.card { |
|
|
max-width: 960px; |
|
|
background: white; |
|
|
margin: 0 auto 40px; |
|
|
padding: 2em 2.5em; |
|
|
box-shadow: 0 4px 14px rgba(0,0,0,0.1); |
|
|
border-radius: 12px; |
|
|
} |
|
|
img { |
|
|
display: block; |
|
|
margin: 20px auto; |
|
|
max-width: 100%; |
|
|
height: auto; |
|
|
border-radius: 8px; |
|
|
box-shadow: 0 4px 8px rgba(0,31,77,0.15); |
|
|
} |
|
|
.centered p { |
|
|
text-align: center; |
|
|
font-style: italic; |
|
|
color: var(--navy-blue); |
|
|
margin-top: 0.4em; |
|
|
} |
|
|
.highlight { |
|
|
color: var(--coral); |
|
|
font-weight: 700; |
|
|
} |
|
|
|
|
|
p > ul { margin-top: 0.3em; } |
|
|
</style> |
|
|
</head> |
|
|
<body> |
|
|
<div class="card"> |
|
|
<h1>IqraEval.2 Challenge Interspeech 2026</h1> |
|
|
<img src="IqraEval.png" alt="Interspeech 2026 Challenge Logo" /> |
|
|
|
|
|
<h2>Overview</h2> |
|
|
<p> |
|
|
This <strong>Challenge Interspeech 2026</strong> is a shared task aimed at advancing <strong>automatic assessment of Modern Standard Arabic (MSA) pronunciation</strong> by leveraging computational methods to detect and diagnose pronunciation errors. The focus on MSA provides a standardized and well-defined context for evaluating Arabic pronunciation. |
|
|
</p> |
|
|
<p> |
|
|
Participants will develop systems capable of detecting mispronunciations (e.g., substitution, deletion, or insertion of phonemes). |
|
|
</p> |
|
|
|
|
|
<h2>Timeline</h2> |
|
|
<ul> |
|
|
<li><strong>1 December 2025</strong>: Registration opens</li> |
|
|
<li><strong>15 December 2025</strong>: Release of training data, validation dataset, Arabic phoneme set, and phonemiser</li> |
|
|
<li><strong>15 January 2026</strong>: Release of evaluation dataset</li> |
|
|
<li><strong>20 February 2026</strong>: Registration closes; leaderboard frozen and results announced</li> |
|
|
<li><strong>25 February 2026</strong>: Challenge paper submission deadline</li> |
|
|
</ul> |
|
|
|
|
|
<h2>Task Description: MSA Mispronunciation Detection System</h2> |
|
|
<p> |
|
|
Design a model to detect and provide detailed feedback on mispronunciations in MSA speech. Users read vowelized sentences; the model predicts the spoken phoneme sequence and flags deviations. Evaluation is on the <strong>MSA-Test</strong> dataset with human‐annotated errors. |
|
|
</p> |
|
|
<div class="centered"> |
|
|
<img src="task.png" alt="System Overview" /> |
|
|
<p>Figure: Overview of the Mispronunciation Detection Workflow</p> |
|
|
</div> |
|
|
|
|
|
<h3>1. Read the Sentence</h3> |
|
|
<p> |
|
|
System shows a <strong>Reference Sentence</strong> plus its <strong>Reference Phoneme Sequence</strong>. |
|
|
</p> |
|
|
<p><strong>Example:</strong></p> |
|
|
<ul> |
|
|
<li><strong>Arabic:</strong> يَتَحَدَّثُ النَّاسُ اللُّغَةَ الْعَرَبِيَّةَ</li> |
|
|
<li> |
|
|
<strong>Phoneme:</strong> |
|
|
<code>< y a t a H a d d a v u n n aa s u l l u g h a t a l E a r a b i y y a t a</code> |
|
|
</li> |
|
|
</ul> |
|
|
|
|
|
<h3>2. Save Recording</h3> |
|
|
<p> |
|
|
User speaks; system captures and stores the audio waveform. |
|
|
</p> |
|
|
|
|
|
<h3>3. Mispronunciation Detection</h3> |
|
|
<p> |
|
|
Model predicts the phoneme sequence—deviations from reference indicate mispronunciations. |
|
|
</p> |
|
|
<p><strong>Example of Mispronunciation:</strong></p> |
|
|
<ul> |
|
|
<li><strong>Reference:</strong> <code>< y a t a H a d d a v u n n aa s u l l u g h a t a l E a r a b i y y a t a</code></li> |
|
|
<li><strong>Predicted:</strong> <code>< y a t a H a d d a <span class="highlight">s</span> u n n aa s u l l u g h a t u l E a r a b i y y a t a</code></li> |
|
|
</ul> |
|
|
<p> |
|
|
Here, <code>v</code>→<code>s</code> (substitution) represents a common pronunciation error. |
|
|
</p> |
|
|
|
|
|
<h2>Phoneme Set Description</h2> |
|
|
<p> |
|
|
The phoneme set used in this work is based on a specialized phonetizer developed for vowelized MSA. It includes a comprehensive range of phonemes designed to capture key phonetic and prosodic features of standard Arabic speech, such as stress, pausing, intonation, emphaticness, and notably, gemination. Gemination—the doubling of consonant sounds—is explicitly represented by duplicating the consonant symbol (e.g., <code>/b/</code> becomes <code>/bb/</code>). |
|
|
This phoneme set provides a detailed yet practical representation of the speech sounds relevant for accurate mispronunciation detection in MSA. |
|
|
For further details, including the full phoneme inventory, see <a href="https://huggingface.co/spaces/IqraEval/ArabicPhoneme">Phoneme Inventory</a>. |
|
|
</p> |
|
|
|
|
|
<h2>Training Dataset: Description</h2> |
|
|
<p> |
|
|
Hosted on Hugging Face: |
|
|
</p> |
|
|
<ul> |
|
|
<li> |
|
|
<strong>Training:</strong> 79 hours of MSA speech augmented with additional Arabic data <strong>(Will be released on 15 December 2025)</strong> |
|
|
</li> |
|
|
<li> |
|
|
<strong>Development:</strong> 3.4 hours as dev set <strong>(Will be released on 15 December 2025)</strong> |
|
|
</li> |
|
|
</ul> |
|
|
<p> |
|
|
<strong>Columns:</strong> |
|
|
<ul> |
|
|
<li><code>audio</code>: waveform</li> |
|
|
<li><code>sentence</code>: original text (sentence)</li> |
|
|
<li><code>index</code>: sentence ID</li> |
|
|
<li><code>tashkeel_sentence</code>: fully diacritized text (sentence)</li> |
|
|
<li><code>phoneme</code>: phoneme sequence (using phonetizer)</li> |
|
|
</ul> |
|
|
</p> |
|
|
|
|
|
<h2>Training Dataset: TTS Data (Optional)</h2> |
|
|
<p> |
|
|
Auxiliary high-quality TTS corpus for augmentation: <strong>(Will be released on 15 December 2025)</strong> |
|
|
</p> |
|
|
|
|
|
<h2>Test Dataset: MSA-Test</h2> |
|
|
<p> |
|
|
98 sentences × 18 speakers ≈ 2 h, with deliberate errors and human annotations. |
|
|
<code>load_dataset("Interspeech26/MSA_Test_v2")</code> |
|
|
</p> |
|
|
|
|
|
|
|
|
<h2>Submission Details (Draft)</h2> |
|
|
<p> |
|
|
Submit a UTF-8 CSV named <code>teamID_submission.csv</code> with two columns: |
|
|
</p> |
|
|
<ul> |
|
|
<li><strong>ID:</strong> audio filename (no extension)</li> |
|
|
<li><strong>Labels:</strong> predicted phoneme sequence (space-separated)</li> |
|
|
</ul> |
|
|
<pre>ID,Labels |
|
|
0000_0001, y a t a H a d d a ... |
|
|
0000_0002, m a a n a n s a ... |
|
|
... |
|
|
</pre> |
|
|
<p> |
|
|
<strong>Note:</strong> no extra spaces, single CSV, no archives. |
|
|
</p> |
|
|
|
|
|
<h2>Evaluation Criteria</h2> |
|
|
<p> |
|
|
The Leaderboard is based on phoneme-level <strong>F1-score</strong>. |
|
|
We use a hierarchical evaluation (detection + diagnostic) per <a href="https://arxiv.org/pdf/2310.13974" target="_blank">MDD Overview</a>. |
|
|
</p> |
|
|
<ul> |
|
|
<li><em><strong>What is said</strong></em>: annotated phoneme sequence</li> |
|
|
<li><em><strong>What is predicted</strong></em>: model output</li> |
|
|
<li><em><strong>What should have been said</strong></em>: reference sequence</li> |
|
|
</ul> |
|
|
<p>From these we compute:</p> |
|
|
<ul> |
|
|
<li><strong>TA:</strong> correct phonemes accepted</li> |
|
|
<li><strong>TR:</strong> mispronunciations correctly detected</li> |
|
|
<li><strong>FR:</strong> correct phonemes flagged as errors</li> |
|
|
<li><strong>FA:</strong> mispronunciations missed</li> |
|
|
</ul> |
|
|
<p>Rates:</p> |
|
|
<ul> |
|
|
<li><strong>FRR:</strong> FR/(TA+FR)</li> |
|
|
<li><strong>FAR:</strong> FA/(FA+TR)</li> |
|
|
<li><strong>DER:</strong> DE/(CD+DE)</li> |
|
|
</ul> |
|
|
<p> |
|
|
Plus standard Precision, Recall, F1 for detection: |
|
|
<ul> |
|
|
<li>Precision = TR/(TR+FR)</li> |
|
|
<li>Recall = TR/(TR+FA)</li> |
|
|
<li>F1 = 2·P·R/(P+R)</li> |
|
|
</ul> |
|
|
</p> |
|
|
|
|
|
<h2>Suggested Research Directions</h2> |
|
|
<ol> |
|
|
<li> |
|
|
<strong>Advanced Mispronunciation Detection Models</strong><br> |
|
|
Apply state-of-the-art self-supervised models (e.g., Wav2Vec2.0, HuBERT), using variants that are pre-trained/fine-tuned on Arabic speech. These models can then be fine-tuned on MSA datasets to improve phoneme-level accuracy. |
|
|
</li> |
|
|
<li> |
|
|
<strong>Data Augmentation Strategies</strong><br> |
|
|
Create synthetic mispronunciation examples using pipelines like |
|
|
<a href="https://arxiv.org/abs/2211.00923" target="_blank">SpeechBlender</a>. |
|
|
Augmenting limited Arabic speech data helps mitigate data scarcity and improves model robustness. |
|
|
</li> |
|
|
<li> |
|
|
<strong>Analysis of Common Mispronunciation Patterns</strong><br> |
|
|
Perform statistical analysis on the MSA-Test dataset to identify prevalent errors (e.g., substituting similar phonemes, swapping vowels). |
|
|
These insights can drive targeted training and tailored feedback rules. |
|
|
</li> |
|
|
</ol> |
|
|
|
|
|
<h2>Registration</h2> |
|
|
<p> |
|
|
Teams and individual participants must register to gain access to the test set. Please complete the registration form using the link below: |
|
|
</p> |
|
|
<p> |
|
|
<a href="https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.google.com%2Fforms%2Fd%2Fe%2F1FAIpQLSdDyEP7vzJnpvthiEK6WPws2vpuI_yqbzOzEVqHKs0wdDY_Lg%2Fviewform%3Fusp%3Dheader&data=05%7C02%7C%7C828e4c0463a24cca40de08de2e808b16%7C13a8d02d59f3416a8231b3080e639cad%7C0%7C0%7C638999326802565605%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=CUdgz9Az%2FrFF%2FThZgSkvaXYZneSeVNTfv5drPhbKK44%3D&reserved=0" target="_blank">Registration Form</a> |
|
|
</p> |
|
|
<p> |
|
|
Registration opens on December 1, 2025. |
|
|
</p> |
|
|
|
|
|
<h2>Future Updates</h2> |
|
|
<p> |
|
|
Further details on the open-set leaderboard submission will be posted on the shared task website (December 15, 2025). Stay tuned! |
|
|
</p> |
|
|
|
|
|
<h2>Contact and Support</h2> |
|
|
<p> |
|
|
For inquiries and support, reach out to the task coordinators. |
|
|
</p> |
|
|
|
|
|
<h2>References</h2> |
|
|
<ul> |
|
|
<li>El Kheir Y. et al., “SpeechBlender: Speech Augmentation Framework for Mispronunciation Data Generation,” arXiv:2211.00923, 2022.</li> |
|
|
<li>Aly S. A. et al., “ASMDD: Arabic Speech Mispronunciation Detection Dataset,” arXiv:2111.01136, 2021.</li> |
|
|
<li>Moustafa A. & Aly S. A., “Efficient Voice Identification Using Wav2Vec2.0 and HuBERT…,” arXiv:2111.06331, 2021.</li> |
|
|
<li>El Kheir Y. et al., “Automatic Pronunciation Assessment – A Review,” arXiv:2310.13974, 2021.</li> |
|
|
</ul> |
|
|
</div> |
|
|
</body> |
|
|
</html> |
|
|
|
|
|
|
