#
Hands-on exercise

In this unit, we explored the challenges of fine-tuning ASR models, acknowledging the time and resources required to fine-tune a model like Whisper (even a small checkpoint) on a new language. To provide a hands-on experience, we have designed an exercise that allows you to navigate the process of fine-tuning an ASR model while using a smaller dataset. The main goal of this exercise is to familiarize you with the process rather than expecting production-level results. We have intentionally set a low metric to ensure that even with limited resources, you should be able to achieve it.

Here are the instructions:

- Fine-tune the `”openai/whisper-tiny”` model using the American English (“en-US”) subset of the `”PolyAI/minds14”` dataset.
- Use the first 450 examples for training, and the rest for evaluation. Ensure you set `num_proc=1` when pre-processing the dataset using the `.map` method (this will ensure your model is submitted correctly for assessment).
- To evaluate the model, use the `wer` and `wer_ortho` metrics as described in this Unit. However, do not convert the metric into percentages by multiplying by 100 (E.g. if WER is 42%, we’ll expect to see the value of 0.42 in this exercise).

Once you have fine-tuned a model, make sure to upload it to the 🤗 Hub with the following `kwargs`:

    kwargs = {
         "dataset_tags": "PolyAI/minds14",
        "finetuned_from": "openai/whisper-tiny",
        "tasks": "automatic-speech-recognition",
    }

You will pass this assignment if your model’s normalised WER (`wer`) is lower than 0.37.

Feel free to build a demo of your model, and share it on Discord! If you have questions, post them in the #audio-study-group channel.

Update on GitHub