distilbert-base-uncased-finetuned-squad-17

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.9001

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	66	3.1226
No log	2.0	132	2.9462
No log	3.0	198	2.5605
No log	4.0	264	2.7058
No log	5.0	330	2.5132
No log	6.0	396	2.6381
No log	7.0	462	2.9053
2.0474	8.0	528	3.0954
2.0474	9.0	594	2.9889
2.0474	10.0	660	3.1002
2.0474	11.0	726	3.1546
2.0474	12.0	792	3.2659
2.0474	13.0	858	3.3862
2.0474	14.0	924	3.6622
2.0474	15.0	990	3.5243
0.4528	16.0	1056	3.4678
0.4528	17.0	1122	3.4637
0.4528	18.0	1188	3.7447
0.4528	19.0	1254	3.5467
0.4528	20.0	1320	3.6459
0.4528	21.0	1386	3.6336
0.4528	22.0	1452	3.7221
0.1412	23.0	1518	3.9210
0.1412	24.0	1584	3.9117
0.1412	25.0	1650	3.8321
0.1412	26.0	1716	3.8429
0.1412	27.0	1782	3.9275
0.1412	28.0	1848	3.8642
0.1412	29.0	1914	3.9054
0.1412	30.0	1980	3.9001

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1
• Datasets 2.14.4
• Tokenizers 0.13.3