02ae2ee64c482dc0195df7603a14436f

This model is a fine-tuned version of FacebookAI/xlm-roberta-large-finetuned-conll03-english on the nyu-mll/glue [stsb] dataset. It achieves the following results on the evaluation set:

Model description

More information needed

More information needed

More information needed

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
distributed_type: multi-GPU
num_devices: 4
total_train_batch_size: 32
total_eval_batch_size: 32
optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: constant
num_epochs: 50

Training Loss	Epoch	Step	Validation Loss	Data Size	Epoch Runtime	Mse	Mae	R2
No log	0	0	8.5180	0	2.6304	8.5193	2.5063	-2.8110
No log	1	179	3.0070	0.0078	3.2396	3.0077	1.4102	-0.3455
No log	2	358	6.1398	0.0156	3.7437	6.1402	2.0618	-1.7467
No log	3	537	2.3541	0.0312	5.0857	2.3549	1.2827	-0.0534
No log	4	716	2.7737	0.0625	6.4949	2.7744	1.3554	-0.2411
No log	5	895	1.0113	0.125	9.4204	1.0117	0.8162	0.5474
0.1158	6	1074	0.7006	0.25	13.3036	0.7009	0.6706	0.6865
0.8871	7	1253	0.6890	0.5	20.6175	0.6895	0.6517	0.6915
0.6328	8.0	1432	0.5982	1.0	34.6432	0.5984	0.6096	0.7323
0.4819	9.0	1611	0.5746	1.0	33.6333	0.5749	0.5757	0.7428
0.3352	10.0	1790	0.7025	1.0	33.6064	0.7026	0.6354	0.6857
0.2879	11.0	1969	0.5954	1.0	32.6148	0.5956	0.5862	0.7336
0.2524	12.0	2148	0.5738	1.0	34.1349	0.5740	0.5573	0.7432
0.2163	13.0	2327	0.5680	1.0	33.9930	0.5682	0.5717	0.7458
0.2	14.0	2506	0.5136	1.0	32.6220	0.5138	0.5313	0.7702
0.1794	15.0	2685	0.5070	1.0	33.7685	0.5071	0.5441	0.7731
1.5192	16.0	2864	0.5400	1.0	33.7515	0.5401	0.5549	0.7584
0.1882	17.0	3043	0.5486	1.0	32.9490	0.5489	0.5567	0.7544
0.1478	18.0	3222	0.6155	1.0	33.8709	0.6156	0.5937	0.7246
0.1634	19.0	3401	0.5588	1.0	32.3731	0.5590	0.5565	0.7499

Safetensors

Model size

0.6B params

Tensor type

F32

Base model

Finetuned

(24)

this model