Akirami/distillbert-uncased-ag-news

Model Details

Model Description

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

• Developed by: Akirami
• Model type: DistillBert
• License: MIT
• Finetuned from model [optional]: distilbert/distilbert-base-uncased

Model Sources

• Repository: Akirami/distillbert-uncased-ag-news

How to Get Started with the Model

Use the code below to get started with the model.

from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("Akirami/distillbert-uncased-ag-news")
model = AutoModelForSequenceClassification.from_pretrained("Akirami/distillbert-uncased-ag-news")

Training Details

Training Data

AG News Dataset

Training Procedure

The model has been trained through Knowledge Distillation, where the teacher model is nateraw/bert-base-uncased-ag-news and the student model is distilbert/distilbert-base-uncased

Preprocessing [optional]

[More Information Needed]

Training Hyperparameters

• Training regime: Trained in fp16 format

Speeds, Sizes, Times [optional]

[More Information Needed]

Evaluation

Testing Data, Factors & Metrics

Testing Data

The test portion of AG News data is used for testing

Metrics

Classification Report:

Class	Precision	Recall	F1-Score	Support
0	0.95	0.92	0.94	1900
1	0.98	0.98	0.98	1900
2	0.90	0.88	0.89	1900
3	0.88	0.92	0.90	1900
Accuracy			0.93	7600
Macro Avg	0.93	0.93	0.93	7600
Weighted Avg	0.93	0.93	0.93	7600

Balanced Accuracy Score: 0.926578947368421

Accuracy Score: 0.9265789473684211

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: [T4 GPU]
• Hours used: [25 Minutes]
• Cloud Provider: [Google Colab]
• Carbon Emitted: [0.01]