Sharded checkpoints
Collection
useful sharded checkpoints for users to run inference / fine-tuning on a Google colab without having to deal with CPU OOM issues. • 7 items • Updated • 4
How to use ybelkada/flan-t5-xl-sharded-bf16 with Transformers:
# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
tokenizer = AutoTokenizer.from_pretrained("ybelkada/flan-t5-xl-sharded-bf16")
model = AutoModelForSeq2SeqLM.from_pretrained("ybelkada/flan-t5-xl-sharded-bf16", device_map="auto")No model card