Testing the stdout/stderr output

ue with capturing stdout is that it may contain \r characters that in normal print
reset everything that has been printed so far. There is no problem with pytest, but with pytest -s these
characters get included in the buffer, so to be able to have the test run with and without -s, you have to make an
extra cleanup to the captured output, using re.sub(r'~.*\r', '', buf, 0, re.M).
But, then we have a helper context manager wrapper to automatically take care of it all, regardless of whether it has
some \r’s in it or not, so it’s a simple:
  Copied from transformers.testing_utils import CaptureStdout
with CaptureStdout() as cs:
    function_that_writes_to_stdout()
print(cs.out)
Here is a full test example:
  Copied from transformers.testing_utils import


Testing the stdout/stderr output

CaptureStdout
msg = "Secret message\r"
final = "Hello World"
with CaptureStdout() as cs:
    print(msg + final)
assert cs.out == final + "\n", f"captured: {cs.out}, expecting {final}"
If you’d like to capture stderr use the CaptureStderr class instead:
  Copied from transformers.testing_utils import CaptureStderr
with CaptureStderr() as cs:
    function_that_writes_to_stderr()
print(cs.err)
If you need to capture both streams at once, use the parent CaptureStd class:
  Copied from transformers.testing_utils import CaptureStd
with CaptureStd() as cs:
    function_that_writes_to_stdout_and_stderr()
print(cs.err, cs.out)
Also, to aid debugging test issues, by default these context managers automatically replay the captured streams on exit
from the contex


Capturing logger stream

If you need to validate the output of a logger, you can use CaptureLogger:
  Copied from transformers import logging
from transformers.testing_utils import CaptureLogger
msg = "Testing 1, 2, 3"
logging.set_verbosity_info()
logger = logging.get_logger("transformers.models.bart.tokenization_bart")
with CaptureLogger(logger) as cl:
    logger.info(msg)
assert cl.out, msg + "\n"


Testing with environment variables

If you want to test the impact of environment variables for a specific test you can use a helper decorator
transformers.testing_utils.mockenv
  Copied from transformers.testing_utils import mockenv
class HfArgumentParserTest(unittest.TestCase):
    @mockenv(TRANSFORMERS_VERBOSITY="error")
    def test_env_override(self):
        env_level_str = os.getenv("TRANSFORMERS_VERBOSITY", None)
At times an external program needs to be called, which requires setting PYTHONPATH in os.environ to include
multiple local paths. A helper class transformers.test_utils.TestCasePlus comes to help:
  Copied from transformers.testing


Testing with environment variables

_utils import TestCasePlus
class EnvExampleTest(TestCasePlus):
    def test_external_prog(self):
        env = self.get_env()
        # now call the external program, passing `env` to it
Depending on whether the test file was under the tests test suite or examples it’ll correctly set up
env[PYTHONPATH] to include one of these two directories, and also the src directory to ensure the testing is
done against the current repo, and finally with whatever env[PYTHONPATH] was already set to before the test was
called if anything.
This helper method creates a copy of the os.environ object, so the original remains intact.


Getting reproducible results

In some situations you may want to remove randomness for your tests. To get identical reproducible results set, you
will need to fix the seed:
  Copied seed = 42
# python RNG
import random
random.seed(seed)
# pytorch RNGs
import torch
torch.manual_seed(seed)
torch.backends.cudnn.deterministic = True
if torch.cuda.is_available():
    torch.cuda.manual_seed_all(seed)
# numpy RNG
import numpy as np
np.random.seed(seed)
# tf RNG
tf.random.set_seed(seed)


Debugging tests

To start a debugger at the point of the warning, do this:
  Copied pytest tests/utils/test_logging.py -W error::UserWarning --pdb


Working with github actions workflows

To trigger a self-push workflow CI job, you must:
Create a new branch on transformers origin (not a fork!). The branch name has to start with either ci_ or ci- (main triggers it too, but we can’t do PRs on
main). It also gets triggered only for specific paths - you can find the up-to-date definition in case it
changed since this document has been written here under push: Create a PR from this branch. Then you can see the job appear here. It may not run right away if there
is a backlog.


Testing Experimental CI Features

Testing CI features can be potentially problematic as it can interfere with the normal CI functioning. Therefore if a
new CI feature is to be added, it should be done as following.
Create a new dedicated job that tests what needs to be tested The new job must always succeed so that it gives us a green ✓ (details below). Let it run for some days to see that a variety of different PR types get to run on it (user fork branches,
non-forked branches, branches originating from github.com UI direct file edit, various forced pushes, etc. - there
are so many) while monitoring the experimental job’s logs (not the overall job green as it’s purposefully always
green) When it’s clear that everything is solid, then merge the new changes into existing jobs.
That way experiments on CI functionality itself won’t interfere with the


Testing Experimental CI Features

 normal workflow.
Now how can we make the job always succeed while the new CI feature is being developed?
Some CIs, like TravisCI support ignore-step-failure and will report the overall job as successful, but CircleCI and
Github Actions as of this writing don’t support that.
So the following workaround can be used:
set +euo pipefail at the beginning of the run command to suppress most potential failures in the bash script. the last command must be a success: echo "done" or just true will do
Here is an example:
  Copied - run:
    name: run CI experiment
    command: |
        set +euo pipefail
        echo "setting run-all-despite-any-errors-mode"
        this_command_will_fail
        echo "but bash continues to run"
        # emulate another failure
        false
        # but the last command must be a success


Testing Experimental CI Features

        echo "during experiment do not remove: reporting success to CI, even if there were failures"
For simple commands you could also do:
  Copied cmd_that_may_fail || true
Of course, once satisfied with the results, integrate the experimental step or job with the rest of the normal jobs,
while removing set +euo pipefail or any other things you may have added to ensure that the experimental job doesn’t
interfere with the normal CI functioning.
This whole process would have been much easier if we only could set something like allow-failure for the
experimental step, and let it fail without impacting the overall status of PRs. But as mentioned earlier CircleCI and
Github Actions don’t support it at the moment.
You can vote for this feature and see where it is at these CI-specific threads:
Github Actions: CircleCI:


Installation

Install 🤗 Transformers for whichever deep learning library you’re working with, setup your cache, and optionally configure 🤗 Transformers to run offline.
🤗 Transformers is tested on Python 3.6+, PyTorch 1.1.0+, TensorFlow 2.0+, and Flax. Follow the installation instructions below for the deep learning library you are using:
PyTorch installation instructions. TensorFlow 2.0 installation instructions. Flax installation instructions.


Install with pip

You should install 🤗 Transformers in a virtual environment. If you’re unfamiliar with Python virtual environments, take a look at this guide. A virtual environment makes it easier to manage different projects, and avoid compatibility issues between dependencies.
Start by creating a virtual environment in your project directory:
  Copied python -m venv .env
Activate the virtual environment. On Linux and MacOs:
  Copied source .env/bin/activate
Activate Virtual environment on Windows
  Copied .env/Scripts/activate
Now you’re ready to install 🤗 Transformers with the following command:
  Copied pip install transformers
For CPU-support only, you can conveniently install 🤗 Transformers and a deep learning library in one l


Install with pip

ine. For example, install 🤗 Transformers and PyTorch with:
  Copied pip install 'transformers[torch]'
🤗 Transformers and TensorFlow 2.0:
  Copied pip install 'transformers[tf-cpu]'
M1 / ARM Users You will need to install the following before installing TensorFLow 2.0   Copied brew install cmake
brew install pkg-config
🤗 Transformers and Flax:
  Copied pip install 'transformers[flax]'
Finally, check if 🤗 Transformers has been properly installed by running the following command. It will download a pretrained model:
  Copied python -c "from transformers import pipeline; print(pipeline('sentiment-analysis')('we love you'))"
Then print out the label and score:
  Copied [{'label': 'POSITIVE', 'score': 0.9998704791069031}]


Install from source

Install 🤗 Transformers from source with the following command:
  Copied pip install git+https://github.com/huggingface/transformers
This command installs the bleeding edge main version rather than the latest stable version. The main version is useful for staying up-to-date with the latest developments. For instance, if a bug has been fixed since the last official release but a new release hasn’t been rolled out yet. However, this means the main version may not always be stable. We strive to keep the main version operational, and most issues are usually resolved within a few hours or a day. If you run into a problem, please open an Issue so we can fix it even sooner!
Check if 🤗 Transformers has been properly installed by running the following command:
  Copied python -c "from transformers import pipeline; print(pipeline('sentiment-analysis')('I love you'))"


Editable install

You will need an editable install if you’d like to:
Use the main version of the source code. Contribute to 🤗 Transformers and need to test changes in the code.
Clone the repository and install 🤗 Transformers with the following commands:
  Copied git clone https://github.com/huggingface/transformers.git
cd transformers
pip install -e .
These commands will link the folder you cloned the repository to and your Python library paths. Python will now look inside the folder you cloned to in addition to the no


Editable install

rmal library paths. For example, if your Python packages are typically installed in ~/anaconda3/envs/main/lib/python3.7/site-packages/, Python will also search the folder you cloned to: ~/transformers/.
You must keep the transformers folder if you want to keep using the library.
Now you can easily update your clone to the latest version of 🤗 Transformers with the following command:
  Copied cd ~/transformers/
git pull
Your Python environment will find the main version of 🤗 Transformers on the next run.


Cache setup

Pretrained models are downloaded and locally cached at: ~/.cache/huggingface/hub. This is the default directory given by the shell environment variable TRANSFORMERS_CACHE. On Windows, the default directory is given by C:\Users\username\.cache\huggingface\hub. You can change the shell environment variables shown below - in order of priority - to specify a different cache directory:
Shell environment variable (default): HUGGINGFACE_HUB_CACHE or TRANSFORMERS_CACHE. Shell environment variable: HF_HOME. Shell environment variable: XDG_CACHE_HOME + /huggingface.
🤗 Transformers will use the shell environment variables PYTORCH_TRANSFORMERS_CACHE or PYTORCH_PRETRAINED_BERT_CACHE if you are coming from an earlier iteration of this library and have set those environment variables, unless you specify the shell environment variable TRANSFORMERS_CACHE.


Offline mode

Run 🤗 Transformers in a firewalled or offline environment with locally cached files by setting the environment variable TRANSFORMERS_OFFLINE=1.
Add 🤗 Datasets to your offline training workflow with the environment variable HF_DATASETS_OFFLINE=1.
  Copied HF_DATASETS_OFFLINE=1 TRANSFORMERS_OFFLINE=1 \
python examples/pytorch/translation/run_translation.py --model_name_or_path t5-small --dataset_name wmt16 --dataset_config ro-en ...
This script should run without hanging or waiting to timeout because it won’t attempt to download the model from the Hub.
You can also bypass loading a model from the Hub from each from_pretrained() call with the local_files_only parameter. When set to True, only local files are loaded:
  Copied from transformers import T5Model
model = T5Model.from_pretrained("./path/to/local/directory", local_files_only=True)


Fetch models and tokenizers to use offline

Another option for using 🤗 Transformers offline is to download the files ahead of time, and then point to their local path when you need to use them offline. There are three ways to do this:
Download a file through the user interface on the Model Hub by clicking on the ↓ icon.  Use the PreTrainedModel.from_pretrained() and PreTrainedModel.save_pretrained() workflow: Download your files ahead of time with PreTrainedModel.from_pretrained():   Copied >>> from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
>>> tokenizer = AutoTokenizer.from_pretrained("bigscience/T0_3B")
>>> model = AutoModelForSeq2SeqLM.from_pretrained("bigscience/T0_3B") Save your files to a specified directory with PreTrainedModel.save_pretrained():   Copied >>> tokenizer.save_pretrained("./your/path/bigscience_t0")
>>> model.save_pretrained("./your/path/bigscience_t0") Now when you’re offline, reload your files with PreTrainedModel.from_pretrained() from the specified directory:   Copied >>> toke


nizer = AutoTokenizer.from_pretrained("./your/path/bigscience_t0")
>>> model = AutoModel.from_pretrained("./your/path/bigscience_t0") Programmatically download files with the huggingface_hub library: Install the huggingface_hub library in your virtual environment:   Copied python -m pip install huggingface_hub Use the hf_hub_download function to download a file to a specific path. For example, the following command downloads the config.json file from the T0 model to your desired path:   Copied >>> from huggingface_hub import hf_hub_download
>>> hf_hub_download(repo_id="bigscience/T0_3B", filename="config.json", cache_dir="./your/path/bigscience_t0")
Once your file is downloaded and locally cached, specify it’s local path to load and use it:
  Copied >>> from transformers import AutoConfig
>>> config = AutoConfig.from_pretrained("./your/path/bigscience_t0/config.json")
See the How to download files from the Hub section for more details on downloading files stored on the Hub


Instantiating a big model

When you want to use a very big pretrained model, one challenge is to minimize the use of the RAM. The usual workflow
from PyTorch is:
Create your model with random weights. Load your pretrained weights. Put those pretrained weights in your random model.
Step 1 and 2 both require a full version of the model in memory, which is not a problem in most cases, but if your model starts weighing several GigaBytes, those two copies can make you get out of RAM. Even worse, if you are using torch.distributed to launch a distributed training, each process will load the pretrained model and store these two copies in RAM.
Note that the


Instantiating a big model

 randomly created model is initialized with “empty” tensors, which take the space in memory without filling it (thus the random values are whatever was in this chunk of memory at a given time). The random initialization following the appropriate distribution for the kind of model/parameters instantiated (like a normal distribution for instance) is only performed after step 3 on the non-initialized weights, to be as fast as possible!
In this guide, we explore the solutions Transformers offer to deal with this issue. Note that this is an area of active development, so the APIs explained here may change slightly in the future


Sharded checkpoints

Since version 4.18.0, model checkpoints that end up taking more than 10GB of space are automatically sharded in smaller pieces. In terms of having one single checkpoint when you do model.save_pretrained(save_dir), you will end up with several partial checkpoints (each of which being of size < 10GB) and an index that maps parameter names to the files they are stored in.
You can control the maximum size before sharding with the max_shard_size parameter, so for the sake of an example, we’ll use a normal-size models with a small shard size: let’s take a traditional BERT model.
  Copied from transformers import AutoModel
model = AutoModel.from_pretrained("bert-base-cased")
If you save it using save_pretrained(), you will get a new folder with two files: the config of the model and its weights:
  Copied >>> import os
>>> import tempfile
>>> with tempfile.Tempora


Sharded checkpoints

ryDirectory() as tmp_dir:
...     model.save_pretrained(tmp_dir)
...     print(sorted(os.listdir(tmp_dir)))
['config.json', 'pytorch_model.bin']
Now let’s use a maximum shard size of 200MB:
  Copied >>> with tempfile.TemporaryDirectory() as tmp_dir:
...     model.save_pretrained(tmp_dir, max_shard_size="200MB")
...     print(sorted(os.listdir(tmp_dir)))
['config.json', 'pytorch_model-00001-of-00003.bin', 'pytorch_model-00002-of-00003.bin', 'pytorch_model-00003-of-00003.bin', 'pytorch_model.bin.index.json']
On top of the configuration of the model, we see three different weights files, and an index.json file which is our index. A checkpoint like this can be fully reloaded using the from_pretrained() method:
  Copied >>> with tempfile.TemporaryDirectory() as tmp_dir:
...     model.save_pretrained(tmp_dir, max_shard_size="200MB")
...     new_model = AutoModel


Sharded checkpoints

.from_pretrained(tmp_dir)
The main advantage of doing this for big models is that during step 2 of the workflow shown above, each shard of the checkpoint is loaded after the previous one, capping the memory usage in RAM to the model size plus the size of the biggest shard.
Behind the scenes, the index file is used to determine which keys are in the checkpoint, and where the corresponding weights are stored. We can load that index like any json and get a dictionary:
  Copied >>> import json
>>> with tempfile.TemporaryDirectory() as tmp_dir:
...     model.save_pretrained(tmp_dir, max_shard_size="200MB")
...     with open(os.path.join(tmp_dir, "pytorch_model.bin.index.json"), "r") as f:
...         index = json.load(f)
>>> print(index.keys())
dict_keys(['metadata', 'weight_map'])
The metadata just consists of the total size of the model for now. We plan to ad


Sharded checkpoints

d other information in the future:
  Copied >>> index["metadata"]
{'total_size': 433245184}
The weights map is the main part of this index, which maps each parameter name (as usually found in a PyTorch model state_dict) to the file it’s stored in:
  Copied >>> index["weight_map"]
{'embeddings.LayerNorm.bias': 'pytorch_model-00001-of-00003.bin',
 'embeddings.LayerNorm.weight': 'pytorch_model-00001-of-00003.bin',
 ...
If you want to directly load such a sharded checkpoint inside a model without using from_pretrained() (like you would do model.load_state_dict() for a full checkpoint) you should use load_sharded_checkpoint():
  Copied >>> from transformers.modeling_utils import load_sharded_checkpoint
>>> with tempfile.TemporaryDirectory() as tmp_dir:
...     model.save_pretrained(tmp_dir, max_shard_size="200MB")
...     load_sharded_checkpoint(model, tmp_dir)


Low memory loading

Sharded checkpoints reduce the memory usage during step 2 of the workflow mentioned above, but in order to use that model in a low memory setting, we recommend leveraging our tools based on the Accelerate library.
Please read the following guide for more information: Large model loading using Accelerate


Philosophy

🤗 Transformers is an opinionated library built for:
machine learning researchers and educators seeking to use, study or extend large-scale Transformers models. hands-on practitioners who want to fine-tune those models or serve them in production, or both. engineers who just want to download a pretrained model and use it to solve a given machine learning task.
The library was designed with two strong goals in mind:
Be as easy and fast to use as possible:
We strongly limited the number of user-facing abstractions to learn, in fact, there are almost no abstractions,
just three standard classes required to use each model: configuration,
models, and a preprocessing class (tokenizer for NLP, image processor for vision, feature extractor for audio, and processor for multimodal inputs). All of these classes can be initialized in a simple and unified way from pretrained instances by using a common
from_pretrained() method which downloads (if needed), caches and


Philosophy

loads the related class instance and associated data (configurations’ hyperparameters, tokenizers’ vocabulary,
and models’ weights) from a pretrained checkpoint provided on Hugging Face Hub or your own saved checkpoint. On top of those three base classes, the library provides two APIs: pipeline() for quickly
using a model for inference on a given task and Trainer to quickly train or fine-tune a PyTorch model (all TensorFlow models are compatible with Keras.fit). As a consequence, this library is NOT a modular toolbox of building blocks for neural nets. If you want to
extend or build upon the library, just use regular Python, PyTorch, TensorFlow, Keras modules and inherit from the base
classes of the library to reuse functionalities like model loading and saving. If you’d like to learn more about our coding philosophy for models, check out our Repeat Yourself blog post.
Provide state-of-the-art models with performances as close as possible to the origina


Philosophy

l models:
We provide at least one example for each architecture which reproduces a result provided by the official authors
of said architecture. The code is usually as close to the original code base as possible which means some PyTorch code may be not as
pytorchic as it could be as a result of being converted TensorFlow code and vice versa.
A few other goals:
Expose the models’ internals as consistently as possible: We give access, using a single API, to the full hidden-states and attention weights. The preprocessing classes and base model APIs are standardized to easily switch between models. Incorporate a subjective selection of promising tools for fine-tuning and investigating these models: A simple and consistent way to add new tokens to the vocabulary and embeddings for fine-tuning. Simple ways to mask and prune Transformer heads. Easily switch between PyTorch, TensorFlow 2.0 and Flax, allowing training with one framework and inference with anothe


Main concepts

The library is built around three types of classes for each model:
Model classes can be PyTorch models (torch.nn.Module), Keras models (tf.keras.Model) or JAX/Flax models (flax.linen.Module) that work with the pretrained weights provided in the library. Configuration classes store the hyperparameters required to build a model (such as the number of layers and hidden size). You don’t always need to instantiate these yourself. In particular, if you are using a pretrained model without any modification, creating the model will automatically take care of instantiating the configuration (which is part of the model). Preprocessing classes convert the raw data into a format accepted by the model. A tokenizer stores the vocabulary for each model and provide methods for encoding and decoding strings in a list of to


Main concepts

ken embedding indices to be fed to a model. Image processors preprocess vision inputs, feature extractors preprocess audio inputs, and a processor handles multimodal inputs.
All these classes can be instantiated from pretrained instances, saved locally, and shared on the Hub with three methods:
from_pretrained() lets you instantiate a model, configuration, and preprocessing class from a pretrained version either
provided by the library itself (the supported models can be found on the Model Hub) or
stored locally (or on a server) by the user. save_pretrained() lets you save a model, configuration, and preprocessing class locally so that it can be reloaded using
from_pretrained(). push_to_hub() lets you share a model, configuration, and a preprocessing class to the Hub, so it is easily accessible to everyone


Transformers Agents

Transformers Agents is an experimental API which is subject to change at any time. Results returned by the agents
can vary as the APIs or underlying models are prone to change.
Transformers version v4.29.0, building on the concept of tools and agents. You can play with in
this colab.
In short, it provides a natural language API on top of transformers: we define a set of curated tools and design an
agent to interpret natural language and to use these tools. It is extensible by design; we curated some relevant tools,
but we’ll show you how the system can be extended easily to use any tool developed by the community.
Let’s start with a few examples of what can be achieved with this new API. It is particularly powerful when it comes
to multimodal tasks, so let’s take it for a spin to generate ima


Transformers Agents

ges and read text out loud.
  Copied agent.run("Caption the following image", image=image)
|   Input | Output                            |
|---------|-----------------------------------|
|     nan | A beaver is swimming in the water |
  Copied agent.run("Read the following text out loud", text=text)
| Input                             | Output                                           |
|-----------------------------------|--------------------------------------------------|
| A beaver is swimming in the water | your browser does not support the audio element. |
  Copied agent.run(
    "In the following `document`, where will the TRRF Scientific Advisory Council Meeting take place?",
    document=document,
)
|   Input | Output         |
|---------|----------------|
|     nan | ballroom foyer |


Quickstart

Before being able to use agent.run, you will need to instantiate an agent, which is a large language model (LLM).
We provide support for openAI models as well as opensource alternatives from BigCode and OpenAssistant. The openAI
models perform better (but require you to have an openAI API key, so cannot be used for free); Hugging Face is
providing free access to endpoints for BigCode and OpenAssistant models.
To start with, please install the agents extras in order to install all default dependencies.
  Copied pip install transformers[agents]
To use openAI models, you instantiate an OpenAiAgent after installing the openai dependency:
  Copied pip install openai
  Copied from transformers import OpenAiAgent
agent = OpenAiAgent(model="text-davinci-003", api_key="<your_api_key>")
To use BigCode or OpenAssistant, start by logging in to have access to the Inference API:
  Copied from huggingface_hub import login
login("<YOUR_TOKEN>")
Then, instantiate the agent
  Copi


Quickstart

ed from transformers import HfAgent
# Starcoder
agent = HfAgent("https://api-inference.huggingface.co/models/bigcode/starcoder")
# StarcoderBase
# agent = HfAgent("https://api-inference.huggingface.co/models/bigcode/starcoderbase")
# OpenAssistant
# agent = HfAgent(url_endpoint="https://api-inference.huggingface.co/models/OpenAssistant/oasst-sft-4-pythia-12b-epoch-3.5")
This is using the inference API that Hugging Face provides for free at the moment. If you have your own inference
endpoint for this model (or another one) you can replace the URL above with your URL endpoint.
StarCoder and OpenAssistant are free to use and perform admirably well on simple tasks. However, the checkpoints
don’t hold up when handling more complex prompts. If you’re facing such an issue, we recommend trying out the OpenAI
model which, while sadly not open-source, performs better at this given time.
You’re now good to go! Let’s dive into the two APIs that you now have at your disposal.


Single execution (run)

The single execution method is when using the run() method of the agent:
  Copied agent.run("Draw me a picture of rivers and lakes.")
It automatically selects the tool (or tools) appropriate for the task you want to perform and runs them appropriately. It
can perform one or several tasks in the same instruction (though the more complex your instruction, the more likely
the agent is to fail).
  Copied agent.run("Draw me a picture of the sea then transform the picture to add an island")
Every run() operation is independent, so you can run it several times in a row with different tasks.
Note that your agent is just a large-language model, so small variations in your prompt might yield completely
different results. It’s important to explain as clearly as possible the task you want to perform. We go more in-depth
on how to write good prompts here.
If you’d like to keep a state across executions or to pass non-text objects to the agent, you can do so by specifying
variabl


Single execution (run)

es that you would like the agent to use. For example, you could generate the first image of rivers and lakes,
and ask the model to update that picture to add an island by doing the following:
  Copied picture = agent.run("Generate a picture of rivers and lakes.")
updated_picture = agent.run("Transform the image in `picture` to add an island to it.", picture=picture)
This can be helpful when the model is unable to understand your request and mixes tools. An example would be:   Copied agent.run("Draw me the picture of a capybara swimming in the sea") Here, the model could interpret in two ways: Have the text-to-image generate a capybara swimming in the sea Or, have the text-to-image generate capybara, then use the image-transformation tool to have it swim in the sea In case you would like to force the first scenario, you could do so by passing it the prompt as an argument:   Copied agent.run("Draw me a picture of the `prompt`", prompt="a capybara swimming in the sea")


Chat-based execution (chat)

The agent also has a chat-based approach, using the chat() method:
  Copied agent.chat("Generate a picture of rivers and lakes")
  Copied agent.chat("Transform the picture so that there is a rock in there")
This is an interesting approach when you want to keep the state across instructions. It’s better for experimentation,
but will tend to be much better at single instructions rather than complex instructions (which the run()
method is better at handling).
This method can also take arguments if you would like to pass non-text types or specific prompts.


⚠ Remote execution

For demonstration purposes and so that it could be used with all setups, we had created remote executors for several
of the default tools the agent has access for the release. These are created using
inference endpoints.
We have turned these off for now, but in order to see how to set up remote executors tools yourself,
we recommend reading the custom tool guide.


What’s happening here? What are tools, and what are agents?

Agents
The “agent” here is a large language model, and we’re prompting it so that it has access to a specific set of tools.
LLMs are pretty good at generating small samples of code, so this API takes advantage of that by prompting the
LLM gives a small sample of code performing a task with a set of tools. This prompt is then completed by the
task you give your agent and the description of the tools you give it. This way it gets access to the doc of the
tools you are using, especially their expected inputs and outputs, and can generate the relevant code.
 Tools
Tools are very simple: they’re a single function, with a name, and a description. We then use these tools’ descriptions
to prompt the agent. Th


rough the prompt, we show the agent how it would leverage tools to perform what was
requested in the query.
This is using brand-new tools and not pipelines, because the agent writes better code with very atomic tools.
Pipelines are more refactored and often combine several tasks in one. Tools are meant to be focused on
one very simple task only.
 Code-execution?!
This code is then executed with our small Python interpreter on the set of inputs passed along with your tools.
We hear you screaming “Arbitrary code execution!” in the back, but let us explain why that is not the case.
The only functions that can be called are the tools you provided and the print function, so you’re already
limited in what c


an be executed. You should be safe if it’s limited to Hugging Face tools.
Then, we don’t allow any attribute lookup or imports (which shouldn’t be needed anyway for passing along
inputs/outputs to a small set of functions) so all the most obvious attacks (and you’d need to prompt the LLM
to output them anyway) shouldn’t be an issue. If you want to be on the super safe side, you can execute the
run() method with the additional argument return_code=True, in which case the agent will just return the code
to execute and you can decide whether to do it or not.
The execution will stop at any line trying to perform an illegal operation or if there is a regular Python error
with the code generated by the agen


A curated set of tools

We identify a set of tools that can empower such agents. Here is an updated list of the tools we have integrated
in transformers:
Document question answering: given a document (such as a PDF) in image format, answer a question on this document (Donut) Text question answering: given a long text and a question, answer the question in the text (Flan-T5) Unconditional image captioning: Caption the image! (BLIP) Image question answering: given an image, answer a question on this image (VILT) Image segmentation: given an image and a prompt, output the segmentation mask of that prompt (CLIPSeg) Speech to text: given


A curated set of tools

 an audio recording of a person talking, transcribe the speech into text (Whisper) Text to speech: convert text to speech (SpeechT5) Zero-shot text classification: given a text and a list of labels, identify to which label the text corresponds the most (BART) Text summarization: summarize a long text in one or a few sentences (BART) Translation: translate the text into a given language (NLLB)
These tools have an integration in transformers, and can be used manually as well, for example:
  Copied from transformers import load_tool
tool = load_tool("text-to-speech")
audio = tool("This is a text to speech tool")


Custom tools

While we identify a curated set of tools, we strongly believe that the main value provided by this implementation is
the ability to quickly create and share custom tools.
By pushing the code of a tool to a Hugging Face Space or a model repository, you’re then able to leverage the tool
directly with the agent. We’ve added a few
transformers-agnostic tools to the huggingface-tools organization:
Text downloader: to download a text from a web URL Text to image: generate an image according to a prompt, leveraging stable diffusion Image transformation: modify an image given an initial image


Custom tools

and a prompt, leveraging instruct pix2pix stable diffusion Text to video: generate a small video according to a prompt, leveraging damo-vilab
The text-to-image tool we have been using since the beginning is a remote tool that lives in
huggingface-tools/text-to-image! We will
continue releasing such tools on this and other organizations, to further supercharge this implementation.
The agents have by default access to tools that reside on huggingface-tools.
We explain how to you can write and share your tools as well as leverage any custom tool that resides on the Hub in following guide.


Code generation

So far we have shown how to use the agents to perform actions for you. However, the agent is only generating code
that we then execute using a very restricted Python interpreter. In case you would like to use the code generated in
a different setting, the agent can be prompted to return the code, along with tool definition and accurate imports.
For example, the following instruction
  Copied agent.run("Draw me a picture of rivers and lakes", return_code=True)
returns the following code
  Copied from transformers import load_tool
image_generator = load_tool("huggingface-tools/text-to-image")
image = image_generator(prompt="rivers and lakes")
that you can then modify and execute yourself.


Quick tour

Get up and running with 🤗 Transformers! Whether you’re a developer or an everyday user, this quick tour will help you get started and show you how to use the pipeline() for inference, load a pretrained model and preprocessor with an AutoClass, and quickly train a model with PyTorch or TensorFlow. If you’re a beginner, we recommend checking out our tutorials or course next for more in-depth explanations of the concepts introduced here.
Before you begin, make sure you have all the necessary libraries installed:
  Copied !pip install transformers datasets
You’ll also need to install your preferred machine learning framework:
  Pytorch  Hide Pytorch content    Copied pip install torch  TensorFlow  Hide TensorFlow content    Copied pip install tensorflow


Pipeline

The pipeline() is the easiest and fastest way to use a pretrained model for inference. You can use the pipeline() out-of-the-box for many tasks across different modalities, some of which are shown in the table below:
For a complete list of available tasks, check out the pipeline API reference.
| Task                         | Description                                                                                | Modality        | Pipeline identifier                           |
|------------------------------|--------------------------------------------------------------------------------------------------------------|-----------------|-----------------------------------------------|
| Text classification          | assign a label to a given sequence of text                                                                   | NLP             | pipeline(task=“sentiment-analysis”)           |
| Text generation              | generate text give


Pipeline

n a prompt                                                                                | NLP             | pipeline(task=“text-generation”)              |
| Summarization                | generate a summary of a sequence of text or document                                                         | NLP             | pipeline(task=“summarization”)                |
| Image classification         | assign a label to an image                                                                                | Computer vision | pipeline(task=“image-classification”)         |
| Image segmentation           | assign a label to each individual pixel of an image (supports semantic, panoptic, and instance segmentation) | Computer vision | pipeline(task=“image-segmentation”)           |
| Object detection             | predict the bounding boxes and classes of objects in an image                                                | Computer vision | pipeline(task=“object-det


Pipeline

ection”)             |
| Audio classification         | assign a label to some audio data                                                                            | Audio           | pipeline(task=“audio-classification”)         |
| Automatic speech recognition | transcribe speech into text                                                                                | Audio           | pipeline(task=“automatic-speech-recognition”) |
| Visual question answering    | answer a question about the image, given an image and a question                                             | Multimodal      | pipeline(task=“vqa”)                          |
| Document question answering  | answer a question about the document, given a document and a question                                        | Multimodal      | pipeline(task=“document-question-answering”)  |
| Image captioning             | generate a caption for a given image


Pipeline

                             | Multimodal      | pipeline(task=“image-to-text”)                |
Start by creating an instance of pipeline() and specifying a task you want to use it for. In this guide, you’ll use the pipeline() for sentiment analysis as an example:
  Copied >>> from transformers import pipeline
>>> classifier = pipeline("sentiment-analysis")
The pipeline() downloads and caches a default pretrained model and tokenizer for sentiment analysis. Now you can use the classifier on your target text:
  Copied >>> classifier("We are very happy to show you the 🤗 Transformers library.")
[{'label': 'POSITIVE', 'score': 0.9998}]
If you have more than one input, pass your inputs as a list to the pipeline() to return a list of dictionaries:
  Copied >>> results = classifier(["We are very happy to show you the 🤗 Transformers library.", "We hope you don't hate it."])
>>> for result in results:
...     print(f"label: {result['label']}, with score: {round(result['s


Pipeline

core'], 4)}")
label: POSITIVE, with score: 0.9998
label: NEGATIVE, with score: 0.5309
The pipeline() can also iterate over an entire dataset for any task you like. For this example, let’s choose automatic speech recognition as our task:
  Copied >>> import torch
>>> from transformers import pipeline
>>> speech_recognizer = pipeline("automatic-speech-recognition", model="facebook/wav2vec2-base-960h")
Load an audio dataset (see the 🤗 Datasets Quick Start for more details) you’d like to iterate over. For example, load the MInDS-14 dataset:
  Copied >>> from datasets import load_dataset, Audio
>>> dataset = load_dataset("PolyAI/minds14", name="en-US", split="train")
You need to make sure the sampling rate of the dataset matches the sampling
rate facebook/wav2vec2-base-960h was trained on:
  Copied >>> dataset = dataset.cast_column("audio", Audio(sampling_rate=speech_recognizer.feature_extractor.sampling_rate))
The audio files are automatically loaded and resampled w


Pipeline

hen calling the "audio" column.
Extract the raw waveform arrays from the first 4 samples and pass it as a list to the pipeline:
  Copied >>> result = speech_recognizer(dataset[:4]["audio"])
>>> print([d["text"] for d in result])
['I WOULD LIKE TO SET UP A JOINT ACCOUNT WITH MY PARTNER HOW DO I PROCEED WITH DOING THAT', "FONDERING HOW I'D SET UP A JOIN TO HELL T WITH MY WIFE AND WHERE THE AP MIGHT BE", "I I'D LIKE TOY SET UP A JOINT ACCOUNT WITH MY PARTNER I'M NOT SEEING THE OPTION TO DO IT ON THE APSO I CALLED IN TO GET SOME HELP CAN I JUST DO IT OVER THE PHONE WITH YOU AND GIVE YOU THE INFORMATION OR SHOULD I DO IT IN THE AP AN I'M MISSING SOMETHING UQUETTE HAD PREFERRED TO JUST DO IT OVER THE PHONE OF POSSIBLE THINGS", 'HOW DO I FURN A JOINA COUT']
For larger datasets where the inputs are big (like in speech or vision), you’ll want to pass a generator instead of a list to load all the inputs in memory. Take a look at the pipeline API reference for more informa


Use another model and tokenizer in the pipeline

The pipeline() can accommodate any model from the Hub, making it easy to adapt the pipeline() for other use-cases. For example, if you’d like a model capable of handling French text, use the tags on the Hub to filter for an appropriate model. The top filtered result returns a multilingual BERT model finetuned for sentiment analysis you can use for French text:
  Copied >>> model_name = "nlptown/bert-base-multilingual-uncased-sentiment"
  Pytorch  Hide Pytorch content  Use AutoModelForSequenceClassification and AutoTokenizer to load the pretrained model and it’s associated tokenizer (more on an AutoClass in the next section):   Copied >>> from transformers import AutoTokenizer, AutoModelForSequenceClassification
>>> model = AutoModelForSequenceClassification.from_pretrained(model_name)
>>> tokenizer = AutoTokenizer.from_pretrained(model_name)  TensorFlow  Hide TensorFlow content  Use TFAutoModelForSequenceClassification and AutoTokenizer to load the pre


trained model and it’s associated tokenizer (more on an TFAutoClass in the next section):   Copied >>> from transformers import AutoTokenizer, TFAutoModelForSequenceClassification
>>> model = TFAutoModelForSequenceClassification.from_pretrained(model_name)
>>> tokenizer = AutoTokenizer.from_pretrained(model_name)
Specify the model and tokenizer in the pipeline(), and now you can apply the classifier on French text:
  Copied >>> classifier = pipeline("sentiment-analysis", model=model, tokenizer=tokenizer)
>>> classifier("Nous sommes très heureux de vous présenter la bibliothèque 🤗 Transformers.")
[{'label': '5 stars', 'score': 0.7273}]
If you can’t find a model for your use-case, you’ll need to finetune a pretrained model on your data. Take a look at our finetuning tutorial to learn how. Finally, after you’ve finetuned your pretrained model, please consider sharing the model with the community on the Hub to democratize machine learning for everyone! 🤗


AutoClass

Under the hood, the AutoModelForSequenceClassification and AutoTokenizer classes work together to power the pipeline() you used above. An AutoClass is a shortcut that automatically retrieves the architecture of a pretrained model from its name or path. You only need to select the appropriate AutoClass for your task and it’s associated preprocessing class.
Let’s return to the example from the previous section and see how you can use the AutoClass to replicate the results of the pipeline().


AutoTokenizer

A tokenizer is responsible for preprocessing text into an array of numbers as inputs to a model. There are multiple rules that govern the tokenization process, including how to split a word and at what level words should be split (learn more about tokenization in the tokenizer summary). The most important thing to remember is you need to instantiate a tokenizer with the same model name to ensure you’re using the same tokenization rules a model was pretrained with.
Load a tokenizer with AutoTokenizer:
  Copied >>> from transformers import AutoTokenizer
>>> model_name = "nlptown/bert-base-multilingual-uncased-sentiment"
>>> tokenizer = AutoTokenizer.from_pretrained(model_name)
Pass your text


AutoTokenizer

 to the tokenizer:
  Copied >>> encoding = tokenizer("We are very happy to show you the 🤗 Transformers library.")
>>> print(encoding)
{'input_ids': [101, 11312, 10320, 12495, 19308, 10114, 11391, 10855, 10103, 100, 58263, 13299, 119, 102],
 'token_type_ids': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]}
The tokenizer returns a dictionary containing:
input_ids: numerical representations of your tokens. attention_mask: indicates which tokens should be attended to.
A tokenizer can also accept a list of inputs, and pad and truncate the text to return a batch with uniform length:
  Pytorch  Hide Pytorch content    Copied >>> pt_batch


AutoTokenizer

= tokenizer(
...     ["We are very happy to show you the 🤗 Transformers library.", "We hope you don't hate it."],
...     padding=True,
...     truncation=True,
...     max_length=512,
...     return_tensors="pt",
... )  TensorFlow  Hide TensorFlow content    Copied >>> tf_batch = tokenizer(
...     ["We are very happy to show you the 🤗 Transformers library.", "We hope you don't hate it."],
...     padding=True,
...     truncation=True,
...     max_length=512,
...     return_tensors="tf",
... )
Check out the preprocess tutorial for more details about tokenization, and how to use an AutoImageProcessor, AutoFeatureExtractor and AutoProcessor to preprocess image, audio, and multimodal input


AutoModel

Pytorch  Hide Pytorch content  🤗 Transformers provides a simple and unified way to load pretrained instances. This means you can load an AutoModel like you would load an AutoTokenizer. The only difference is selecting the correct AutoModel for the task. For text (or sequence) classification, you should load AutoModelForSequenceClassification:   Copied >>> from transformers import AutoModelForSequenceClassification
>>> model_name = "nlptown/bert-base-multilingual-uncased-sentiment"
>>> pt_model = AutoModelForSequenceClassification.from_pretrained(model_name) See the task summary for tasks supported by an AutoModel class. Now pass your preprocessed batch of inputs directly to the model. You just have to unpack the dictionary by adding **:   Copied >>> pt_outputs = pt_model(**pt_batch) The model outputs the final activations in the logits attribute. Apply the softmax function to t


AutoModel

he logits to retrieve the probabilities:   Copied >>> from torch import nn
>>> pt_predictions = nn.functional.softmax(pt_outputs.logits, dim=-1)
>>> print(pt_predictions)
tensor([[0.0021, 0.0018, 0.0115, 0.2121, 0.7725],
        [0.2084, 0.1826, 0.1969, 0.1755, 0.2365]], grad_fn=<SoftmaxBackward0>)  TensorFlow  Hide TensorFlow content  🤗 Transformers provides a simple and unified way to load pretrained instances. This means you can load an TFAutoModel like you would load an AutoTokenizer. The only difference is selecting the correct TFAutoModel for the task. For text (or sequence) classification, you should load TFAutoModelForSequenceClassification:   Copied >>> from transformers import TFAutoModelForSequenceClassification
>>> model_name = "nlptown/bert-base-multilingual-uncased-sentiment"
>>> tf_model = TFAutoModelForSequenceClassification.from_pretrained(model_name) See the t


AutoModel

ask summary for tasks supported by an AutoModel class. Now pass your preprocessed batch of inputs directly to the model. You can pass the tensors as-is:   Copied >>> tf_outputs = tf_model(tf_batch) The model outputs the final activations in the logits attribute. Apply the softmax function to the logits to retrieve the probabilities:   Copied >>> import tensorflow as tf
>>> tf_predictions = tf.nn.softmax(tf_outputs.logits, axis=-1)
>>> tf_predictions
All 🤗 Transformers models (PyTorch or TensorFlow) output the tensors before the final activation
function (like softmax) because the final activation function is often fused with the loss. Model outputs are special dataclasses so their attributes are autocompleted in an IDE. The model outputs behave like a tuple or a dictionary (you can index with an integer, a slice or a string) in which case, attributes that are None are ignored


Save a model

Pytorch  Hide Pytorch content  Once your model is fine-tuned, you can save it with its tokenizer using PreTrainedModel.save_pretrained():   Copied >>> pt_save_directory = "./pt_save_pretrained"
>>> tokenizer.save_pretrained(pt_save_directory)
>>> pt_model.save_pretrained(pt_save_directory) When you are ready to use the model again, reload it with PreTrainedModel.from_pretrained():   Copied >>> pt_model = AutoModelForSequenceClassification.from_pretrained("./pt_save_pretrained")  TensorFlow  Hide TensorFlow content  Once your model is fine-tuned, you can save it with its tokenizer using TFPreTrainedModel.save_pretrained():   Copied >>> tf_save_directory = "./tf_save_pretrained"
>>> tokenizer.save_pretrained(tf_save_directory)
>>> tf_model.save_pretrained(tf_save_directory) When you are ready to use the model again, reload it with TFPreTra


Save a model

inedModel.from_pretrained():   Copied >>> tf_model = TFAutoModelForSequenceClassification.from_pretrained("./tf_save_pretrained")
One particularly cool 🤗 Transformers feature is the ability to save a model and reload it as either a PyTorch or TensorFlow model. The from_pt or from_tf parameter can convert the model from one framework to the other:
  Pytorch  Hide Pytorch content    Copied >>> from transformers import AutoModel
>>> tokenizer = AutoTokenizer.from_pretrained(tf_save_directory)
>>> pt_model = AutoModelForSequenceClassification.from_pretrained(tf_save_directory, from_tf=True)  TensorFlow  Hide TensorFlow content    Copied >>> from transformers import TFAutoModel
>>> tokenizer = AutoTokenizer.from_pretrained(pt_save_directory)
>>> tf_model = TFAutoModelForSequenceClassification.from_pretrained(pt_save_directory, from_pt=True)


Custom model builds

You can modify the model’s configuration class to change how a model is built. The configuration specifies a model’s attributes, such as the number of hidden layers or attention heads. You start from scratch when you initialize a model from a custom configuration class. The model attributes are randomly initialized, and you’ll need to train the model before you can use it to get meaningful results.
Start by importing AutoConfig, and then load the pretrained model you want to modify. Within AutoConfig.from_pretrained(), you can specify the attribute you want to change, such as the number of attention heads:
  Copied >>> from transfor


Custom model builds

mers import AutoConfig
>>> my_config = AutoConfig.from_pretrained("distilbert-base-uncased", n_heads=12)
  Pytorch  Hide Pytorch content  Create a model from your custom configuration with AutoModel.from_config():   Copied >>> from transformers import AutoModel
>>> my_model = AutoModel.from_config(my_config)  TensorFlow  Hide TensorFlow content  Create a model from your custom configuration with TFAutoModel.from_config():   Copied >>> from transformers import TFAutoModel
>>> my_model = TFAutoModel.from_config(my_config)
Take a look at the Create a custom architecture guide for more information about building custom configurations.


Trainer - a PyTorch optimized training loop

All models are a standard torch.nn.Module so you can use them in any typical training loop. While you can write your own training loop, 🤗 Transformers provides a Trainer class for PyTorch, which contains the basic training loop and adds additional functionality for features like distributed training, mixed precision, and more.
Depending on your task, you’ll typically pass the following parameters to Trainer:
You’ll start with a PreTrainedModel or a torch.nn.Module:   Copied >>> from transformers import AutoModelForSequenceClassification
>>> model = AutoModelForSequenceClassification.from_pretrained("distilbert-base-uncased") TrainingArguments contains the model hyperparameters you can change like learning rate, batch size, and the number of epochs to train for. T


he default values are used if you don’t specify any training arguments:   Copied >>> from transformers import TrainingArguments
>>> training_args = TrainingArguments(
...     output_dir="path/to/save/folder/",
...     learning_rate=2e-5,
...     per_device_train_batch_size=8,
...     per_device_eval_batch_size=8,
...     num_train_epochs=2,
... ) Load a preprocessing class like a tokenizer, image processor, feature extractor, or processor:   Copied >>> from transformers import AutoTokenizer
>>> tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased") Load a dataset:   Copied >>> from datasets import load_dataset
>>> dataset = load_dataset("rotten_tomatoes")  # doctest: +IGNORE_RESULT Create a function to tokenize the dataset:   Copied >>> def tokenize


_dataset(dataset):
...     return tokenizer(dataset["text"]) Then apply it over the entire dataset with map:   Copied >>> dataset = dataset.map(tokenize_dataset, batched=True) A DataCollatorWithPadding to create a batch of examples from your dataset:   Copied >>> from transformers import DataCollatorWithPadding
>>> data_collator = DataCollatorWithPadding(tokenizer=tokenizer)
Now gather all these classes in Trainer:
  Copied >>> from transformers import Trainer
>>> trainer = Trainer(
...     model=model,
...     args=training_args,
...     train_dataset=dataset["train"],
...     eval_dataset=dataset["test"],
...     tokenizer=tokenizer,
...     data_collator=data_collator,
... )  # doctest: +SKIP
When you’re ready, call train() to start training:
  Copied >>> trai


ner.train()
For tasks - like translation or summarization - that use a sequence-to-sequence model, use the Seq2SeqTrainer and Seq2SeqTrainingArguments classes instead.
You can customize the training loop behavior by subclassing the methods inside Trainer. This allows you to customize features such as the loss function, optimizer, and scheduler. Take a look at the Trainer reference for which methods can be subclassed.
The other way to customize the training loop is by using Callbacks. You can use callbacks to integrate with other libraries and inspect the training loop to report on progress or stop the training early. Callbacks do not modify anything in the training loop itself. To customize something like the loss function, you need to subclass the Trainer instea


Train with TensorFlow

All models are a standard tf.keras.Model so they can be trained in TensorFlow with the Keras API. 🤗 Transformers provides the prepare_tf_dataset() method to easily load your dataset as a tf.data.Dataset so you can start training right away with Keras’ compile and fit methods.
You’ll start with a TFPreTrainedModel or a tf.keras.Model:   Copied >>> from transformers import TFAutoModelForSequenceClassification
>>> model = TFAutoModelForSequenceClassification.from_pretrained("distilbert-base-uncased") Load a preprocessing class like a tokenizer, image processor, feature extractor, or processor:   Copied >>> from transformers import AutoTokenizer
>>> tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased") Create a function to tokenize the dataset:   Copied >>> def tokenize_dataset(dataset):
...     re


Train with TensorFlow

turn tokenizer(dataset["text"])  # doctest: +SKIP Apply the tokenizer over the entire dataset with map and then pass the dataset and tokenizer to prepare_tf_dataset(). You can also change the batch size and shuffle the dataset here if you’d like:   Copied >>> dataset = dataset.map(tokenize_dataset)  # doctest: +SKIP
>>> tf_dataset = model.prepare_tf_dataset(
...     dataset["train"], batch_size=16, shuffle=True, tokenizer=tokenizer
... )  # doctest: +SKIP When you’re ready, you can call compile and fit to start training. Note that Transformers models all have a default task-relevant loss function, so you don’t need to specify one unless you want to:   Copied >>> from tensorflow.keras.optimizers import Adam
>>> model.compile(optimizer=Adam(3e-5))  # No loss argument!
>>> model.fit(tf_dataset)  # doctest: +SKIP


What’s next?

Now that you’ve completed the 🤗 Transformers quick tour, check out our guides and learn how to do more specific things like writing a custom model, fine-tuning a model for a task, and how to train a model with a script. If you’re interested in learning more about 🤗 Transformers core concepts, grab a cup of coffee and take a look at our Conceptual Guides!


Benchmarks

Hugging Face’s Benchmarking tools are deprecated and it is advised to use external Benchmarking libraries to measure the speed
and memory complexity of Transformer models.

Let’s take a look at how 🤗 Transformers models can be benchmarked, best practices, and already available benchmarks.
A notebook explaining in more detail how to benchmark 🤗 Transformers models can be found here.


How to benchmark 🤗 Transformers models

The classes PyTorchBenchmark and TensorFlowBenchmark allow to flexibly benchmark 🤗 Transformers models. The benchmark classes allow us to measure the peak memory usage and required time for both inference and training.
Hereby, inference is defined by a single forward pass, and training is defined by a single forward pass and
backward pass.
The benchmark classes PyTorchBenchmark and TensorFlowBenchmark expect an object of type PyTorchBenchmarkArguments and
TensorFlowBenchmarkArguments, respectively, for instantiation. PyTorchBenchmarkArguments and TensorFlowBenchmarkArguments are data classes and contain all relevant configurations for their corresponding benchmark class. In the following example, it is shown how a BERT model of type bert-base-cased can be benchmarked.
  Pytorch  Hide Pytorch content    Copied >>> from transformers import PyTorchBenchmark, PyTorchBenchmarkArguments
>>> args = PyTorchBenchmarkArguments(models=["bert-base-uncased"], bat


How to benchmark 🤗 Transformers models

ch_sizes=[8], sequence_lengths=[8, 32, 128, 512])
>>> benchmark = PyTorchBenchmark(args)  TensorFlow  Hide TensorFlow content    Copied >>> from transformers import TensorFlowBenchmark, TensorFlowBenchmarkArguments
>>> args = TensorFlowBenchmarkArguments(
...     models=["bert-base-uncased"], batch_sizes=[8], sequence_lengths=[8, 32, 128, 512]
... )
>>> benchmark = TensorFlowBenchmark(args)
Here, three arguments are given to the benchmark argument data classes, namely models, batch_sizes, and
sequence_lengths. The argument models is required and expects a list of model identifiers from the
model hub The list arguments batch_sizes and sequence_lengths define
the size of the input_ids on which the model is benchmarked. There are many more parameters that can be configured
via the benchmark argument data classes. For more detail on these one can either directly consult the files
src/transformers/benchmark/benchmark_args_utils.py, src/transformers/benc


How to benchmark 🤗 Transformers models

hmark/benchmark_args.py (for PyTorch)
and src/transformers/benchmark/benchmark_args_tf.py (for Tensorflow). Alternatively, running the following shell
commands from root will print out a descriptive list of all configurable parameters for PyTorch and Tensorflow
respectively.
  Pytorch  Hide Pytorch content    Copied python examples/pytorch/benchmarking/run_benchmark.py --help An instantiated benchmark object can then simply be run by calling benchmark.run().   Copied >>> results = benchmark.run()
>>> print(results)
====================       INFERENCE - SPEED - RESULT       ====================
--------------------------------------------------------------------------------
Model Name             Batch Size     Seq Length     Time in s
--------------------------------------------------------------------------------
bert-base-uncased          8               8             0.006
bert-base-uncased          8               32


How to benchmark 🤗 Transformers models

      0.006
bert-base-uncased          8              128            0.018
bert-base-uncased          8              512            0.088
--------------------------------------------------------------------------------
====================      INFERENCE - MEMORY - RESULT       ====================
--------------------------------------------------------------------------------
Model Name             Batch Size     Seq Length    Memory in MB
--------------------------------------------------------------------------------
bert-base-uncased          8               8             1227
bert-base-uncased          8               32            1281
bert-base-uncased          8              128            1307
bert-base-uncased          8              512            1539
--------------------------------------------------------------------------------
====================        ENVIRONMENT INFORMATION         ====================
- transform


How to benchmark 🤗 Transformers models

ers_version: 2.11.0
- framework: PyTorch
- use_torchscript: False
- framework_version: 1.4.0
- python_version: 3.6.10
- system: Linux
- cpu: x86_64
- architecture: 64bit
- date: 2020-06-29
- time: 08:58:43.371351
- fp16: False
- use_multiprocessing: True
- only_pretrain_model: False
- cpu_ram_mb: 32088
- use_gpu: True
- num_gpus: 1
- gpu: TITAN RTX
- gpu_ram_mb: 24217
- gpu_power_watts: 280.0
- gpu_performance_state: 2
- use_tpu: False  TensorFlow  Hide TensorFlow content    Copied python examples/tensorflow/benchmarking/run_benchmark_tf.py --help An instantiated benchmark object can then simply be run by calling benchmark.run().   Copied >>> results = benchmark.run()
>>> print(results)
>>> results = benchmark.run()
>>> print(results)
====================       INFERENCE - SPEED - RESULT       ====================
--------------------------------------------------------------------------------
Model Name             Batch Size     Seq Length     Time


How to benchmark 🤗 Transformers models

 in s
--------------------------------------------------------------------------------
bert-base-uncased          8               8             0.005
bert-base-uncased          8               32            0.008
bert-base-uncased          8              128            0.022
bert-base-uncased          8              512            0.105
--------------------------------------------------------------------------------
====================      INFERENCE - MEMORY - RESULT       ====================
--------------------------------------------------------------------------------
Model Name             Batch Size     Seq Length    Memory in MB
--------------------------------------------------------------------------------
bert-base-uncased          8               8             1330
bert-base-uncased          8               32            1330
bert-base-uncased          8              128            1330
bert-base-uncased          8


How to benchmark 🤗 Transformers models

           512            1770
--------------------------------------------------------------------------------
====================        ENVIRONMENT INFORMATION         ====================
- transformers_version: 2.11.0
- framework: Tensorflow
- use_xla: False
- framework_version: 2.2.0
- python_version: 3.6.10
- system: Linux
- cpu: x86_64
- architecture: 64bit
- date: 2020-06-29
- time: 09:26:35.617317
- fp16: False
- use_multiprocessing: True
- only_pretrain_model: False
- cpu_ram_mb: 32088
- use_gpu: True
- num_gpus: 1
- gpu: TITAN RTX
- gpu_ram_mb: 24217
- gpu_power_watts: 280.0
- gpu_performance_state: 2
- use_tpu: False
By default, the time and the required memory for inference are benchmarked. In the example output above the first
two sections show the result corresponding to inference time and inference memory. In addition, all relevant
information about the computing environment, e.g. the GPU type, the system, the library versions, et


How to benchmark 🤗 Transformers models

c… are printed
out in the third section under ENVIRONMENT INFORMATION. This information can optionally be saved in a .csv file
when adding the argument save_to_csv=True to PyTorchBenchmarkArguments and
TensorFlowBenchmarkArguments respectively. In this case, every section is saved in a separate
.csv file. The path to each .csv file can optionally be defined via the argument data classes.
Instead of benchmarking pre-trained models via their model identifier, e.g. bert-base-uncased, the user can
alternatively benchmark an arbitrary configuration of any available model class. In this case, a list of
configurations must be inserted with the benchmark args as follows.
  Pytorch  Hide Pytorch content    Copied >>> from transformers import PyTorchBenchmark, PyTorchBenchmarkArguments, BertConfig
>>> args = PyTorchBenchmarkArguments(
...     models=["bert-base", "bert-384-hid", "bert-6-lay"], batch_sizes=[8], sequence_lengths=[8, 32, 128, 512]
... )
>>> confi


How to benchmark 🤗 Transformers models

g_base = BertConfig()
>>> config_384_hid = BertConfig(hidden_size=384)
>>> config_6_lay = BertConfig(num_hidden_layers=6)
>>> benchmark = PyTorchBenchmark(args, configs=[config_base, config_384_hid, config_6_lay])
>>> benchmark.run()
====================       INFERENCE - SPEED - RESULT       ====================
--------------------------------------------------------------------------------
Model Name             Batch Size     Seq Length       Time in s
--------------------------------------------------------------------------------
bert-base                  8              128            0.006
bert-base                  8              512            0.006
bert-base                  8              128            0.018
bert-base                  8              512            0.088
bert-384-hid              8               8             0.006
bert-384-hid              8               32            0.006
bert-384


How to benchmark 🤗 Transformers models

-hid              8              128            0.011
bert-384-hid              8              512            0.054
bert-6-lay                 8               8             0.003
bert-6-lay                 8               32            0.004
bert-6-lay                 8              128            0.009
bert-6-lay                 8              512            0.044
--------------------------------------------------------------------------------
====================      INFERENCE - MEMORY - RESULT       ====================
--------------------------------------------------------------------------------
Model Name             Batch Size     Seq Length      Memory in MB
--------------------------------------------------------------------------------
bert-base                  8               8             1277
bert-base                  8               32            1281
bert-base                  8              128


How to benchmark 🤗 Transformers models

  1307
bert-base                  8              512            1539
bert-384-hid              8               8             1005
bert-384-hid              8               32            1027
bert-384-hid              8              128            1035
bert-384-hid              8              512            1255
bert-6-lay                 8               8             1097
bert-6-lay                 8               32            1101
bert-6-lay                 8              128            1127
bert-6-lay                 8              512            1359
--------------------------------------------------------------------------------
====================        ENVIRONMENT INFORMATION         ====================
- transformers_version: 2.11.0
- framework: PyTorch
- use_torchscript: False
- framework_version: 1.4.0
- python_version: 3.6.10
- system: Linux
- cpu: x86_64
- architecture: 64bit
- date: 2020-06


How to benchmark 🤗 Transformers models

-29
- time: 09:35:25.143267
- fp16: False
- use_multiprocessing: True
- only_pretrain_model: False
- cpu_ram_mb: 32088
- use_gpu: True
- num_gpus: 1
- gpu: TITAN RTX
- gpu_ram_mb: 24217
- gpu_power_watts: 280.0
- gpu_performance_state: 2
- use_tpu: False  TensorFlow  Hide TensorFlow content    Copied >>> from transformers import TensorFlowBenchmark, TensorFlowBenchmarkArguments, BertConfig
>>> args = TensorFlowBenchmarkArguments(
...     models=["bert-base", "bert-384-hid", "bert-6-lay"], batch_sizes=[8], sequence_lengths=[8, 32, 128, 512]
... )
>>> config_base = BertConfig()
>>> config_384_hid = BertConfig(hidden_size=384)
>>> config_6_lay = BertConfig(num_hidden_layers=6)
>>> benchmark = TensorFlowBenchmark(args, configs=[config_base, config_384_hid, config_6_lay])
>>> benchmark.run()
====================       INFERENCE - SPEED - RESULT       ====================
--------------------------------------------------------------------------------
Mode


How to benchmark 🤗 Transformers models

l Name             Batch Size     Seq Length       Time in s
--------------------------------------------------------------------------------
bert-base                  8               8             0.005
bert-base                  8               32            0.008
bert-base                  8              128            0.022
bert-base                  8              512            0.106
bert-384-hid              8               8             0.005
bert-384-hid              8               32            0.007
bert-384-hid              8              128            0.018
bert-384-hid              8              512            0.064
bert-6-lay                 8               8             0.002
bert-6-lay                 8               32            0.003
bert-6-lay                 8              128            0.0011
bert-6-lay                 8              512            0.074
---------------------------------------------------


How to benchmark 🤗 Transformers models

-----------------------------
====================      INFERENCE - MEMORY - RESULT       ====================
--------------------------------------------------------------------------------
Model Name             Batch Size     Seq Length      Memory in MB
--------------------------------------------------------------------------------
bert-base                  8               8             1330
bert-base                  8               32            1330
bert-base                  8              128            1330
bert-base                  8              512            1770
bert-384-hid              8               8             1330
bert-384-hid              8               32            1330
bert-384-hid              8              128            1330
bert-384-hid              8              512            1540
bert-6-lay                 8               8             1330
bert-6-lay                 8               32            1330
bert-6-


How to benchmark 🤗 Transformers models

lay                 8              128            1330
bert-6-lay                 8              512            1540
--------------------------------------------------------------------------------
====================        ENVIRONMENT INFORMATION         ====================
- transformers_version: 2.11.0
- framework: Tensorflow
- use_xla: False
- framework_version: 2.2.0
- python_version: 3.6.10
- system: Linux
- cpu: x86_64
- architecture: 64bit
- date: 2020-06-29
- time: 09:38:15.487125
- fp16: False
- use_multiprocessing: True
- only_pretrain_model: False
- cpu_ram_mb: 32088
- use_gpu: True
- num_gpus: 1
- gpu: TITAN RTX
- gpu_ram_mb: 24217
- gpu_power_watts: 280.0
- gpu_performance_state: 2
- use_tpu: False
Again, inference time and required memory for inference are measured, but this time for customized configurations
of the BertModel class. This feature can especially be helpful when deciding for which configuration the model
should be tr


Benchmark best practices

This section lists a couple of best practices one should be aware of when benchmarking a model.
Currently, only single device benchmarking is supported. When benchmarking on GPU, it is recommended that the user
specifies on which device the code should be run by setting the CUDA_VISIBLE_DEVICES environment variable in the
shell, e.g. export CUDA_VISIBLE_DEVICES=0 before running the code. The option no_multi_processing should only be set to True for testing and debugging. To ensure accurate
memory measurement it is recommended to run each memory benchmark in a separate process by making sure
no_multi_processing is set to True. One should always state the environment information when sharing the results of a model benchmark. Results can vary
heavily between different GPU devices, library versions, etc., so that benchmark results on their own are not very
useful for the community.


Sharing your benchmark

Previously all available core models (10 at the time) have been benchmarked for inference time, across many different
settings: using PyTorch, with and without TorchScript, using TensorFlow, with and without XLA. All of those tests were
done across CPUs (except for TensorFlow XLA) and GPUs.
The approach is detailed in the following blogpost and the results are
available here.
With the new benchmark tools, it is easier than ever to share your benchmark results with the community
PyTorch Benchmarking Results. TensorFlow Benchmarking Results.


Image classification

Image classification assigns a label or class to an image. Unlike text or audio classification, the inputs are the
pixel values that comprise an image. There are many applications for image classification, such as detecting damage
after a natural disaster, monitoring crop health, or helping screen medical images for signs of disease.
This guide illustrates how to:
Fine-tune ViT on the Food-101 dataset to classify a food item in an image. Use your fine-tuned model for inference.
The task illustrated in this tutorial is supported by the following model architectures:
 BEiT, BiT, ConvNeXT, ConvNeXTV2, CvT, Data2VecVision,


Image classification

 DeiT, DiNAT, DINOv2, EfficientFormer, EfficientNet, FocalNet, ImageGPT, LeViT, MobileNetV1, MobileNetV2, MobileViT, MobileViTV2, NAT, Perceiver, PoolFormer, PVT, RegNet, ResNet, SegFormer, SwiftFormer, Swin Transformer, Swin Transformer V2, VAN, ViT, ViT Hybrid, ViTMSN
Before you begin, make sure you have all the necessary libraries installed:
  Copied pip install transformers datasets evaluate
We encourage you to log in to your Hugging Face account to upload and share your model with the community. When prompted, enter your token to log in:
  Copied >>> from huggingface_hub import notebook_login
>>> notebook_login()


Load Food-101 dataset

Start by loading a smaller subset of the Food-101 dataset from the 🤗 Datasets library. This will give you a chance to
experiment and make sure everything works before spending more time training on the full dataset.
  Copied >>> from datasets import load_dataset
>>> food = load_dataset("food101", split="train[:5000]")
Split the dataset’s train split into a train and test set with the train_test_split method:
  Copied >>> food = food.train_test_split(test_size=0.2)
Then take a look at an example:
  Copied >>> food["train"][0]
{'image': <PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=512x5


Load Food-101 dataset

12 at 0x7F52AFC8AC50>,
 'label': 79}
Each example in the dataset has two fields:
image: a PIL image of the food item label: the label class of the food item
To make it easier for the model to get the label name from the label id, create a dictionary that maps the label name
to an integer and vice versa:
  Copied >>> labels = food["train"].features["label"].names
>>> label2id, id2label = dict(), dict()
>>> for i, label in enumerate(labels):
...     label2id[label] = str(i)
...     id2label[str(i)] = label
Now you can convert the label id to a label name:
  Copied >>> id2label[str(79)]
'prime_rib


Preprocess

The next step is to load a ViT image processor to process the image into a tensor:
  Copied >>> from transformers import AutoImageProcessor
>>> checkpoint = "google/vit-base-patch16-224-in21k"
>>> image_processor = AutoImageProcessor.from_pretrained(checkpoint)
  Pytorch  Hide Pytorch content  Apply some image transformations to the images to make the model more robust against overfitting. Here you’ll use torchvision’s transforms module, but you can also use any image library you like. Crop a random part of the image, resize it, and normalize it with the image mean and standard deviation:   Copied >>> from torchvision.transforms import RandomResizedCrop, Compose, Normalize, ToTensor
>>> normalize = Normalize(mean=image_processor.image_mean, std=image_processor.image_std)
>>> size = (
...     image_processor.size["shortest_edge"]
...     if "shortest_edge" in image_processor.size
...     else (image_processor.size["height"], image_processor.size[


Preprocess

"width"])
... )
>>> _transforms = Compose([RandomResizedCrop(size), ToTensor(), normalize]) Then create a preprocessing function to apply the transforms and return the pixel_values - the inputs to the model - of the image:   Copied >>> def transforms(examples):
...     examples["pixel_values"] = [_transforms(img.convert("RGB")) for img in examples["image"]]
...     del examples["image"]
...     return examples To apply the preprocessing function over the entire dataset, use 🤗 Datasets with_transform method. The transforms are applied on the fly when you load an element of the dataset:   Copied >>> food = food.with_transform(transforms) Now create a batch of examples using DefaultDataCollator. Unlike other data collators in 🤗 Transformers, the DefaultDataCollator does not apply additional preprocessing such as padding.   Copied >>> from transformers import DefaultDataCollator
>>> data_collator = DefaultDataCollator()
  TensorFlow  Hide TensorFlo


Preprocess

w content  To avoid overfitting and to make the model more robust, add some data augmentation to the training part of the dataset.
Here we use Keras preprocessing layers to define the transformations for the training data (includes data augmentation),
and transformations for the validation data (only center cropping, resizing and normalizing). You can use tf.imageor
any other library you prefer.   Copied >>> from tensorflow import keras
>>> from tensorflow.keras import layers
>>> size = (image_processor.size["height"], image_processor.size["width"])
>>> train_data_augmentation = keras.Sequential(
...     [
...         layers.RandomCrop(size[0], size[1]),
...         layers.Rescaling(scale=1.0 / 127.5, offset=-1),
...         layers.RandomFlip("horizontal"),
...         layers.RandomRotation(factor=0.02),
...         layers.RandomZoom(height_factor=0.2, width_factor=0.2),
...     ],
...     name="train_data_augmentation",
... )
>>> val_data_augme


Preprocess

ntation = keras.Sequential(
...     [
...         layers.CenterCrop(size[0], size[1]),
...         layers.Rescaling(scale=1.0 / 127.5, offset=-1),
...     ],
...     name="val_data_augmentation",
... ) Next, create functions to apply appropriate transformations to a batch of images, instead of one image at a time.   Copied >>> import numpy as np
>>> import tensorflow as tf
>>> from PIL import Image
>>> def convert_to_tf_tensor(image: Image):
...     np_image = np.array(image)
...     tf_image = tf.convert_to_tensor(np_image)
...     # `expand_dims()` is used to add a batch dimension since
...     # the TF augmentation layers operates on batched inputs.
...     return tf.expand_dims(tf_image, 0)
>>> def preprocess_train(example_batch):
...     """Apply train_transforms across a batch."""
...     images = [
...         train_data_augmentation(convert_to_tf_tensor(image.convert("RGB"))) for image in example_batch["image"]
...     ]
...     example_


Preprocess

batch["pixel_values"] = [tf.transpose(tf.squeeze(image)) for image in images]
...     return example_batch
... def preprocess_val(example_batch):
...     """Apply val_transforms across a batch."""
...     images = [
...         val_data_augmentation(convert_to_tf_tensor(image.convert("RGB"))) for image in example_batch["image"]
...     ]
...     example_batch["pixel_values"] = [tf.transpose(tf.squeeze(image)) for image in images]
...     return example_batch Use 🤗 Datasets set_transform to apply the transformations on the fly:   Copied food["train"].set_transform(preprocess_train)
food["test"].set_transform(preprocess_val) As a final preprocessing step, create a batch of examples using DefaultDataCollator. Unlike other data collators in 🤗 Transformers, the
DefaultDataCollator does not apply additional preprocessing, such as padding.   Copied >>> from transformers import DefaultDataCollator
>>> data_collator = DefaultDataCollator(return_tensors="


Evaluate

Including a metric during training is often helpful for evaluating your model’s performance. You can quickly load an
evaluation method with the 🤗 Evaluate library. For this task, load
the accuracy metric (see the 🤗 Evaluate quick tour to learn more about how to load and compute a metric):
  Copied >>> import evaluate
>>> accuracy = evaluate.load("accuracy")
Then create a function that passes your predictions and labels to compute to calculate the accuracy:
  Copied >>> import numpy as np
>>> def compute_metrics(eval_pred):
...     predictions, labels = eval_pred
...     predictions = np.argmax(predictions, axis=1)
...     return accuracy.compute(predictions=predictions, references=labels)
Your compute_metrics function is ready to go now, and you’ll return to it when you set up your training.


Train

Pytorch  Hide Pytorch content  If you aren’t familiar with finetuning a model with the Trainer, take a look at the basic tutorial here! You’re ready to start training your model now! Load ViT with AutoModelForImageClassification. Specify the number of labels along with the number of expected labels, and the label mappings:   Copied >>> from transformers import AutoModelForImageClassification, TrainingArguments, Trainer
>>> model = AutoModelForImageClassification.from_pretrained(
...     checkpoint,
...     num_labels=len(labels),
...     id2label=id2label,
...     label2id=label2id,
... ) At this point, only three steps remain: Define your training hyperparameters in TrainingArguments. It is important you don’t remove unused columns because that’ll drop the image column. Without the image column, you can’t create pixel_values. Set remove_unused_columns=False to preve


Train

nt this behavior! The only other required parameter is output_dir which specifies where to save your model. You’ll push this model to the Hub by setting push_to_hub=True (you need to be signed in to Hugging Face to upload your model). At the end of each epoch, the Trainer will evaluate the accuracy and save the training checkpoint. Pass the training arguments to Trainer along with the model, dataset, tokenizer, data collator, and compute_metrics function. Call train() to finetune your model.   Copied >>> training_args = TrainingArguments(
...     output_dir="my_awesome_food_model",
...     remove_unused_columns=False,
...     evaluation_strategy="epoch",
...     save_strategy="epoch",
...     learning_rate=5e-5,
...     per_device_train_batch_size=16,
...     gradient_accumulation_steps=4,
...     per_device_eval_batch_size=16,
...     num_train_epochs=3,
...     war


Train

mup_ratio=0.1,
...     logging_steps=10,
...     load_best_model_at_end=True,
...     metric_for_best_model="accuracy",
...     push_to_hub=True,
... )
>>> trainer = Trainer(
...     model=model,
...     args=training_args,
...     data_collator=data_collator,
...     train_dataset=food["train"],
...     eval_dataset=food["test"],
...     tokenizer=image_processor,
...     compute_metrics=compute_metrics,
... )
>>> trainer.train() Once training is completed, share your model to the Hub with the push_to_hub() method so everyone can use your model:   Copied >>> trainer.push_to_hub()
  TensorFlow  Hide TensorFlow content  If you are unfamiliar with fine-tuning a model with Keras, check out the basic tutorial first! To fine-tune a model in TensorFlow, follow these steps: Define the training hyperparameters, and set up an optimizer and a learning rate schedule. Instantia


Train

te a pre-trained model. Convert a 🤗 Dataset to a tf.data.Dataset. Compile your model. Add callbacks and use the fit() method to run the training. Upload your model to 🤗 Hub to share with the community. Start by defining the hyperparameters, optimizer and learning rate schedule:   Copied >>> from transformers import create_optimizer
>>> batch_size = 16
>>> num_epochs = 5
>>> num_train_steps = len(food["train"]) * num_epochs
>>> learning_rate = 3e-5
>>> weight_decay_rate = 0.01
>>> optimizer, lr_schedule = create_optimizer(
...     init_lr=learning_rate,
...     num_train_steps=num_train_steps,
...     weight_decay_rate=weight_decay_rate,
...     num_warmup_steps=0,
... ) Then, load ViT with TFAutoModelForImageClassification along with the label mappings:   Copied >>> from transformers import TFAutoModelForImageClassification
>>> model = TFAutoModelForImageClassificati


Train

on.from_pretrained(
...     checkpoint,
...     id2label=id2label,
...     label2id=label2id,
... ) Convert your datasets to the tf.data.Dataset format using the to_tf_dataset and your data_collator:   Copied >>> # converting our train dataset to tf.data.Dataset
>>> tf_train_dataset = food["train"].to_tf_dataset(
...     columns="pixel_values", label_cols="label", shuffle=True, batch_size=batch_size, collate_fn=data_collator
... )
>>> # converting our test dataset to tf.data.Dataset
>>> tf_eval_dataset = food["test"].to_tf_dataset(
...     columns="pixel_values", label_cols="label", shuffle=True, batch_size=batch_size, collate_fn=data_collator
... ) Configure the model for training with compile():   Copied >>> from tensorflow.keras.losses import SparseCategoricalCrossentropy
>>> loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
>>> model.compile(


Train

optimizer=optimizer, loss=loss) To compute the accuracy from the predictions and push your model to the 🤗 Hub, use Keras callbacks.
Pass your compute_metrics function to KerasMetricCallback,
and use the PushToHubCallback to upload the model:   Copied >>> from transformers.keras_callbacks import KerasMetricCallback, PushToHubCallback
>>> metric_callback = KerasMetricCallback(metric_fn=compute_metrics, eval_dataset=tf_eval_dataset)
>>> push_to_hub_callback = PushToHubCallback(
...     output_dir="food_classifier",
...     tokenizer=image_processor,
...     save_strategy="no",
... )
>>> callbacks = [metric_callback, push_to_hub_callback] Finally, you are ready to train your model! Call fit() with your training and validation datasets, the number of epochs,
and your callbacks to fine-tune the model:   Copied >>> model.fit(tf_train_dataset, validation_data=tf_eval_dataset


Train

, epochs=num_epochs, callbacks=callbacks)
Epoch 1/5
250/250 [==============================] - 313s 1s/step - loss: 2.5623 - val_loss: 1.4161 - accuracy: 0.9290
Epoch 2/5
250/250 [==============================] - 265s 1s/step - loss: 0.9181 - val_loss: 0.6808 - accuracy: 0.9690
Epoch 3/5
250/250 [==============================] - 252s 1s/step - loss: 0.3910 - val_loss: 0.4303 - accuracy: 0.9820
Epoch 4/5
250/250 [==============================] - 251s 1s/step - loss: 0.2028 - val_loss: 0.3191 - accuracy: 0.9900
Epoch 5/5
250/250 [==============================] - 238s 949ms/step - loss: 0.1232 - val_loss: 0.3259 - accuracy: 0.9890 Congratulations! You have fine-tuned your model and shared it on the 🤗 Hub. You can now use it for inference!
For a more in-depth example of how to finetune a model for image classification, take a look at the corresponding PyTorch notebo


Inference

Great, now that you’ve fine-tuned a model, you can use it for inference!
Load an image you’d like to run inference on:
  Copied >>> ds = load_dataset("food101", split="validation[:10]")
>>> image = ds["image"][0]
The simplest way to try out your finetuned model for inference is to use it in a pipeline(). Instantiate a pipeline for image classification with your model, and pass your image to it:
  Copied >>> from transformers import pipeline
>>> classifier = pipeline("image-classification", model="my_awesome_food_model")
>>> classifier(image)
[{'score': 0.31856709718704224, 'label': 'beignets'},
 {'score': 0.015232225880026817, 'label': 'bruschetta'},
 {'score': 0.01519392803311348, 'label': 'chicken_wings'},
 {'score': 0.013022331520915031, 'label': 'pork_chop'},
 {'score': 0.012728818692266941, 'label': 'prime_rib'}]
You can also manually rep


Inference

licate the results of the pipeline if you’d like:
  Pytorch  Hide Pytorch content  Load an image processor to preprocess the image and return the input as PyTorch tensors:   Copied >>> from transformers import AutoImageProcessor
>>> import torch
>>> image_processor = AutoImageProcessor.from_pretrained("my_awesome_food_model")
>>> inputs = image_processor(image, return_tensors="pt") Pass your inputs to the model and return the logits:   Copied >>> from transformers import AutoModelForImageClassification
>>> model = AutoModelForImageClassification.from_pretrained("my_awesome_food_model")
>>> with torch.no_grad():
...     logits = model(**inputs).logits Get the predicted label with the highest probability, and use the model’s id2label mapping to convert it to a label:   Copied >>> predicted_label = logits.argmax(-1).item()
>>> model.config.id2lab


Inference

el[predicted_label]
'beignets'
  TensorFlow  Hide TensorFlow content  Load an image processor to preprocess the image and return the input as TensorFlow tensors:   Copied >>> from transformers import AutoImageProcessor
>>> image_processor = AutoImageProcessor.from_pretrained("MariaK/food_classifier")
>>> inputs = image_processor(image, return_tensors="tf") Pass your inputs to the model and return the logits:   Copied >>> from transformers import TFAutoModelForImageClassification
>>> model = TFAutoModelForImageClassification.from_pretrained("MariaK/food_classifier")
>>> logits = model(**inputs).logits Get the predicted label with the highest probability, and use the model’s id2label mapping to convert it to a label:   Copied >>> predicted_class_id = int(tf.math.argmax(logits, axis=-1)[0])
>>> model.config.id2label[predicted_class_id]
'beignets


Monocular depth estimation

Monocular depth estimation is a computer vision task that involves predicting the depth information of a scene from a
single image. In other words, it is the process of estimating the distance of objects in a scene from
a single camera viewpoint.
Monocular depth estimation has various applications, including 3D reconstruction, augmented reality, autonomous driving,
and robotics. It is a challenging task as it requires the model to understand the complex relationships between objects
in the scene and the corresponding depth information, which can be affected by factors such as lighting conditions,
occlusion, and texture.
The task illustrated in this tutorial is supported by the following model architectures:
 DPT, GLPN
In this guide you’ll learn how to:
create a depth estimation pipeline run depth estimation inference by hand
Before you begin, make sure you have all the necessary libraries installed:
  Copied pip install -q transformers


Depth estimation pipeline

The simplest way to try out inference with a model supporting depth estimation is to use the corresponding pipeline().
Instantiate a pipeline from a checkpoint on the Hugging Face Hub:
  Copied >>> from transformers import pipeline
>>> checkpoint = "vinvino02/glpn-nyu"
>>> depth_estimator = pipeline("depth-estimation", model=checkpoint)
Next, choose an image to analyze:
  Copied >>> from PIL import Image
>>> import requests
>>> url = "https://unsplash.com/photos/HwBAsSbPBDU/download?ixid=MnwxMjA3fDB8MXxzZWFyY2h8MzR8fGNhciUyMGluJTIwdGh


Depth estimation pipeline

lJTIwc3RyZWV0fGVufDB8MHx8fDE2Nzg5MDEwODg&force=true&w=640"
>>> image = Image.open(requests.get(url, stream=True).raw)
>>> image
Pass the image to the pipeline.
  Copied >>> predictions = depth_estimator(image)
The pipeline returns a dictionary with two entries. The first one, called predicted_depth, is a tensor with the values
being the depth expressed in meters for each pixel.
The second one, depth, is a PIL image that visualizes the depth estimation result.
Let’s take a look at the visualized result:
  Copied >>> predictions["depth"


Depth estimation inference by hand

Now that you’ve seen how to use the depth estimation pipeline, let’s see how we can replicate the same result by hand.
Start by loading the model and associated processor from a checkpoint on the Hugging Face Hub.
Here we’ll use the same checkpoint as before:
  Copied >>> from transformers import AutoImageProcessor, AutoModelForDepthEstimation
>>> checkpoint = "vinvino02/glpn-nyu"
>>> image_processor = AutoImageProcessor.from_pretrained(checkpoint)
>>> model = AutoModelForDepthEstimation.from_pretrained(checkpoint)
Prepare the image input for the model using the image_processor that will take care of the necessary image transformations
such as resizing and normalization:
  Cop


Depth estimation inference by hand

ied >>> pixel_values = image_processor(image, return_tensors="pt").pixel_values
Pass the prepared inputs through the model:
  Copied >>> import torch
>>> with torch.no_grad():
...     outputs = model(pixel_values)
...     predicted_depth = outputs.predicted_depth
Visualize the results:
  Copied >>> import numpy as np
>>> # interpolate to original size
>>> prediction = torch.nn.functional.interpolate(
...     predicted_depth.unsqueeze(1),
...     size=image.size[::-1],
...     mode="bicubic",
...     align_corners=False,
... ).squeeze()
>>> output = prediction.numpy()
>>> formatted = (output * 255 / np.max(output)).astype("uint8")
>>> depth = Image.fromarray(formatted)
>>> dept


Causal language modeling

There are two types of language modeling, causal and masked. This guide illustrates causal language modeling.
Causal language models are frequently used for text generation. You can use these models for creative applications like
choosing your own text adventure or an intelligent coding assistant like Copilot or CodeParrot.
Causal language modeling predicts the next token in a sequence of tokens, and the model can only attend to tokens on
the left. This means the model cannot see future tokens. GPT-2 is an example of a causal language model.
This guide will show you how to:
Finetune DistilGPT2 on the r/askscience subset of the ELI5 dataset. Use your finetuned model for inference.
You can finetune other architectures for causal language modeling following the same steps in this guide.
Choose one of the following architectures:
 BART, BERT, Bert Generation, BigBird, BigBird-Pegasus, BioGpt


Causal language modeling

, Blenderbot, BlenderbotSmall, BLOOM, CamemBERT, CodeLlama, CodeGen, CPM-Ant, CTRL, Data2VecText, ELECTRA, ERNIE, Falcon, GIT, GPT-Sw3, OpenAI GPT-2, GPTBigCode, GPT Neo, GPT NeoX, GPT NeoX Japanese, GPT-J, LLaMA, Marian, mBART, MEGA, Megatron-BERT, Mistral, MPT, MusicGen, MVP, OpenLlama, OpenAI GPT, OPT, Pegasus, Persimmon, PLBart, ProphetNet, QDQBert, Reformer, RemBERT, RoBERTa, RoBERTa-PreLayerNorm, RoCBert, RoFormer, RWKV, Speech2Text2, Transformer-XL, TrOCR, XGLM, XLM, XLM-ProphetNet, XLM-RoBERTa, XLM-RoBERTa-XL, XLNet, X-MOD
Before you begin, make sure you have all the necessary libraries installed:
  Copied pip install transformers datasets evaluate
We encourage you to log in to your Hugging Face account so you can upload and share your model with the community. When prompted, enter your token to log in:
  Copied >>> from huggingface_hub import notebook_login
>>> notebook_login()


Load ELI5 dataset

Start by loading a smaller subset of the r/askscience subset of the ELI5 dataset from the 🤗 Datasets library.
This’ll give you a chance to experiment and make sure everything works before spending more time training on the full dataset.
  Copied >>> from datasets import load_dataset
>>> eli5 = load_dataset("eli5", split="train_asks[:5000]")
Split the dataset’s train_asks split into a train and test set with the train_test_split method:
  Copied >>> eli5 = eli5.train_test_split(test_size=0.2)
Then take a look at an example:
  Copied >>> eli5["train"][0]
{'answers': {'a_id': ['c3d1aib', 'c3d4lya'],
  'score': [6, 3],
  'text': ["The velocity needed to remain in orbit is equal to the square root of Newton's constant times the mass of eart


Load ELI5 dataset

h divided by the distance from the center of the earth. I don't know the altitude of that specific mission, but they're usually around 300 km. That means he's going 7-8 km/s.\n\nIn space there are no other forces acting on either the shuttle or the guy, so they stay in the same position relative to each other. If he were to become unable to return to the ship, he would presumably run out of oxygen, or slowly fall into the atmosphere and burn up.",
   "Hope you don't mind me asking another question, but why aren't there any stars visible in this photo?"]},
 'answers_urls': {'url': []},
 'document': '',
 'q_id': 'nyxfp',
 'selftext': '_URL_0_\n\nThis was on the front page earlier and I have a few questions about it. Is it possible to cal