TrainerCallback

class transformers.TrainerCallback  < source > ( )   Parameters   args (TrainingArguments) —
The training arguments used to instantiate the Trainer.   state (TrainerState) —
The current state of the Trainer.   control (TrainerControl) —
The object that is returned to the Trainer and can be used to make some decisions.   model (PreTrainedModel or torch.nn.Module) —
The model being trained.   tokenizer (PreTrainedTokenizer) —
The tokenizer used for encoding the data.   optimizer (torch.optim.Optimizer) —
The optimizer used for the training steps.   lr_scheduler (torch.optim.lr_scheduler.LambdaLR) —
The scheduler used for setting the learning rate.   train_dataloader (torch.utils.data.DataLoader, optional) —
The current dataloader used for training.   eval_dataloader (torch.utils.data.DataLoader, optional) —
The current dataloader used for training.   metrics (Dict[str, float]) —
The metrics computed by the last evaluation phase.
Those are only accessible


TrainerCallback

in the event on_evaluate.   logs  (Dict[str, float]) —
The values to log.
Those are only accessible in the event on_log.    A class for objects that will inspect the state of the training loop at some events and take some decisions. At
each of those events the following arguments are available: The control object is the only one that can be changed by the callback, in which case the event that changes it
should return the modified version. The argument args, state and control are positionals for all events, all the others are grouped in kwargs.
You can unpack the ones you need in the signature of the event using them. As an example, see the code of the
simple ~transformer.PrinterCallback.  Example:   Copied class PrinterCallback(TrainerCallback):
    def on_log(self, args, state, control, logs=None, **kwargs):
        _ = logs.pop("total_flos", None)
        if state.is_local_process_zero:
            print(logs)  on_epoch_begin  < source > ( args: Trai


TrainerCallback

ningArguments state: TrainerState control: TrainerControl **kwargs  )    Event called at the beginning of an epoch.  on_epoch_end  < source > ( args: TrainingArguments state: TrainerState control: TrainerControl **kwargs  )    Event called at the end of an epoch.  on_evaluate  < source > ( args: TrainingArguments state: TrainerState control: TrainerControl **kwargs  )    Event called after an evaluation phase.  on_init_end  < source > ( args: TrainingArguments state: TrainerState control: TrainerControl **kwargs  )    Event called at the end of the initialization of the Trainer.  on_log  < source > ( args: TrainingArguments state: TrainerState control: TrainerControl **kwargs  )    Event called after logging the last logs.  on_predict  < source > ( args: TrainingArguments state: TrainerState control: TrainerControl metrics **kwargs  )    Event called after a successful prediction.  on_prediction_step  < source > ( args: TrainingArguments state: TrainerS


TrainerCallback

tate control: TrainerControl **kwargs  )    Event called after a prediction step.  on_save  < source > ( args: TrainingArguments state: TrainerState control: TrainerControl **kwargs  )    Event called after a checkpoint save.  on_step_begin  < source > ( args: TrainingArguments state: TrainerState control: TrainerControl **kwargs  )    Event called at the beginning of a training step. If using gradient accumulation, one training step might take
several inputs.  on_step_end  < source > ( args: TrainingArguments state: TrainerState control: TrainerControl **kwargs  )    Event called at the end of a training step. If using gradient accumulation, one training step might take
several inputs.  on_substep_end  < source > ( args: TrainingArguments state: TrainerState control: TrainerControl **kwargs  )    Event called at the end of an substep during gradient accumulation.  on_train_begin  < source > ( args: TrainingArguments state: TrainerState control: Trainer


TrainerCallback

Control **kwargs  )    Event called at the beginning of training.  on_train_end  < source > ( args: TrainingArguments state: TrainerState control: TrainerControl **kwargs  )    Event called at the end of training.
Here is an example of how to register a custom callback with the PyTorch Trainer:
  Copied class MyCallback(TrainerCallback):
    "A callback that prints a message at the beginning of training"
    def on_train_begin(self, args, state, control, **kwargs):
        print("Starting training")
trainer = Trainer(
    model,
    args,
    train_dataset=train_dataset,
    eval_dataset=eval_dataset,
    callbacks=[MyCallback],  # We can either pass the callback class this way or an instance of it (MyCallback())
)
Another way to register a callback is to call trainer.add_callback() as follows:
  Copied trainer = Trainer(...)
trainer.add_callback(MyCallback)
# Alternatively, we can pass an instance of the callback class
trainer.add_callback(MyCallback()


TrainerState

class transformers.TrainerState  < source > ( epoch: typing.Optional[float] = None global_step: int = 0 max_steps: int = 0 logging_steps: int = 500 eval_steps: int = 500 save_steps: int = 500 num_train_epochs: int = 0 total_flos: float = 0 log_history: typing.List[typing.Dict[str, float]] = None best_metric: typing.Optional[float] = None best_model_checkpoint: typing.Optional[str] = None is_local_process_zero: bool = True is_world_process_zero: bool = True is_hyper_param_search: bool = False trial_name: str = None trial_params: typing.Dict[str, typing.Union[str, float, int, bool]] = None  )   Parameters   epoch (float, optional) —
Only set during training, will represent the epoch the training is at (the decimal part being the
percentage of the curre


TrainerState

nt epoch completed).   global_step (int, optional, defaults to 0) —
During training, represents the number of update steps completed.   max_steps (int, optional, defaults to 0) —
The number of update steps to do during the current training.   logging_steps (int, optional, defaults to 500) —
Log every X updates steps   eval_steps (int, optional) —
Run an evaluation every X steps.   save_steps (int, optional, defaults to 500) —
Save checkpoint every X updates steps.   total_flos (float, optional, defaults to 0) —
The total number of floating operations done by the model since the beginning of training (stored as floats
to avoid overflow).   log_history (List[Dict[str, float]], optional) —
The list of logs done since the beginning of training.   best_me


TrainerState

tric (float, optional) —
When tracking the best model, the value of the best metric encountered so far.   best_model_checkpoint (str, optional) —
When tracking the best model, the value of the name of the checkpoint for the best model encountered so
far.   is_local_process_zero (bool, optional, defaults to True) —
Whether or not this process is the local (e.g., on one machine if training in a distributed fashion on
several machines) main process.   is_world_process_zero (bool, optional, defaults to True) —
Whether or not this process is the global main process (when training in a distributed fashion on several
machines, this is only going to be True for one process).   is_hyper_param_search (bool, optional, defaults to False) —
Whether we are in the


TrainerState

process of a hyper parameter search using Trainer.hyperparameter_search. This will
impact the way data will be logged in TensorBoard.    A class containing the Trainer inner state that will be saved along the model and optimizer when checkpointing
and passed to the TrainerCallback. In all this class, one step is to be understood as one update step. When using gradient accumulation, one update
step may require several forward and backward passes: if you use gradient_accumulation_steps=n, then one update
step requires going through n batches.  load_from_json  < source > ( json_path: str  )    Create an instance from the content of json_path.  save_to_json  < source > ( json_path: str  )    Save the content of this instance in JSON format inside json_pa


TrainerControl

class transformers.TrainerControl  < source > ( should_training_stop: bool = False should_epoch_stop: bool = False should_save: bool = False should_evaluate: bool = False should_log: bool = False  )   Parameters   should_training_stop (bool, optional, defaults to False) —
Whether or not the training should be interrupted.
If True, this variable will not be set back to False. The training will just stop.   should_epoch_stop (bool, optional, defaults to False) —
Whether or not the current epoch should be interrupted.
If True, this variable will be set back to False at the beginning of the next epoch.   should_save (bool, optional, defaults to False) —
Wheth


TrainerControl

er or not the model should be saved at this step.
If True, this variable will be set back to False at the beginning of the next step.   should_evaluate (bool, optional, defaults to False) —
Whether or not the model should be evaluated at this step.
If True, this variable will be set back to False at the beginning of the next step.   should_log (bool, optional, defaults to False) —
Whether or not the logs should be reported at this step.
If True, this variable will be set back to False at the beginning of the next step.    A class that handles the Trainer control flow. This class is used by the TrainerCallback to activate some
switches in the training loop


Pipelines

The pipelines are a great and easy way to use models for inference. These pipelines are objects that abstract most of
the complex code from the library, offering a simple API dedicated to several tasks, including Named Entity
Recognition, Masked Language Modeling, Sentiment Analysis, Feature Extraction and Question Answering. See the
task summary for examples of use.
There are two categories of pipeline abstractions to be aware about:
The pipeline() which is the most powerful object encapsulating all other pipelines. Task-specific pipelines are available for audio, computer vision, natural language processing, and multimodal tasks.


The pipeline abstraction

The pipeline abstraction is a wrapper around all the other available pipelines. It is instantiated as any other
pipeline but can provide additional quality of life.
Simple call on one item:
  Copied >>> pipe = pipeline("text-classification")
>>> pipe("This restaurant is awesome")
[{'label': 'POSITIVE', 'score': 0.9998743534088135}]
If you want to use a specific model from the hub you can ignore the task if the model on
the hub already defines it:
  Copied >>> pipe = pipeline(model="roberta-large-mnli")
>>> pipe("This restaurant is awesome")
[{'label': 'NEUTRAL', 'score': 0.7313136458396912}]
To call a pipeline on many items, you can call it with a list.
  Copied >>> pipe = pipeline("text-classification")
>>> pipe(["This restaurant is awesome", "This restaurant is awful"])
[{'label': 'POSITIVE', 'score': 0.9998743534088135},
 {'label': 'NEGATIVE', 'score': 0.9996669292449951}]
To iterate over full datasets it is recommended to


The pipeline abstraction

use a dataset directly. This means you don’t need to allocate
the whole dataset at once, nor do you need to do batching yourself. This should work just as fast as custom loops on
GPU. If it doesn’t don’t hesitate to create an issue.
  Copied import datasets
from transformers import pipeline
from transformers.pipelines.pt_utils import KeyDataset
from tqdm.auto import tqdm
pipe = pipeline("automatic-speech-recognition", model="facebook/wav2vec2-base-960h", device=0)
dataset = datasets.load_dataset("superb", name="asr", split="test")
# KeyDataset (only *pt*) will simply return the item in the dict returned by the dataset item
# as we're not interested in the *target* part of the dataset. For sentence pair use KeyPairDataset
for out in tqdm(pipe(KeyDataset(dataset, "file"))):
    print(out)
    # {"text": "NUMBER TEN FRESH NELLY IS WAITING ON YOU GOOD NIGHT HUSBAND"}
    # {"text": ....}
    # ....
For ease of use, a generator is


The pipeline abstraction

also possible:
  Copied from transformers import pipeline
pipe = pipeline("text-classification")
def data():
    while True:
        # This could come from a dataset, a database, a queue or HTTP request
        # in a server
        # Caveat: because this is iterative, you cannot use `num_workers > 1` variable
        # to use multiple threads to preprocess data. You can still have 1 thread that
        # does the preprocessing while the main runs the big inference
        yield "This is a test"
for out in pipe(data()):
    print(out)
    # {"text": "NUMBER TEN FRESH NELLY IS WAITING ON YOU GOOD NIGHT HUSBAND"}
    # {"text": ....}
    # ....
 transformers.pipeline  < source > ( task: str = None model: typing.Union[str, ForwardRef('PreTrainedModel'), ForwardRef('TFPreTrainedModel'), NoneType] = None config: typing.Union[str, transformers.configuration_utils.PretrainedConfig, NoneType] = None tokenizer: typing.Union[str, transf


The pipeline abstraction

ormers.tokenization_utils.PreTrainedTokenizer, ForwardRef('PreTrainedTokenizerFast'), NoneType] = None feature_extractor: typing.Union[str, ForwardRef('SequenceFeatureExtractor'), NoneType] = None image_processor: typing.Union[str, transformers.image_processing_utils.BaseImageProcessor, NoneType] = None framework: typing.Optional[str] = None revision: typing.Optional[str] = None use_fast: bool = True token: typing.Union[bool, str, NoneType] = None device: typing.Union[int, str, ForwardRef('torch.device'), NoneType] = None device_map = None torch_dtype = None trust_remote_code: typing.Optional[bool] = None model_kwargs: typing.Dict[str, typing.Any] = None pipeline_class: typing.Optional[typing.Any] = None **kwargs  ) → Pipeline  Parameters   task (str) —
The task defining which pipeline will be returned. Currently accepted tasks are:
"audio-classification": will return a AudioClassificationPipeline.
"automatic-speech-recognitio


The pipeline abstraction

n": will return a AutomaticSpeechRecognitionPipeline.
"conversational": will return a ConversationalPipeline.
"depth-estimation": will return a DepthEstimationPipeline.
"document-question-answering": will return a DocumentQuestionAnsweringPipeline.
"feature-extraction": will return a FeatureExtractionPipeline.
"fill-mask": will return a FillMaskPipeline:.
"image-classification": will return a ImageClassificationPipeline.
"image-segmentation": will return a ImageSegmentationPipeline.
"image-to-image": will return a ImageToImagePipeline.
"image-to-text": will return a ImageToTextPipeline.
"mask-generation": will return a MaskGenerationPipeline.
"object-detection": will return a ObjectDetectionPipeline.
"question-answering": will return a QuestionAnsweringPipeline.
"summarization": will return a SummarizationPipeline.
"table-question-answering": will return a TableQuestionAnsweringPipeline.
"text2text-generation": will return a T


The pipeline abstraction

ext2TextGenerationPipeline.
"text-classification" (alias "sentiment-analysis" available): will return a
TextClassificationPipeline.
"text-generation": will return a TextGenerationPipeline:.
"text-to-audio" (alias "text-to-speech" available): will return a TextToAudioPipeline:.
"token-classification" (alias "ner" available): will return a TokenClassificationPipeline.
"translation": will return a TranslationPipeline.
"translation_xx_to_yy": will return a TranslationPipeline.
"video-classification": will return a VideoClassificationPipeline.
"visual-question-answering": will return a VisualQuestionAnsweringPipeline.
"zero-shot-classification": will return a ZeroShotClassificationPipeline.
"zero-shot-image-classification": will return a ZeroShotImageClassificationPipeline.
"zero-shot-audio-classification": will return a ZeroShotAudioClassificationPipeline.
"zero-shot-object-detection": will return a ZeroShotObjectDetectionPipeline


The pipeline abstraction

.
   model (str or PreTrainedModel or TFPreTrainedModel, optional) —
The model that will be used by the pipeline to make predictions. This can be a model identifier or an
actual instance of a pretrained model inheriting from PreTrainedModel (for PyTorch) or
TFPreTrainedModel (for TensorFlow).
If not provided, the default for the task will be loaded.   config (str or PretrainedConfig, optional) —
The configuration that will be used by the pipeline to instantiate the model. This can be a model
identifier or an actual pretrained model configuration inheriting from PretrainedConfig.
If not provided, the default configuration file for the requested model will be used. That means that if
model is given, its default configuration will be used. However, if model is not supplied, this
task’s default model’s config is used instead.   tokenizer (str or PreTrainedTokenizer, optional) —
The tokenizer that will be used by the pipeline to en


The pipeline abstraction

code data for the model. This can be a model
identifier or an actual pretrained tokenizer inheriting from PreTrainedTokenizer.
If not provided, the default tokenizer for the given model will be loaded (if it is a string). If model
is not specified or not a string, then the default tokenizer for config is loaded (if it is a string).
However, if config is also not given or not a string, then the default tokenizer for the given task
will be loaded.   feature_extractor (str or PreTrainedFeatureExtractor, optional) —
The feature extractor that will be used by the pipeline to encode data for the model. This can be a model
identifier or an actual pretrained feature extractor inheriting from PreTrainedFeatureExtractor.
Feature extractors are used for non-NLP models, such as Speech or Vision models as well as multi-modal
models. Multi-modal models will also require a tokenizer to be passed.
If not provided, the default feature extracto


The pipeline abstraction

r for the given model will be loaded (if it is a string). If
model is not specified or not a string, then the default feature extractor for config is loaded (if it
is a string). However, if config is also not given or not a string, then the default feature extractor
for the given task will be loaded.   framework (str, optional) —
The framework to use, either "pt" for PyTorch or "tf" for TensorFlow. The specified framework must be
installed.
If no framework is specified, will default to the one currently installed. If no framework is specified and
both frameworks are installed, will default to the framework of the model, or to PyTorch if no model is
provided.   revision (str, optional, defaults to "main") —
When passing a task name or a string model identifier: The specific model version to use. It can be a
branch name, a tag name, or a commit id, since we use a git-based system for storing models and other
artifacts on hugging


The pipeline abstraction

face.co, so revision can be any identifier allowed by git.   use_fast (bool, optional, defaults to True) —
Whether or not to use a Fast tokenizer if possible (a PreTrainedTokenizerFast).   use_auth_token (str or bool, optional) —
The token to use as HTTP bearer authorization for remote files. If True, will use the token generated
when running huggingface-cli login (stored in ~/.huggingface).   device (int or str or torch.device) —
Defines the device (e.g., "cpu", "cuda:1", "mps", or a GPU ordinal rank like 1) on which this
pipeline will be allocated.   device_map (str or Dict[str, Union[int, str, torch.device], optional) —
Sent directly as model_kwargs (just a simpler shortcut). When accelerate library is present, set
device_map="auto" to compute the most optimized device_map automatically (see
here
for more information).
Do not use device_map AND device at the same time as they will conflict
   torch_dtype (str or torch.dtype


The pipeline abstraction

, optional) —
Sent directly as model_kwargs (just a simpler shortcut) to use the available precision for this model
(torch.float16, torch.bfloat16, … or "auto").   trust_remote_code (bool, optional, defaults to False) —
Whether or not to allow for custom code defined on the Hub in their own modeling, configuration,
tokenization or even pipeline files. This option should only be set to True for repositories you trust
and in which you have read the code, as it will execute code present on the Hub on your local machine.   model_kwargs (Dict[str, Any], optional) —
Additional dictionary of keyword arguments passed along to the model’s from_pretrained(..., **model_kwargs) function.   kwargs (Dict[str, Any], optional) —
Additional keyword arguments passed along to the specific pipeline init (see the documentation for the
corresponding pipeline class for possible values).   Returns
Pipeline

A suitable pipeline for the task.
  Util


The pipeline abstraction

ity factory method to build a Pipeline. Pipelines are made of: A tokenizer in charge of mapping raw textual input to token. A model to make predictions from the inputs. Some (optional) post processing for enhancing model’s output.  Examples:   Copied >>> from transformers import pipeline, AutoModelForTokenClassification, AutoTokenizer
>>> # Sentiment analysis pipeline
>>> analyzer = pipeline("sentiment-analysis")
>>> # Question answering pipeline, specifying the checkpoint identifier
>>> oracle = pipeline(
...     "question-answering", model="distilbert-base-cased-distilled-squad", tokenizer="bert-base-cased"
... )
>>> # Named entity recognition pipeline, passing in a specific model and tokenizer
>>> model = AutoModelForTokenClassification.from_pretrained("dbmdz/bert-large-cased-finetuned-conll03-english")
>>> tokenizer = AutoTokenizer.from_pretrained("bert-base-cased")
>>> recognizer = pipeline("ner", model=model, tokenizer=t


Pipeline batching

All pipelines can use batching. This will work
whenever the pipeline uses its streaming ability (so when passing lists or Dataset or generator).
  Copied from transformers import pipeline
from transformers.pipelines.pt_utils import KeyDataset
import datasets
dataset = datasets.load_dataset("imdb", name="plain_text", split="unsupervised")
pipe = pipeline("text-classification", device=0)
for out in pipe(KeyDataset(dataset, "text"), batch_size=8, truncation="only_first"):
    print(out)
    # [{'label': 'POSITIVE', 'score': 0.9998743534088135}]
    # Exactly the same output as before, but the content are passed
    # as batches to the model
However, this is not automatically a win for performance. It can be either a 10x speedup or 5x slowdown depending
on hardware, data and the actual model being used. Example where it’s mostly a speedup:
  Copied from transformers import pipeline
from torch.utils.data import Dataset
from tqdm.auto import


Pipeline batching

tqdm
pipe = pipeline("text-classification", device=0)
class MyDataset(Dataset):
    def __len__(self):
        return 5000
    def __getitem__(self, i):
        return "This is a test"
dataset = MyDataset()
for batch_size in [1, 8, 64, 256]:
    print("-" * 30)
    print(f"Streaming batch_size={batch_size}")
    for out in tqdm(pipe(dataset, batch_size=batch_size), total=len(dataset)):
        pass
  Copied # On GTX 970
------------------------------
Streaming no batching
100%|██████████████████████████████████████████████████████████████████████| 5000/5000 [00:26<00:00, 187.52it/s]
------------------------------
Streaming batch_size=8
100%|█████████████████████████████████████████████████████████████████████| 5000/5000 [00:04<00:00, 1205.95it/s]
------------------------------
Streaming batch_size=64
100%|█████████████████████████████████████████████████████████████████████| 5000/5000 [00:02<00:00, 2478.24it/s]
-------------------------


Pipeline batching

-----
Streaming batch_size=256
100%|█████████████████████████████████████████████████████████████████████| 5000/5000 [00:01<00:00, 2554.43it/s]
(diminishing returns, saturated the GPU)
Example where it’s most a slowdown:
  Copied class MyDataset(Dataset):
    def __len__(self):
        return 5000
    def __getitem__(self, i):
        if i % 64 == 0:
            n = 100
        else:
            n = 1
        return "This is a test" * n
This is a occasional very long sentence compared to the other. In that case, the whole batch will need to be 400
tokens long, so the whole batch will be [64, 400] instead of [64, 4], leading to the high slowdown. Even worse, on
bigger batches, the program simply crashes.
  Copied ------------------------------
Streaming no batching
100%|█████████████████████████████████████████████████████████████████████| 1000/1000 [00:05<00:00, 183.69it/s]
------------------------------
Streaming batch_size=8
100%|████


Pipeline batching

█████████████████████████████████████████████████████████████████| 1000/1000 [00:03<00:00, 265.74it/s]
------------------------------
Streaming batch_size=64
100%|██████████████████████████████████████████████████████████████████████| 1000/1000 [00:26<00:00, 37.80it/s]
------------------------------
Streaming batch_size=256
  0%|                                                                                | 0/1000 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/home/nicolas/src/transformers/test.py", line 42, in <module>
    for out in tqdm(pipe(dataset, batch_size=256), total=len(dataset)):
....
    q = q / math.sqrt(dim_per_head)  # (bs, n_heads, q_length, dim_per_head)
RuntimeError: CUDA out of memory. Tried to allocate 376.00 MiB (GPU 0; 3.95 GiB total capacity; 1.72 GiB already allocated; 354.88 MiB free; 2.46 GiB reserved in total by PyTorch)
There are no good (general) solutions for this problem, and your mileage


Pipeline batching

may vary depending on your use cases. Rule of
thumb:
For users, a rule of thumb is:
Measure performance on your load, with your hardware. Measure, measure, and keep measuring. Real numbers are the
only way to go. If you are latency constrained (live product doing inference), don’t batch If you are using CPU, don’t batch. If you are using throughput (you want to run your model on a bunch of static data), on GPU, then: If you have no clue about the size of the sequence_length (“natural” data), by default don’t batch, measure and
try tentatively to add it, add OOM checks to recover when it will fail (and it will at some point if you don’t
control the sequence_length.) If your sequence_length is super regular, then batching is more likely to be VERY interesting, measure and push
it until you get OOMs. The larger the GPU the more likely batching is going to be more interesting As soon as you enable batching, make sure you can handle OOMs nic


Pipeline chunk batching

zero-shot-classification and question-answering are slightly specific in the sense, that a single input might yield
multiple forward pass of a model. Under normal circumstances, this would yield issues with batch_size argument.
In order to circumvent this issue, both of these pipelines are a bit specific, they are ChunkPipeline instead of
regular Pipeline. In short:
  Copied preprocessed = pipe.preprocess(inputs)
model_outputs = pipe.forward(preprocessed)
outputs = pipe.postprocess(model_outputs)
Now becomes:
  Copied all_model_outputs = []
for preprocess


Pipeline chunk batching

ed in pipe.preprocess(inputs):
    model_outputs = pipe.forward(preprocessed)
    all_model_outputs.append(model_outputs)
outputs = pipe.postprocess(all_model_outputs)
This should be very transparent to your code because the pipelines are used in
the same way.
This is a simplified view, since the pipeline can handle automatically the batch to ! Meaning you don’t have to care
about how many forward passes you inputs are actually going to trigger, you can optimize the batch_size
independently of the inputs. The caveats from the previous section still apply.


Pipeline custom code

If you want to override a specific pipeline.
Don’t hesitate to create an issue for your task at hand, the goal of the pipeline is to be easy to use and support most
cases, so transformers could maybe support your use case.
If you want to try simply you can:
Subclass your pipeline of choice
  Copied class MyPipeline(TextClassificationPipeline):
    def postprocess():
        # Your code goes here
        scores = scores * 100
        # And here
my_pipeline = MyPipeline(model=model, tokenizer=tokenizer, ...)
# or if you use *pipeline* function, then:
my_pipeline = pipeline(model="xxxx", pipeline_class=MyPipeline)
That should enable you to do all the custom code you want.


AudioClassificationPipeline

class transformers.AudioClassificationPipeline  < source > ( *args **kwargs  )   Parameters   model (PreTrainedModel or TFPreTrainedModel) —
The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from
PreTrainedModel for PyTorch and TFPreTrainedModel for TensorFlow.   tokenizer (PreTrainedTokenizer) —
The tokenizer that will be used by the pipeline to encode data for the model. This object inherits from
PreTrainedTokenizer.   modelcard (str or ModelCard, optional) —
Model card attributed to the model for this pipeline.   framework (str, optional) —
The framework to use, either "pt" for PyTorch or "tf" for TensorFlow. The specified framework must be
installed.
If no framework is specified, will default to the one currently installed. If no framework is specified and
both frameworks are installe


AudioClassificationPipeline

d, will default to the framework of the model, or to PyTorch if no model is
provided.   task (str, defaults to "") —
A task-identifier for the pipeline.   num_workers (int, optional, defaults to 8) —
When the pipeline will use DataLoader (when passing a dataset, on GPU for a Pytorch model), the number of
workers to be used.   batch_size (int, optional, defaults to 1) —
When the pipeline will use DataLoader (when passing a dataset, on GPU for a Pytorch model), the size of
the batch to use, for inference this is not always beneficial, please read Batching with
pipelines .   args_parser (ArgumentHandler, optional) —
Reference to the object in charge of parsing supplied pipeline parameters.   device (int, optional, defaults to -1) —
Device ordinal for CPU/GPU supports. Setting this to -1 will leverage CPU, a positive will run the model on
the


AudioClassificationPipeline

 associated CUDA device id. You can pass native torch.device or a str too.   binary_output (bool, optional, defaults to False) —
Flag indicating if the output the pipeline should happen in a binary format (i.e., pickle) or as raw text.    Audio classification pipeline using any AutoModelForAudioClassification. This pipeline predicts the class of a
raw waveform or an audio file. In case of an audio file, ffmpeg should be installed to support multiple audio
formats.  Example:   Copied >>> from transformers import pipeline
>>> classifier = pipeline(model="superb/wav2vec2-base-superb-ks")
>>> classifier("https://huggingface.co/datasets/Narsil/asr_dummy/resolve/main/1.flac")
[{'score': 0.997, 'label': '_unknown_'}, {'score': 0.002, 'label': 'left'}, {'score': 0.0, 'label': 'yes'}, {'score': 0.0, 'label': 'down'}, {'score': 0.0, 'label': 'stop'


AudioClassificationPipeline

}] Learn more about the basics of using a pipeline in the pipeline tutorial This pipeline can currently be loaded from pipeline() using the following task identifier:
"audio-classification". See the list of available models on
huggingface.co/models.  __call__  < source > ( inputs: typing.Union[numpy.ndarray, bytes, str] **kwargs  ) → A list of dict with the following keys  Parameters   inputs (np.ndarray or bytes or str or dict) —
The inputs is either :
str that is the filename of the audio file, the file will be read at the correct sampling rate
to get the waveform using ffmpeg. This requires ffmpeg to be installed on the system.
bytes it is supposed to be the content of an audio file and is interpreted by ffmpeg in the
same way.
(np.ndarray of shape (n, ) of type np.float32 or np.float64)
Raw audio at the correct sampling rate (no furth


AudioClassificationPipeline

er check will be done)
dict form can be used to pass raw audio sampled at arbitrary sampling_rate and let this
pipeline do the resampling. The dict must be either be in the format {"sampling_rate": int, "raw": np.array}, or {"sampling_rate": int, "array": np.array}, where the key "raw" or
"array" is used to denote the raw audio waveform.
   top_k (int, optional, defaults to None) —
The number of top labels that will be returned by the pipeline. If the provided number is None or
higher than the number of labels available in the model configuration, it will default to the number of
labels.   Returns
A list of dict with the following keys

label (str) — The label predicted.
score (float) — The corresponding probability.
  Classify the sequence(s) given as inputs. See the AutomaticSpeechRecognitionPipeline documentation for more
informatio


AutomaticSpeechRecognitionPipeline

class transformers.AutomaticSpeechRecognitionPipeline  < source > ( model: PreTrainedModel feature_extractor: typing.Union[ForwardRef('SequenceFeatureExtractor'), str] = None tokenizer: typing.Optional[transformers.tokenization_utils.PreTrainedTokenizer] = None decoder: typing.Union[ForwardRef('BeamSearchDecoderCTC'), str, NoneType] = None modelcard: typing.Optional[transformers.modelcard.ModelCard] = None framework: typing.Optional[str] = None task: str = '' args_parser: ArgumentHandler = None device: typing.Union[int, ForwardRef('torch.device')] = None torch_dtype: typing.Union[str, ForwardRef('torch.dtype'), NoneType] = None binary_output: bool = False **kwargs  )   Parameters   model (PreTrainedModel or TFPreTrainedModel) —
The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from
PreTrainedModel for PyTorch and TFPreTrainedModel for TensorFlow.   tokenizer (PreTrainedTokenizer) —
The tokenizer that will be used by the


AutomaticSpeechRecognitionPipeline

pipeline to encode data for the model. This object inherits from
PreTrainedTokenizer.   feature_extractor (SequenceFeatureExtractor) —
The feature extractor that will be used by the pipeline to encode waveform for the model.   chunk_length_s (float, optional, defaults to 0) —
The input length for in each chunk. If chunk_length_s = 0 then chunking is disabled (default).
For more information on how to effectively use chunk_length_s, please have a look at the ASR chunking
blog post.
   stride_length_s (float, optional, defaults to chunk_length_s / 6) —
The length of stride on the left and right of each chunk. Used only with chunk_length_s > 0. This enables
the model to see more context and infer letters better than without this context but the pipeline
discards the stride bits at the end to make the final reconstitution as perfect as possible.
For more information on how to effectively use stride_length_s, please have a look at the ASR chunking
blog post.
   framework (str,


AutomaticSpeechRecognitionPipeline

 optional) —
The framework to use, either "pt" for PyTorch or "tf" for TensorFlow. The specified framework must be
installed. If no framework is specified, will default to the one currently installed. If no framework is
specified and both frameworks are installed, will default to the framework of the model, or to PyTorch if
no model is provided.   device (Union[int, torch.device], optional) —
Device ordinal for CPU/GPU supports. Setting this to None will leverage CPU, a positive will run the
model on the associated CUDA device id.   decoder (pyctcdecode.BeamSearchDecoderCTC, optional) —
PyCTCDecode’s
BeamSearchDecoderCTC
can be passed for language model boosted decoding. See Wav2Vec2ProcessorWithLM for more information.    Pipeline that aims at extracting spoken text contained within some audio. The input can be either a raw waveform or a audio file. In case of the audio file, ffmpeg should be installed for
to support multiple audio formats  Example:   Copied >>> from tr


AutomaticSpeechRecognitionPipeline

ansformers import pipeline
>>> transcriber = pipeline(model="openai/whisper-base")
>>> transcriber("https://huggingface.co/datasets/Narsil/asr_dummy/resolve/main/1.flac")
{'text': ' He hoped there would be stew for dinner, turnips and carrots and bruised potatoes and fat mutton pieces to be ladled out in thick, peppered flour-fatten sauce.'} Learn more about the basics of using a pipeline in the pipeline tutorial  __call__  < source > ( inputs: typing.Union[numpy.ndarray, bytes, str] **kwargs  ) → Dict  Parameters   inputs (np.ndarray or bytes or str or dict) —
The inputs is either :
str that is either the filename of a local audio file, or a public URL address to download the
audio file. The file will be read at the correct sampling rate to get the waveform using
ffmpeg. This requires ffmpeg to be installed on the system.
bytes it is supposed to be the content of an audio file and is interpreted by ffmpeg in the
same way.
(np.ndarray of shape (n, ) of type np.float32 or


AutomaticSpeechRecognitionPipeline

 np.float64)
Raw audio at the correct sampling rate (no further check will be done)
dict form can be used to pass raw audio sampled at arbitrary sampling_rate and let this
pipeline do the resampling. The dict must be in the format {"sampling_rate": int, "raw": np.array} with optionally a "stride": (left: int, right: int) than can ask the pipeline to
treat the first left samples and last right samples to be ignored in decoding (but used at
inference to provide more context to the model). Only use stride with CTC models.
   return_timestamps (optional, str or bool) —
Only available for pure CTC models (Wav2Vec2, HuBERT, etc) and the Whisper model. Not available for
other sequence-to-sequence models.
For CTC models, timestamps can take one of two formats:
"char": the pipeline will return timestamps along the text for every character in the text. For
instance, if you get [{"text": "h", "timestamp": (0.5, 0.6)}, {"text": "i", "timestamp": (0.7, 0.9)}], then it means the model


AutomaticSpeechRecognitionPipeline

 predicts that the letter “h” was spoken after 0.5 and before
0.6 seconds.
"word": the pipeline will return timestamps along the text for every word in the text. For
instance, if you get [{"text": "hi ", "timestamp": (0.5, 0.9)}, {"text": "there", "timestamp": (1.0, 1.5)}], then it means the model predicts that the word “hi” was spoken after 0.5 and
before 0.9 seconds.
For the Whisper model, timestamps can take one of two formats:
"word": same as above for word-level CTC timestamps. Word-level timestamps are predicted
through the dynamic-time warping (DTW) algorithm, an approximation to word-level timestamps
by inspecting the cross-attention weights.
True: the pipeline will return timestamps along the text for segments of words in the text.
For instance, if you get [{"text": " Hi there!", "timestamp": (0.5, 1.5)}], then it means the
model predicts that the segment “Hi there!” was spoken after 0.5 and before 1.5 seconds.
Note that a segment of text refers to a sequence of


AutomaticSpeechRecognitionPipeline

 one or more words, rather than individual
words as with word-level timestamps.
   generate_kwargs (dict, optional) —
The dictionary of ad-hoc parametrization of generate_config to be used for the generation call. For a
complete overview of generate, check the following
guide.   max_new_tokens (int, optional) —
The maximum numbers of tokens to generate, ignoring the number of tokens in the prompt.   Returns
Dict

A dictionary with the following keys:
text (str): The recognized text.
chunks (optional(, List[Dict])
When using return_timestamps, the chunks will become a list containing all the various text
chunks identified by the model, e.g.* [{"text": "hi ", "timestamp": (0.5, 0.9)}, {"text": "there", "timestamp": (1.0, 1.5)}]. The original full text can roughly be recovered by doing
"".join(chunk["text"] for chunk in output["chunks"]).
  Transcribe the audio sequence(s) given as inputs to text. See the AutomaticSpeechRecognitionPipeline
documentation for more informat


TextToAudioPipeline

class transformers.TextToAudioPipeline  < source > ( *args vocoder = None sampling_rate = None **kwargs  )    Text-to-audio generation pipeline using any AutoModelForTextToWaveform or AutoModelForTextToSpectrogram. This
pipeline generates an audio file from an input text and optional other conditional inputs.  Example:   Copied >>> from transformers import pipeline
>>> pipe = pipeline(model="suno/bark-small")
>>> output = pipe("Hey it's HuggingFace on the phone!")
>>> audio = output["audio"]
>>> sampling_rate = output["sampling_rate"] Learn more about the basics of using a pipeline in the pipeline tutorial This pipeline can currently be loaded from pipeline() using the following task identifiers: "te


TextToAudioPipeline

xt-to-speech" or
"text-to-audio". See the list of available models on huggingface.co/models.  __call__  < source > ( text_inputs: typing.Union[str, typing.List[str]] **forward_params  ) → A dict or a list of dict  Parameters   text_inputs (str or List[str]) —
The text(s) to generate.   forward_params (optional) —
Parameters passed to the model generation/forward method.   Returns
A dict or a list of dict

The dictionaries have two keys:
audio (np.ndarray of shape (nb_channels, audio_length)) — The generated audio waveform.
sampling_rate (int) — The sampling rate of the generated audio waveform.
  Generates speech/audio from the inputs. See the TextToAudioPipeline documentation for more information


ZeroShotAudioClassificationPipeline

class transformers.ZeroShotAudioClassificationPipeline  < source > ( **kwargs  )   Parameters   model (PreTrainedModel or TFPreTrainedModel) —
The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from
PreTrainedModel for PyTorch and TFPreTrainedModel for TensorFlow.   tokenizer (PreTrainedTokenizer) —
The tokenizer that will be used by the pipeline to encode data for the model. This object inherits from
PreTrainedTokenizer.   modelcard (str or ModelCard, optional) —
Model card attributed to the model for this pipeline.   framework (str, optional) —
The framework to use, either "pt" for PyTorch or "tf" for TensorFlow. The specified framework must be
installed.
If no framework is specified, will default to the one currently installed. If no framework is specified and
both frameworks are installed, will default to the framework of the mod


ZeroShotAudioClassificationPipeline

el, or to PyTorch if no model is
provided.   task (str, defaults to "") —
A task-identifier for the pipeline.   num_workers (int, optional, defaults to 8) —
When the pipeline will use DataLoader (when passing a dataset, on GPU for a Pytorch model), the number of
workers to be used.   batch_size (int, optional, defaults to 1) —
When the pipeline will use DataLoader (when passing a dataset, on GPU for a Pytorch model), the size of
the batch to use, for inference this is not always beneficial, please read Batching with
pipelines .   args_parser (ArgumentHandler, optional) —
Reference to the object in charge of parsing supplied pipeline parameters.   device (int, optional, defaults to -1) —
Device ordinal for CPU/GPU supports. Setting this to -1 will leverage CPU, a positive will run the model on
the associated CUDA device id. You can pass native torch.device or a str too.   binary_outp


ZeroShotAudioClassificationPipeline

ut (bool, optional, defaults to False) —
Flag indicating if the output the pipeline should happen in a binary format (i.e., pickle) or as raw text.    Zero shot audio classification pipeline using ClapModel. This pipeline predicts the class of an audio when you
provide an audio and a set of candidate_labels.  Example:   Copied >>> from transformers import pipeline
>>> from datasets import load_dataset
>>> dataset = load_dataset("ashraq/esc50")
>>> audio = next(iter(dataset["train"]["audio"]))["array"]
>>> classifier = pipeline(task="zero-shot-audio-classification", model="laion/clap-htsat-unfused")
>>> classifier(audio, candidate_labels=["Sound of a dog", "Sound of vaccum cleaner"])
[{'score': 0.9996, 'label': 'Sound of a dog'}, {'score': 0.0004, 'label': 'Sound of vaccum cleaner'}] Learn more about the basics of using a pipeline in the pipeline tutorial This audio
classification pi


ZeroShotAudioClassificationPipeline

peline can currently be loaded from pipeline() using the following task identifier:
"zero-shot-audio-classification". See the list of available models on
huggingface.co/models.  __call__  < source > ( audios: typing.Union[numpy.ndarray, bytes, str] **kwargs  )   Parameters   audios (str, List[str], np.array or List[np.array]) —
The pipeline handles three types of inputs:
A string containing a http link pointing to an audio
A string containing a local path to an audio
An audio loaded in numpy
   candidate_labels (List[str]) —
The candidate labels for this audio   hypothesis_template (str, optional, defaults to "This is a sound of {}") —
The sentence used in cunjunction with candidate_labels to attempt the audio classification by
replacing the placeholder with the candidate_labels. Then likelihood is estimated by using
logits_per_audio    Assign labels to the audio(s) passed as inputs


DepthEstimationPipeline

class transformers.DepthEstimationPipeline  < source > ( *args **kwargs  )   Parameters   model (PreTrainedModel or TFPreTrainedModel) —
The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from
PreTrainedModel for PyTorch and TFPreTrainedModel for TensorFlow.   tokenizer (PreTrainedTokenizer) —
The tokenizer that will be used by the pipeline to encode data for the model. This object inherits from
PreTrainedTokenizer.   modelcard (str or ModelCard, optional) —
Model card attributed to the model for this pipeline.   framework (str, optional) —
The framework to use, either "pt" for PyTorch or "tf" for TensorFlow. The specified framework must be
installed.
If no framework is specified, will default to the one currently installed. If no framework is specified and
both frameworks are installed, will default to the framework of the model, or to PyTorch if no model is
provided.   task (


DepthEstimationPipeline

str, defaults to "") —
A task-identifier for the pipeline.   num_workers (int, optional, defaults to 8) —
When the pipeline will use DataLoader (when passing a dataset, on GPU for a Pytorch model), the number of
workers to be used.   batch_size (int, optional, defaults to 1) —
When the pipeline will use DataLoader (when passing a dataset, on GPU for a Pytorch model), the size of
the batch to use, for inference this is not always beneficial, please read Batching with
pipelines .   args_parser (ArgumentHandler, optional) —
Reference to the object in charge of parsing supplied pipeline parameters.   device (int, optional, defaults to -1) —
Device ordinal for CPU/GPU supports. Setting this to -1 will leverage CPU, a positive will run the model on
the associated CUDA device id. You can pass native torch.device or a str too.   binary_output (bool, optional, defaults to False) —
Flag indicating if the output the pipeline should happe


DepthEstimationPipeline

n in a binary format (i.e., pickle) or as raw text.    Depth estimation pipeline using any AutoModelForDepthEstimation. This pipeline predicts the depth of an image.  Example:   Copied >>> from transformers import pipeline
>>> depth_estimator = pipeline(task="depth-estimation", model="Intel/dpt-large")
>>> output = depth_estimator("http://images.cocodataset.org/val2017/000000039769.jpg")
>>> # This is a tensor with the values being the depth expressed in meters for each pixel
>>> output["predicted_depth"].shape
torch.Size([1, 384, 384]) Learn more about the basics of using a pipeline in the pipeline tutorial This depth estimation pipeline can currently be loaded from pipeline() using the following task identifier:
"depth-estimation". See the list of available models on huggingface.co/models.  __call__  < source > ( images: typing.Union[str, typing.List[str], ForwardRef('Image.Image'), typing.List[ForwardRef('Image.Image')]] **


DepthEstimationPipeline

kwargs  )   Parameters   images (str, List[str], PIL.Image or List[PIL.Image]) —
The pipeline handles three types of images:
A string containing a http link pointing to an image
A string containing a local path to an image
An image loaded in PIL directly
The pipeline accepts either a single image or a batch of images, which must then be passed as a string.
Images in a batch must all be in the same format: all as http links, all as local paths, or all as PIL
images.   top_k (int, optional, defaults to 5) —
The number of top labels that will be returned by the pipeline. If the provided number is higher than
the number of labels available in the model configuration, it will default to the number of labels.   timeout (float, optional, defaults to None) —
The maximum time in seconds to wait for fetching images from the web. If None, no timeout is set and
the call may block forever.    Assign labels to the image(s) passed as inputs.


ImageClassificationPipeline

class transformers.ImageClassificationPipeline  < source > ( *args **kwargs  )   Parameters   model (PreTrainedModel or TFPreTrainedModel) —
The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from
PreTrainedModel for PyTorch and TFPreTrainedModel for TensorFlow.   tokenizer (PreTrainedTokenizer) —
The tokenizer that will be used by the pipeline to encode data for the model. This object inherits from
PreTrainedTokenizer.   modelcard (str or ModelCard, optional) —
Model card attributed to the model for this pipeline.   framework (str, optional) —
The framework to use, either "pt" for PyTorch or "tf" for TensorFlow. The specified framework must be
installed.
If no framework is specified, will default to the one currently installed. If no framework is specified and
both frameworks are installed, will default to the framework of the model, or to PyTorch if no model is
provided.   task (str, defaults to


ImageClassificationPipeline

 "") —
A task-identifier for the pipeline.   num_workers (int, optional, defaults to 8) —
When the pipeline will use DataLoader (when passing a dataset, on GPU for a Pytorch model), the number of
workers to be used.   batch_size (int, optional, defaults to 1) —
When the pipeline will use DataLoader (when passing a dataset, on GPU for a Pytorch model), the size of
the batch to use, for inference this is not always beneficial, please read Batching with
pipelines .   args_parser (ArgumentHandler, optional) —
Reference to the object in charge of parsing supplied pipeline parameters.   device (int, optional, defaults to -1) —
Device ordinal for CPU/GPU supports. Setting this to -1 will leverage CPU, a positive will run the model on
the associated CUDA device id. You can pass native torch.device or a str too.   binary_output (bool, optional, defaults to False) —
Flag indicating if the output the pipeline should happen in a binary format (i.e., pickle)


ImageClassificationPipeline

or as raw text.    Image classification pipeline using any AutoModelForImageClassification. This pipeline predicts the class of an
image.  Example:   Copied >>> from transformers import pipeline
>>> classifier = pipeline(model="microsoft/beit-base-patch16-224-pt22k-ft22k")
>>> classifier("https://huggingface.co/datasets/Narsil/image_dummy/raw/main/parrots.png")
[{'score': 0.442, 'label': 'macaw'}, {'score': 0.088, 'label': 'popinjay'}, {'score': 0.075, 'label': 'parrot'}, {'score': 0.073, 'label': 'parodist, lampooner'}, {'score': 0.046, 'label': 'poll, poll_parrot'}] Learn more about the basics of using a pipeline in the pipeline tutorial This image classification pipeline can currently be loaded from pipeline() using the following task identifier:
"image-classification". See the list of available models on
huggingface.co/models.  __call__  < source > ( images: typing.Union[str, typing.List[str], ForwardRef('Image.Image'), typing.List[ForwardRef


ImageClassificationPipeline

('Image.Image')]] **kwargs  )   Parameters   images (str, List[str], PIL.Image or List[PIL.Image]) —
The pipeline handles three types of images:
A string containing a http link pointing to an image
A string containing a local path to an image
An image loaded in PIL directly
The pipeline accepts either a single image or a batch of images, which must then be passed as a string.
Images in a batch must all be in the same format: all as http links, all as local paths, or all as PIL
images.   top_k (int, optional, defaults to 5) —
The number of top labels that will be returned by the pipeline. If the provided number is higher than
the number of labels available in the model configuration, it will default to the number of labels.   timeout (float, optional, defaults to None) —
The maximum time in seconds to wait for fetching images from the web. If None, no timeout is set and
the call may block forever.    Assign labels to the image(s) passed as inputs.


ImageSegmentationPipeline

class transformers.ImageSegmentationPipeline  < source > ( *args **kwargs  )   Parameters   model (PreTrainedModel or TFPreTrainedModel) —
The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from
PreTrainedModel for PyTorch and TFPreTrainedModel for TensorFlow.   tokenizer (PreTrainedTokenizer) —
The tokenizer that will be used by the pipeline to encode data for the model. This object inherits from
PreTrainedTokenizer.   modelcard (str or ModelCard, optional) —
Model card attributed to the model for this pipeline.   framework (str, optional) —
The framework to use, either "pt" for PyTorch or "tf" for TensorFlow. The specified framework must be
installed.
If no framework is specified, will default to the one currently installed. If no framework is specifie


ImageSegmentationPipeline

d and
both frameworks are installed, will default to the framework of the model, or to PyTorch if no model is
provided.   task (str, defaults to "") —
A task-identifier for the pipeline.   num_workers (int, optional, defaults to 8) —
When the pipeline will use DataLoader (when passing a dataset, on GPU for a Pytorch model), the number of
workers to be used.   batch_size (int, optional, defaults to 1) —
When the pipeline will use DataLoader (when passing a dataset, on GPU for a Pytorch model), the size of
the batch to use, for inference this is not always beneficial, please read Batching with
pipelines .   args_parser (ArgumentHandler, optional) —
Reference to the object in charge of parsing supplied pipeline parameters.   device (int, optional, defaults to -1) —
Device ordinal for CPU/GPU supports. Sett


ImageSegmentationPipeline

ing this to -1 will leverage CPU, a positive will run the model on
the associated CUDA device id. You can pass native torch.device or a str too.   binary_output (bool, optional, defaults to False) —
Flag indicating if the output the pipeline should happen in a binary format (i.e., pickle) or as raw text.    Image segmentation pipeline using any AutoModelForXXXSegmentation. This pipeline predicts masks of objects and
their classes.  Example:   Copied >>> from transformers import pipeline
>>> segmenter = pipeline(model="facebook/detr-resnet-50-panoptic")
>>> segments = segmenter("https://huggingface.co/datasets/Narsil/image_dummy/raw/main/parrots.png")
>>> len(segments)
2
>>> segments[0]["label"]
'bird'
>>> segments[1]["label"]
'bird'
>>> type(segments[0]["mask"])  # This is a black and white mask showing


ImageSegmentationPipeline

 where is the bird on the original image.
<class 'PIL.Image.Image'>
>>> segments[0]["mask"].size
(768, 512) This image segmentation pipeline can currently be loaded from pipeline() using the following task identifier:
"image-segmentation". See the list of available models on
huggingface.co/models.  __call__  < source > ( images **kwargs  )   Parameters   images (str, List[str], PIL.Image or List[PIL.Image]) —
The pipeline handles three types of images:
A string containing an HTTP(S) link pointing to an image
A string containing a local path to an image
An image loaded in PIL directly
The pipeline accepts either a single image or a batch of images. Images in a batch must all be in the
same format: all as HTTP(S) links, all as local paths, or all as PIL images.   subtask (str, optional) —
Segmentation tas


ImageSegmentationPipeline

k to be performed, choose [semantic, instance and panoptic] depending on model
capabilities. If not set, the pipeline will attempt tp resolve in the following order:
panoptic, instance, semantic.   threshold (float, optional, defaults to 0.9) —
Probability threshold to filter out predicted masks.   mask_threshold (float, optional, defaults to 0.5) —
Threshold to use when turning the predicted masks into binary values.   overlap_mask_area_threshold (float, optional, defaults to 0.5) —
Mask overlap threshold to eliminate small, disconnected segments.   timeout (float, optional, defaults to None) —
The maximum time in seconds to wait for fetching images from the web. If None, no timeout is set and
the call may block forever.    Perform segmentation (detect masks & classes) in the image(s) passed as inputs.


ImageToImagePipeline

class transformers.ImageToImagePipeline  < source > ( *args **kwargs  )   Parameters   model (PreTrainedModel or TFPreTrainedModel) —
The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from
PreTrainedModel for PyTorch and TFPreTrainedModel for TensorFlow.   tokenizer (PreTrainedTokenizer) —
The tokenizer that will be used by the pipeline to encode data for the model. This object inherits from
PreTrainedTokenizer.   modelcard (str or ModelCard, optional) —
Model card attributed to the model for this pipeline.   framework (str, optional) —
The framework to use, either "pt" for PyTorch or "tf" for TensorFlow. The specified framework must be
installed.
If no framework is specified, will default to the one currently installed. If no framework is specified and
both frameworks are installed, will default to the frame


ImageToImagePipeline

work of the model, or to PyTorch if no model is
provided.   task (str, defaults to "") —
A task-identifier for the pipeline.   num_workers (int, optional, defaults to 8) —
When the pipeline will use DataLoader (when passing a dataset, on GPU for a Pytorch model), the number of
workers to be used.   batch_size (int, optional, defaults to 1) —
When the pipeline will use DataLoader (when passing a dataset, on GPU for a Pytorch model), the size of
the batch to use, for inference this is not always beneficial, please read Batching with
pipelines .   args_parser (ArgumentHandler, optional) —
Reference to the object in charge of parsing supplied pipeline parameters.   device (int, optional, defaults to -1) —
Device ordinal for CPU/GPU supports. Setting this to -1 will leverage CPU, a positive will run the model on
the associated CUDA device id. You can pass native t


ImageToImagePipeline

orch.device or a str too.   binary_output (bool, optional, defaults to False) —
Flag indicating if the output the pipeline should happen in a binary format (i.e., pickle) or as raw text.    Image to Image pipeline using any AutoModelForImageToImage. This pipeline generates an image based on a previous
image input.  Example:   Copied >>> from PIL import Image
>>> import requests
>>> from transformers import pipeline
>>> upscaler = pipeline("image-to-image", model="caidas/swin2SR-classical-sr-x2-64")
>>> img = Image.open(requests.get("http://images.cocodataset.org/val2017/000000039769.jpg", stream=True).raw)
>>> img = img.resize((64, 64))
>>> upscaled_img = upscaler(img)
>>> img.size
(64, 64)
>>> upscaled_img.size
(144, 144) This image to image pipeline can currently be loaded from pipeline() using the following task identifier:
"image-to-image". See the list o


ImageToImagePipeline

f available models on huggingface.co/models.  __call__  < source > ( images: typing.Union[str, typing.List[str], ForwardRef('Image.Image'), typing.List[ForwardRef('Image.Image')]] **kwargs  )   Parameters   images (str, List[str], PIL.Image or List[PIL.Image]) —
The pipeline handles three types of images:
A string containing a http link pointing to an image
A string containing a local path to an image
An image loaded in PIL directly
The pipeline accepts either a single image or a batch of images, which must then be passed as a string.
Images in a batch must all be in the same format: all as http links, all as local paths, or all as PIL
images.   timeout (float, optional, defaults to None) —
The maximum time in seconds to wait for fetching images from the web. If None, no timeout is used and
the call may block forever.    Transform the image(s) passed as input


ObjectDetectionPipeline

class transformers.ObjectDetectionPipeline  < source > ( *args **kwargs  )   Parameters   model (PreTrainedModel or TFPreTrainedModel) —
The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from
PreTrainedModel for PyTorch and TFPreTrainedModel for TensorFlow.   tokenizer (PreTrainedTokenizer) —
The tokenizer that will be used by the pipeline to encode data for the model. This object inherits from
PreTrainedTokenizer.   modelcard (str or ModelCard, optional) —
Model card attributed to the model for this pipeline.   framework (str, optional) —
The framework to use, either "pt" for PyTorch or "tf" for TensorFlow. The specified framework must be
installed.
If no framework is specified, will default to the one currently installed. If no framework is specified and
both frameworks are installed, will default to the framework of the model, or to PyT


ObjectDetectionPipeline

orch if no model is
provided.   task (str, defaults to "") —
A task-identifier for the pipeline.   num_workers (int, optional, defaults to 8) —
When the pipeline will use DataLoader (when passing a dataset, on GPU for a Pytorch model), the number of
workers to be used.   batch_size (int, optional, defaults to 1) —
When the pipeline will use DataLoader (when passing a dataset, on GPU for a Pytorch model), the size of
the batch to use, for inference this is not always beneficial, please read Batching with
pipelines .   args_parser (ArgumentHandler, optional) —
Reference to the object in charge of parsing supplied pipeline parameters.   device (int, optional, defaults to -1) —
Device ordinal for CPU/GPU supports. Setting this to -1 will leverage CPU, a positive will run the model on
the associated CUDA device id. You can pass native torch.device or a str too.   binary_output (bool, optional,


ObjectDetectionPipeline

defaults to False) —
Flag indicating if the output the pipeline should happen in a binary format (i.e., pickle) or as raw text.    Object detection pipeline using any AutoModelForObjectDetection. This pipeline predicts bounding boxes of objects
and their classes.  Example:   Copied >>> from transformers import pipeline
>>> detector = pipeline(model="facebook/detr-resnet-50")
>>> detector("https://huggingface.co/datasets/Narsil/image_dummy/raw/main/parrots.png")
[{'score': 0.997, 'label': 'bird', 'box': {'xmin': 69, 'ymin': 171, 'xmax': 396, 'ymax': 507}}, {'score': 0.999, 'label': 'bird', 'box': {'xmin': 398, 'ymin': 105, 'xmax': 767, 'ymax': 507}}]
>>> # x, y  are expressed relative to the top left hand corner. Learn more about the basics of using a pipeline in the pipeline tutorial This object detection pipeline can currently be loaded from pipeline() using the following task identifier:


ObjectDetectionPipeline

"object-detection". See the list of available models on huggingface.co/models.  __call__  < source > ( *args **kwargs  )   Parameters   images (str, List[str], PIL.Image or List[PIL.Image]) —
The pipeline handles three types of images:
A string containing an HTTP(S) link pointing to an image
A string containing a local path to an image
An image loaded in PIL directly
The pipeline accepts either a single image or a batch of images. Images in a batch must all be in the
same format: all as HTTP(S) links, all as local paths, or all as PIL images.   threshold (float, optional, defaults to 0.9) —
The probability necessary to make a prediction.   timeout (float, optional, defaults to None) —
The maximum time in seconds to wait for fetching images from the web. If None, no timeout is set and
the call may block forever.    Detect objects (bounding boxes & classes) in the image(s) passed as inputs.


VideoClassificationPipeline

class transformers.VideoClassificationPipeline  < source > ( *args **kwargs  )   Parameters   model (PreTrainedModel or TFPreTrainedModel) —
The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from
PreTrainedModel for PyTorch and TFPreTrainedModel for TensorFlow.   tokenizer (PreTrainedTokenizer) —
The tokenizer that will be used by the pipeline to encode data for the model. This object inherits from
PreTrainedTokenizer.   modelcard (str or ModelCard, optional) —
Model card attributed to the model for this pipeline.   framework (str, optional) —
The framework to use, either "pt" for PyTorch or "tf" for TensorFlow. The specified framework must be
installed.
If no framework is specified, will default to the one currently installed. If no framework is specified and
both frameworks are installed, w


VideoClassificationPipeline

ill default to the framework of the model, or to PyTorch if no model is
provided.   task (str, defaults to "") —
A task-identifier for the pipeline.   num_workers (int, optional, defaults to 8) —
When the pipeline will use DataLoader (when passing a dataset, on GPU for a Pytorch model), the number of
workers to be used.   batch_size (int, optional, defaults to 1) —
When the pipeline will use DataLoader (when passing a dataset, on GPU for a Pytorch model), the size of
the batch to use, for inference this is not always beneficial, please read Batching with
pipelines .   args_parser (ArgumentHandler, optional) —
Reference to the object in charge of parsing supplied pipeline parameters.   device (int, optional, defaults to -1) —
Device ordinal for CPU/GPU supports. Setting this to -1 will leverage CPU, a positive will run the model on
the associa


VideoClassificationPipeline

ted CUDA device id. You can pass native torch.device or a str too.   binary_output (bool, optional, defaults to False) —
Flag indicating if the output the pipeline should happen in a binary format (i.e., pickle) or as raw text.    Video classification pipeline using any AutoModelForVideoClassification. This pipeline predicts the class of a
video. This video classification pipeline can currently be loaded from pipeline() using the following task identifier:
"video-classification". See the list of available models on
huggingface.co/models.  __call__  < source > ( videos: typing.Union[str, typing.List[str]] **kwargs  )   Parameters   videos (str, List[str]) —
The pipeline handles three types of videos:
A string containing a http link pointing to a video
A string containing a local path to a video
The pipeline accepts either a single video or a b


VideoClassificationPipeline

atch of videos, which must then be passed as a string.
Videos in a batch must all be in the same format: all as http links or all as local paths.   top_k (int, optional, defaults to 5) —
The number of top labels that will be returned by the pipeline. If the provided number is higher than
the number of labels available in the model configuration, it will default to the number of labels.   num_frames (int, optional, defaults to self.model.config.num_frames) —
The number of frames sampled from the video to run the classification on. If not provided, will default
to the number of frames specified in the model configuration.   frame_sampling_rate (int, optional, defaults to 1) —
The sampling rate used to select frames from the video. If not provided, will default to 1, i.e. every
frame will be used.    Assign labels to the video(s) passed as input


ZeroShotImageClassificationPipeline

class transformers.ZeroShotImageClassificationPipeline  < source > ( **kwargs  )   Parameters   model (PreTrainedModel or TFPreTrainedModel) —
The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from
PreTrainedModel for PyTorch and TFPreTrainedModel for TensorFlow.   tokenizer (PreTrainedTokenizer) —
The tokenizer that will be used by the pipeline to encode data for the model. This object inherits from
PreTrainedTokenizer.   modelcard (str or ModelCard, optional) —
Model card attributed to the model for this pipeline.   framework (str, optional) —
The framework to use, either "pt" for PyTorch or "tf" for TensorFlow. The specified framework must be
installed.
If no framework is specified, will default to the one currently installed. If no framework i


ZeroShotImageClassificationPipeline

s specified and
both frameworks are installed, will default to the framework of the model, or to PyTorch if no model is
provided.   task (str, defaults to "") —
A task-identifier for the pipeline.   num_workers (int, optional, defaults to 8) —
When the pipeline will use DataLoader (when passing a dataset, on GPU for a Pytorch model), the number of
workers to be used.   batch_size (int, optional, defaults to 1) —
When the pipeline will use DataLoader (when passing a dataset, on GPU for a Pytorch model), the size of
the batch to use, for inference this is not always beneficial, please read Batching with
pipelines .   args_parser (ArgumentHandler, optional) —
Reference to the object in charge of parsing supplied pipeline parameters.   device (int, optional, defaults to -1) —
Device ordinal for CPU/GP


ZeroShotImageClassificationPipeline

U supports. Setting this to -1 will leverage CPU, a positive will run the model on
the associated CUDA device id. You can pass native torch.device or a str too.   binary_output (bool, optional, defaults to False) —
Flag indicating if the output the pipeline should happen in a binary format (i.e., pickle) or as raw text.    Zero shot image classification pipeline using CLIPModel. This pipeline predicts the class of an image when you
provide an image and a set of candidate_labels.  Example:   Copied >>> from transformers import pipeline
>>> classifier = pipeline(model="openai/clip-vit-large-patch14")
>>> classifier(
...     "https://huggingface.co/datasets/Narsil/image_dummy/raw/main/parrots.png",
...     candidate_labels=["animals", "humans", "landscape"],
... )
[{'score': 0.965, 'label': 'animals'


ZeroShotImageClassificationPipeline

}, {'score': 0.03, 'label': 'humans'}, {'score': 0.005, 'label': 'landscape'}]
>>> classifier(
...     "https://huggingface.co/datasets/Narsil/image_dummy/raw/main/parrots.png",
...     candidate_labels=["black and white", "photorealist", "painting"],
... )
[{'score': 0.996, 'label': 'black and white'}, {'score': 0.003, 'label': 'photorealist'}, {'score': 0.0, 'label': 'painting'}] Learn more about the basics of using a pipeline in the pipeline tutorial This image classification pipeline can currently be loaded from pipeline() using the following task identifier:
"zero-shot-image-classification". See the list of available models on
huggingface.co/models.  __call__  < source > ( images: typing.Union[str, typing.List[str], ForwardRef('Image'), typing.List[ForwardRef('Image')]] **kwargs  )   Paramete


ZeroShotImageClassificationPipeline

rs   images (str, List[str], PIL.Image or List[PIL.Image]) —
The pipeline handles three types of images:
A string containing a http link pointing to an image
A string containing a local path to an image
An image loaded in PIL directly
   candidate_labels (List[str]) —
The candidate labels for this image   hypothesis_template (str, optional, defaults to "This is a photo of {}") —
The sentence used in cunjunction with candidate_labels to attempt the image classification by
replacing the placeholder with the candidate_labels. Then likelihood is estimated by using
logits_per_image   timeout (float, optional, defaults to None) —
The maximum time in seconds to wait for fetching images from the web. If None, no timeout is set and
the call may block forever.    Assign labels to the image(s) passed as inpu


ZeroShotObjectDetectionPipeline

class transformers.ZeroShotObjectDetectionPipeline  < source > ( **kwargs  )   Parameters   model (PreTrainedModel or TFPreTrainedModel) —
The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from
PreTrainedModel for PyTorch and TFPreTrainedModel for TensorFlow.   tokenizer (PreTrainedTokenizer) —
The tokenizer that will be used by the pipeline to encode data for the model. This object inherits from
PreTrainedTokenizer.   modelcard (str or ModelCard, optional) —
Model card attributed to the model for this pipeline.   framework (str, optional) —
The framework to use, either "pt" for PyTorch or "tf" for TensorFlow. The specified framework must be
installed.
If no framework is specified, will default to the one currently installed. If no framework is specified and
both frameworks are installed, will default to the framework of the model, or to PyTorch if no model is
provided.   task (str,


ZeroShotObjectDetectionPipeline

defaults to "") —
A task-identifier for the pipeline.   num_workers (int, optional, defaults to 8) —
When the pipeline will use DataLoader (when passing a dataset, on GPU for a Pytorch model), the number of
workers to be used.   batch_size (int, optional, defaults to 1) —
When the pipeline will use DataLoader (when passing a dataset, on GPU for a Pytorch model), the size of
the batch to use, for inference this is not always beneficial, please read Batching with
pipelines .   args_parser (ArgumentHandler, optional) —
Reference to the object in charge of parsing supplied pipeline parameters.   device (int, optional, defaults to -1) —
Device ordinal for CPU/GPU supports. Setting this to -1 will leverage CPU, a positive will run the model on
the associated CUDA device id. You can pass native torch.device or a str too.   binary_output (bool, optional, defaults to False) —
Flag indicating if the output the pipeline should happen in a binar


ZeroShotObjectDetectionPipeline

y format (i.e., pickle) or as raw text.    Zero shot object detection pipeline using OwlViTForObjectDetection. This pipeline predicts bounding boxes of
objects when you provide an image and a set of candidate_labels.  Example:   Copied >>> from transformers import pipeline
>>> detector = pipeline(model="google/owlvit-base-patch32", task="zero-shot-object-detection")
>>> detector(
...     "http://images.cocodataset.org/val2017/000000039769.jpg",
...     candidate_labels=["cat", "couch"],
... )
[{'score': 0.287, 'label': 'cat', 'box': {'xmin': 324, 'ymin': 20, 'xmax': 640, 'ymax': 373}}, {'score': 0.254, 'label': 'cat', 'box': {'xmin': 1, 'ymin': 55, 'xmax': 315, 'ymax': 472}}, {'score': 0.121, 'label': 'couch', 'box': {'xmin': 4, 'ymin': 0, 'xmax': 642, 'ymax': 476}}]
>>> detector(
...     "https://huggingface.co/datasets/Narsil/image_dummy/raw/main/parrots.png",
...     candidate_labels=["head", "bird"],
... )
[{'score': 0.119, 'labe


ZeroShotObjectDetectionPipeline

l': 'bird', 'box': {'xmin': 71, 'ymin': 170, 'xmax': 410, 'ymax': 508}}] Learn more about the basics of using a pipeline in the pipeline tutorial This object detection pipeline can currently be loaded from pipeline() using the following task identifier:
"zero-shot-object-detection". See the list of available models on
huggingface.co/models.  __call__  < source > ( image: typing.Union[str, ForwardRef('Image.Image'), typing.List[typing.Dict[str, typing.Any]]] candidate_labels: typing.Union[str, typing.List[str]] = None **kwargs  )   Parameters   image (str, PIL.Image or List[Dict[str, Any]]) —
The pipeline handles three types of images:
A string containing an http url pointing to an image
A string containing a local path to an image
An image loaded in PIL directly
You can use this parameter to send directly a list of images, or a dataset or a generator like so:    Detect objects (bounding boxes & classes) in the image(s) passed as inpu


ConversationalPipeline

class transformers.Conversation  < source > ( messages: typing.Union[str, typing.List[typing.Dict[str, str]]] = None conversation_id: UUID = None **deprecated_kwargs  )   Parameters   messages (Union[str, List[Dict[str, str]]], optional) —
The initial messages to start the conversation, either a string, or a list of dicts containing “role” and
“content” keys. If a string is passed, it is interpreted as a single message with the “user” role.   conversation_id (uuid.UUID, optional) —
Unique identifier for the conversation. If not provided, a random UUID4 id will be assigned to the
conversation.    Utility class containing a conversation and its history. This class is meant to be used as an input to the
ConversationalPipeline. The conversation contains several utility functions to manage the addition of new user
inputs and generated model responses.  Usage:   Copied conversation = Conversation("Going to the movies tonight - any suggestions?")
convers


ConversationalPipeline

ation.add_message({"role": "assistant", "content": "The Big lebowski."})
conversation.add_message({"role": "user", "content": "Is it good?"})  add_user_input  < source > ( text: str overwrite: bool = False  )    Add a user input to the conversation for the next round. This is a legacy method that assumes that inputs must
alternate user/assistant/user/assistant, and so will not add multiple user messages in succession. We recommend
just using add_message with role “user” instead.  append_response  < source > ( response: str  )    This is a legacy method. We recommend just using add_message with an appropriate role instead.  mark_processed  < source > ( )    This is a legacy method that no longer has any effect, as the Conversation no longer distinguishes between
processed and unprocessed user input.
 class transformers.ConversationalPipeline  < source > ( *args **kwargs  )   Parameters   model (PreTrainedModel or TFPreTrainedModel) —
The model that


ConversationalPipeline

 will be used by the pipeline to make predictions. This needs to be a model inheriting from
PreTrainedModel for PyTorch and TFPreTrainedModel for TensorFlow.   tokenizer (PreTrainedTokenizer) —
The tokenizer that will be used by the pipeline to encode data for the model. This object inherits from
PreTrainedTokenizer.   modelcard (str or ModelCard, optional) —
Model card attributed to the model for this pipeline.   framework (str, optional) —
The framework to use, either "pt" for PyTorch or "tf" for TensorFlow. The specified framework must be
installed.
If no framework is specified, will default to the one currently installed. If no framework is specified and
both frameworks are installed, will default to the framework of the model, or to PyTorch if no model is
provided.   task (str, defaults to "") —
A task-identifier for the pipeline.   num_workers (int, optional, defaults to 8) —
When the pipeline will use DataLoader (when passing a dataset, on


ConversationalPipeline

GPU for a Pytorch model), the number of
workers to be used.   batch_size (int, optional, defaults to 1) —
When the pipeline will use DataLoader (when passing a dataset, on GPU for a Pytorch model), the size of
the batch to use, for inference this is not always beneficial, please read Batching with
pipelines .   args_parser (ArgumentHandler, optional) —
Reference to the object in charge of parsing supplied pipeline parameters.   device (int, optional, defaults to -1) —
Device ordinal for CPU/GPU supports. Setting this to -1 will leverage CPU, a positive will run the model on
the associated CUDA device id. You can pass native torch.device or a str too.   binary_output (bool, optional, defaults to False) —
Flag indicating if the output the pipeline should happen in a binary format (i.e., pickle) or as raw text.   min_length_for_response (int, optional, defaults to 32) —
The minimum length (in number of tokens) for a response.   minimum_tokens (int, o


ConversationalPipeline

ptional, defaults to 10) —
The minimum length of tokens to leave for a response.    Multi-turn conversational pipeline.  Example:   Copied >>> from transformers import pipeline, Conversation
>>> chatbot = pipeline(model="microsoft/DialoGPT-medium")
>>> conversation = Conversation("Going to the movies tonight - any suggestions?")
>>> conversation = chatbot(conversation)
>>> conversation.generated_responses[-1]
'The Big Lebowski'
>>> conversation.add_user_input("Is it an action movie?")
>>> conversation = chatbot(conversation)
>>> conversation.generated_responses[-1]
"It's a comedy." Learn more about the basics of using a pipeline in the pipeline tutorial This conversational pipeline can currently be loaded from pipeline() using the following task identifier:
"conversational". The models that this pipeline can use are models that have been fine-tuned on a multi-turn conversational task,
currently: ‘microsoft/DialoGPT-small’, ‘microsoft/DialoGPT-medi


ConversationalPipeline

um’, ‘microsoft/DialoGPT-large’. See the
up-to-date list of available models on
huggingface.co/models.  __call__  < source > ( conversations: typing.Union[transformers.pipelines.conversational.Conversation, typing.List[transformers.pipelines.conversational.Conversation]] num_workers = 0 **kwargs  ) → Conversation or a list of Conversation  Parameters   conversations (a Conversation or a list of Conversation) —
Conversations to generate responses for.   clean_up_tokenization_spaces (bool, optional, defaults to False) —
Whether or not to clean up the potential extra spaces in the text output.
generate_kwargs —
Additional keyword arguments to pass along to the generate method of the model (see the generate method
corresponding to your framework here).   Returns
Conversation or a list of Conversation

Conversation(s) with updated generated responses for those
containing a new user input.
  Generate responses for the conversation(s) given as inputs.


FillMaskPipeline

class transformers.FillMaskPipeline  < source > ( model: typing.Union[ForwardRef('PreTrainedModel'), ForwardRef('TFPreTrainedModel')] tokenizer: typing.Optional[transformers.tokenization_utils.PreTrainedTokenizer] = None feature_extractor: typing.Optional[ForwardRef('SequenceFeatureExtractor')] = None image_processor: typing.Optional[transformers.image_processing_utils.BaseImageProcessor] = None modelcard: typing.Optional[transformers.modelcard.ModelCard] = None framework: typing.Optional[str] = None task: str = '' args_parser: ArgumentHandler = None device: typing.Union[int, ForwardRef('torch.device')] = None torch_dtype: typing.Union[str, ForwardRef('torch.dtype'), NoneType] = None binary_output: bool = False **kwargs  )   Parameters   model (PreTrainedModel or TFPreTrainedModel) —
The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from
PreTrainedModel for PyTorch and TFPreTrainedModel for TensorFlow.   t


FillMaskPipeline

okenizer (PreTrainedTokenizer) —
The tokenizer that will be used by the pipeline to encode data for the model. This object inherits from
PreTrainedTokenizer.   modelcard (str or ModelCard, optional) —
Model card attributed to the model for this pipeline.   framework (str, optional) —
The framework to use, either "pt" for PyTorch or "tf" for TensorFlow. The specified framework must be
installed.
If no framework is specified, will default to the one currently installed. If no framework is specified and
both frameworks are installed, will default to the framework of the model, or to PyTorch if no model is
provided.   task (str, defaults to "") —
A task-identifier for the pipeline.   num_workers (int, optional, defaults to 8) —
When the pipeline will use DataLoader (when passing a dataset, on GPU for a Pytorch model), the number of
workers to be used.   batch_size (int, optional, defaults to 1) —
When the pipeline will use DataLoader (when passing a dataset, o


FillMaskPipeline

n GPU for a Pytorch model), the size of
the batch to use, for inference this is not always beneficial, please read Batching with
pipelines .   args_parser (ArgumentHandler, optional) —
Reference to the object in charge of parsing supplied pipeline parameters.   device (int, optional, defaults to -1) —
Device ordinal for CPU/GPU supports. Setting this to -1 will leverage CPU, a positive will run the model on
the associated CUDA device id. You can pass native torch.device or a str too.   binary_output (bool, optional, defaults to False) —
Flag indicating if the output the pipeline should happen in a binary format (i.e., pickle) or as raw text.   top_k (int, defaults to 5) —
The number of predictions to return.   targets (str or List[str], optional) —
When passed, the model will limit the scores to the passed targets instead of looking up in the whole
vocab. If the provided targets are not in the model vocab, they will be tokenized and the first resulting
tok


FillMaskPipeline

en will be used (with a warning, and that might be slower).    Masked language modeling prediction pipeline using any ModelWithLMHead. See the masked language modeling
examples for more information.  Example:   Copied >>> from transformers import pipeline
>>> fill_masker = pipeline(model="bert-base-uncased")
>>> fill_masker("This is a simple [MASK].")
[{'score': 0.042, 'token': 3291, 'token_str': 'problem', 'sequence': 'this is a simple problem.'}, {'score': 0.031, 'token': 3160, 'token_str': 'question', 'sequence': 'this is a simple question.'}, {'score': 0.03, 'token': 8522, 'token_str': 'equation', 'sequence': 'this is a simple equation.'}, {'score': 0.027, 'token': 2028, 'token_str': 'one', 'sequence': 'this is a simple one.'}, {'score': 0.024, 'token': 3627, 'token_str': 'rule', 'sequence': 'this is a simple rule.'}] Learn more about the basics of using a pipeline in the pipeline tutorial This mask filling pipeline can currently be loaded from pipelin


FillMaskPipeline

e() using the following task identifier:
"fill-mask". The models that this pipeline can use are models that have been trained with a masked language modeling objective,
which includes the bi-directional models in the library. See the up-to-date list of available models on
huggingface.co/models. This pipeline only works for inputs with exactly one token masked. Experimental: We added support for multiple
masks. The returned values are raw model output, and correspond to disjoint probabilities where one might expect
joint probabilities (See discussion).   This pipeline now supports tokenizer_kwargs. For example try:   Copied >>> from transformers import pipeline
>>> fill_masker = pipeline(model="bert-base-uncased")
>>> tokenizer_kwargs = {"truncation": True}
>>> fill_masker(
...     "This is a simple [MASK]. " + "...with a large amount of repeated text appended. " * 100,
...     tokenizer_kwargs=tokenizer_kwargs,
... )  __call__  < source > ( inputs *args **


FillMaskPipeline

kwargs  ) → A list or a list of list of dict  Parameters   args (str or List[str]) —
One or several texts (or one list of prompts) with masked tokens.   targets (str or List[str], optional) —
When passed, the model will limit the scores to the passed targets instead of looking up in the whole
vocab. If the provided targets are not in the model vocab, they will be tokenized and the first
resulting token will be used (with a warning, and that might be slower).   top_k (int, optional) —
When passed, overrides the number of predictions to return.   Returns
A list or a list of list of dict

Each result comes as list of dictionaries with the following keys:
sequence (str) — The corresponding input with the mask token prediction.
score (float) — The corresponding probability.
token (int) — The predicted token id (to replace the masked one).
token_str (str) — The predicted token (to replace the masked one).
  Fill the masked token in the text(s) given as inputs


NerPipeline

class transformers.TokenClassificationPipeline  < source > ( args_parser = <transformers.pipelines.token_classification.TokenClassificationArgumentHandler object at 0x7fefb1726880> *args **kwargs  )   Parameters   model (PreTrainedModel or TFPreTrainedModel) —
The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from
PreTrainedModel for PyTorch and TFPreTrainedModel for TensorFlow.   tokenizer (PreTrainedTokenizer) —
The tokenizer that will be used by the pipeline to encode data for the model. This object inherits from
PreTrainedTokenizer.   modelcard (str or ModelCard, optional) —
Model card attributed to the model for this pipeline.   framework (str, optional) —
The framework to use, either "pt" for PyTorch or "tf" for TensorFlow. The specified framework must be
installed.
If no framework is specified, will default to the one currently installed. If no framework is specified and
both framework


NerPipeline

s are installed, will default to the framework of the model, or to PyTorch if no model is
provided.   task (str, defaults to "") —
A task-identifier for the pipeline.   num_workers (int, optional, defaults to 8) —
When the pipeline will use DataLoader (when passing a dataset, on GPU for a Pytorch model), the number of
workers to be used.   batch_size (int, optional, defaults to 1) —
When the pipeline will use DataLoader (when passing a dataset, on GPU for a Pytorch model), the size of
the batch to use, for inference this is not always beneficial, please read Batching with
pipelines .   args_parser (ArgumentHandler, optional) —
Reference to the object in charge of parsing supplied pipeline parameters.   device (int, optional, defaults to -1) —
Device ordinal for CPU/GPU supports. Setting this to -1 will leverage CPU, a positive will run the model on
the associated CUDA device id. You can pass native torch.device or a str too.   binary_output (


NerPipeline

bool, optional, defaults to False) —
Flag indicating if the output the pipeline should happen in a binary format (i.e., pickle) or as raw text.   ignore_labels (List[str], defaults to ["O"]) —
A list of labels to ignore.   grouped_entities (bool, optional, defaults to False) —
DEPRECATED, use aggregation_strategy instead. Whether or not to group the tokens corresponding to the
same entity together in the predictions or not.   stride (int, optional) —
If stride is provided, the pipeline is applied on all the text. The text is split into chunks of size
model_max_length. Works only with fast tokenizers and aggregation_strategy different from NONE. The
value of this argument defines the number of overlapping tokens between chunks. In other words, the model
will shift forward by tokenizer.model_max_length - stride tokens each step.   aggregation_strategy (str, optional, defaults to "none") —
The strategy to fuse (or not) tokens based on the model


NerPipeline

prediction.
“none” : Will simply not do any aggregation and simply return raw results from the model
“simple” : Will attempt to group entities following the default schema. (A, B-TAG), (B, I-TAG), (C,
I-TAG), (D, B-TAG2) (E, B-TAG2) will end up being [{“word”: ABC, “entity”: “TAG”}, {“word”: “D”,
“entity”: “TAG2”}, {“word”: “E”, “entity”: “TAG2”}] Notice that two consecutive B tags will end up as
different entities. On word based languages, we might end up splitting words undesirably : Imagine
Microsoft being tagged as [{“word”: “Micro”, “entity”: “ENTERPRISE”}, {“word”: “soft”, “entity”:
“NAME”}]. Look for FIRST, MAX, AVERAGE for ways to mitigate that and disambiguate words (on languages
that support that meaning, which is basically tokens separated by a space). These mitigations will
only work on real words, “New york” might still be tagged with two different entities.
“first” : (works only on word based models) Will use the SIMPLE strategy


NerPipeline

 except that words, cannot
end up with different tags. Words will simply use the tag of the first token of the word when there
is ambiguity.
“average” : (works only on word based models) Will use the SIMPLE strategy except that words,
cannot end up with different tags. scores will be averaged first across tokens, and then the maximum
label is applied.
“max” : (works only on word based models) Will use the SIMPLE strategy except that words, cannot
end up with different tags. Word entity will simply be the token with the maximum score.
    Named Entity Recognition pipeline using any ModelForTokenClassification. See the named entity recognition
examples for more information.  Example:   Copied >>> from transformers import pipeline
>>> token_classifier = pipeline(model="Jean-Baptiste/camembert-ner", aggregation_strategy="simple")
>>> sentence = "Je m'appelle jean-baptiste et je vis à montréal"
>>> tokens = token_classifier(sentence)
>>> tokens
[{


NerPipeline

'entity_group': 'PER', 'score': 0.9931, 'word': 'jean-baptiste', 'start': 12, 'end': 26}, {'entity_group': 'LOC', 'score': 0.998, 'word': 'montréal', 'start': 38, 'end': 47}]
>>> token = tokens[0]
>>> # Start and end provide an easy way to highlight words in the original text.
>>> sentence[token["start"] : token["end"]]
' jean-baptiste'
>>> # Some models use the same idea to do part of speech.
>>> syntaxer = pipeline(model="vblagoje/bert-english-uncased-finetuned-pos", aggregation_strategy="simple")
>>> syntaxer("My name is Sarah and I live in London")
[{'entity_group': 'PRON', 'score': 0.999, 'word': 'my', 'start': 0, 'end': 2}, {'entity_group': 'NOUN', 'score': 0.997, 'word': 'name', 'start': 3, 'end': 7}, {'entity_group': 'AUX', 'score': 0.994, 'word': 'is', 'start': 8, 'end': 10}, {'entity_group': 'PROPN', 'score': 0.999, 'word': 'sarah', 'start': 11, 'end': 16}, {'entity_group': 'CCONJ', 'score': 0.999, 'word': 'and', 'start': 17, 'end':


NerPipeline

 20}, {'entity_group': 'PRON', 'score': 0.999, 'word': 'i', 'start': 21, 'end': 22}, {'entity_group': 'VERB', 'score': 0.998, 'word': 'live', 'start': 23, 'end': 27}, {'entity_group': 'ADP', 'score': 0.999, 'word': 'in', 'start': 28, 'end': 30}, {'entity_group': 'PROPN', 'score': 0.999, 'word': 'london', 'start': 31, 'end': 37}] Learn more about the basics of using a pipeline in the pipeline tutorial This token recognition pipeline can currently be loaded from pipeline() using the following task identifier:
"ner" (for predicting the classes of tokens in a sequence: person, organisation, location or miscellaneous). The models that this pipeline can use are models that have been fine-tuned on a token classification task. See the
up-to-date list of available models on
huggingface.co/models.  aggregate_words  < source > ( entities: typing.List[dict] aggregation_strategy: AggregationStrategy  )    Override tokens from a given word that disagree to


NerPipeline

 force agreement on word boundaries. Example: micro|soft| com|pany| B-ENT I-NAME I-ENT I-ENT will be rewritten with first strategy as microsoft|
company| B-ENT I-ENT  gather_pre_entities  < source > ( sentence: str input_ids: ndarray scores: ndarray offset_mapping: typing.Union[typing.List[typing.Tuple[int, int]], NoneType] special_tokens_mask: ndarray aggregation_strategy: AggregationStrategy  )    Fuse various numpy arrays into dicts with all the information needed for aggregation  group_entities  < source > ( entities: typing.List[dict]  )   Parameters   entities (dict) — The entities predicted by the pipeline.    Find and group together the adjacent tokens with the same entity predicted.  group_sub_entities  < source > ( entities: typing.List[dict]  )   Parameters   entities (dict) — The entities predicted by the pipeline.    Group together the adjacent tokens with the same entity predicted.
See TokenClassificationPipeline for all details


QuestionAnsweringPipeline

class transformers.QuestionAnsweringPipeline  < source > ( model: typing.Union[ForwardRef('PreTrainedModel'), ForwardRef('TFPreTrainedModel')] tokenizer: PreTrainedTokenizer modelcard: typing.Optional[transformers.modelcard.ModelCard] = None framework: typing.Optional[str] = None task: str = '' **kwargs  )   Parameters   model (PreTrainedModel or TFPreTrainedModel) —
The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from
PreTrainedModel for PyTorch and TFPreTrainedModel for TensorFlow.   tokenizer (PreTrainedTokenizer) —
The tokenizer that will be used by the pipeline to encode data for the model. This object inherits from
PreTrainedTokenizer.   modelcard (str or ModelCard, optional) —
Model card attributed to the model for this pipeline.   framework (str, optional) —
The framework to use, either "pt" for PyTorch or "tf" for TensorFlow. The specified framework must be
installed.
If


QuestionAnsweringPipeline

no framework is specified, will default to the one currently installed. If no framework is specified and
both frameworks are installed, will default to the framework of the model, or to PyTorch if no model is
provided.   task (str, defaults to "") —
A task-identifier for the pipeline.   num_workers (int, optional, defaults to 8) —
When the pipeline will use DataLoader (when passing a dataset, on GPU for a Pytorch model), the number of
workers to be used.   batch_size (int, optional, defaults to 1) —
When the pipeline will use DataLoader (when passing a dataset, on GPU for a Pytorch model), the size of
the batch to use, for inference this is not always beneficial, please read Batching with
pipelines .   args_parser (ArgumentHandler, optional) —
Reference to the object in charge of parsing supplied pipeline parameters.   device (int, optional, defaults to -1) —
Device ordinal for CPU/GPU supports. Setting this to -1 will leverage CPU,


QuestionAnsweringPipeline

 a positive will run the model on
the associated CUDA device id. You can pass native torch.device or a str too.   binary_output (bool, optional, defaults to False) —
Flag indicating if the output the pipeline should happen in a binary format (i.e., pickle) or as raw text.    Question Answering pipeline using any ModelForQuestionAnswering. See the question answering
examples for more information.  Example:   Copied >>> from transformers import pipeline
>>> oracle = pipeline(model="deepset/roberta-base-squad2")
>>> oracle(question="Where do I live?", context="My name is Wolfgang and I live in Berlin")
{'score': 0.9191, 'start': 34, 'end': 40, 'answer': 'Berlin'} Learn more about the basics of using a pipeline in the pipeline tutorial This question answering pipeline can currently be loaded from pipeline() using the following task identifier:
"question-answering". The models that this pipeline can use are models that have been fine-tun


QuestionAnsweringPipeline

ed on a question answering task. See the
up-to-date list of available models on
huggingface.co/models.  __call__  < source > ( *args **kwargs  ) → A dict or a list of dict  Parameters   args (SquadExample or a list of SquadExample) —
One or several SquadExample containing the question and context.   X (SquadExample or a list of SquadExample, optional) —
One or several SquadExample containing the question and context (will be treated the same way as if
passed as the first positional argument).   data (SquadExample or a list of SquadExample, optional) —
One or several SquadExample containing the question and context (will be treated the same way as if
passed as the first positional argument).   question (str or List[str]) —
One or several question(s) (must be used in conjunction with the context argument).   context (str or List[str]) —
One or several context(s) associated with the question(s) (must be used in conjunction with the
que