Utilities for pipelines

This page lists all the utility functions the library provides for pipelines.
Most of those are only useful if you are studying the code of the models in the library.


Argument handling

class transformers.pipelines.ArgumentHandler  < source > ( )    Base interface for handling arguments for each Pipeline.
 class transformers.pipelines.ZeroShotClassificationArgumentHandler  < source > ( )    Handles arguments for zero-shot for text classification by turning each possible label into an NLI
premise/hypothesis pair.
 class transformers.pipelines.QuestionAnsweringArgumentHandler  < source > ( )    QuestionAnsweringPipeline requires the user to provide multiple arguments (i.e. question & context) to be mapped to
internal SquadExample. QuestionAnsweringArgumentHandler manages all the possible to create a SquadExample from the command-line
supplied arguments.


Data format

class transformers.PipelineDataFormat  < source > ( output_path: typing.Optional[str] input_path: typing.Optional[str] column: typing.Optional[str] overwrite: bool = False  )   Parameters   output_path (str) — Where to save the outgoing data.   input_path (str) — Where to look for the input data.   column (str) — The column to read.   overwrite (bool, optional, defaults to False) —
Whether or not to overwrite the output_path.    Base class for all the pipeline supported data format both for reading and writing. Supported data formats
currently includes: JSON CSV stdin/stdout (pipe) PipelineDataFormat also includes some utilities to work with multi-columns like mapping from datasets columns to
pipelines keyword arguments through the dataset_kwarg_1=dataset_column_1 format.  from_str  < source > ( format: str output_path: typing.Optional[str] input_path: typing.Optional[str] column: typing.Optional[str] overwrite = False  ) → PipelineDataFormat  Parameters   format (str) —
The format o


Data format

f the desired pipeline. Acceptable values are "json", "csv" or "pipe".   output_path (str, optional) —
Where to save the outgoing data.   input_path (str, optional) —
Where to look for the input data.   column (str, optional) —
The column to read.   overwrite (bool, optional, defaults to False) —
Whether or not to overwrite the output_path.   Returns
PipelineDataFormat

The proper data format.
  Creates an instance of the right subclass of PipelineDataFormat depending on format.  save  < source > ( data: typing.Union[dict, typing.List[dict]]  )   Parameters   data (dict or list of dict) — The data to store.    Save the provided data object with the representation for the current PipelineDataFormat.  save_binary  < source > ( data: typing.Union[dict, typing.List[dict]]  ) → str  Parameters   data (dict or list of dict) — The data to store.   Returns
str

Path where the data has been saved.
  Save the provided data object as a pickle-formatted binary data on the disk.
 class tran


Data format

sformers.CsvPipelineDataFormat  < source > ( output_path: typing.Optional[str] input_path: typing.Optional[str] column: typing.Optional[str] overwrite = False  )   Parameters   output_path (str) — Where to save the outgoing data.   input_path (str) — Where to look for the input data.   column (str) — The column to read.   overwrite (bool, optional, defaults to False) —
Whether or not to overwrite the output_path.    Support for pipelines using CSV data format.  save  < source > ( data: typing.List[dict]  )   Parameters   data (List[dict]) — The data to store.    Save the provided data object with the representation for the current PipelineDataFormat.
 class transformers.JsonPipelineDataFormat  < source > ( output_path: typing.Optional[str] input_path: typing.Optional[str] column: typing.Optional[str] overwrite = False  )   Parameters   output_path (str) — Where to save the outgoing data.   input_path (str) — Where to look for the input data.   column (str) — The column to read.   ove


Data format

rwrite (bool, optional, defaults to False) —
Whether or not to overwrite the output_path.    Support for pipelines using JSON file format.  save  < source > ( data: dict  )   Parameters   data (dict) — The data to store.    Save the provided data object in a json file.
 class transformers.PipedPipelineDataFormat  < source > ( output_path: typing.Optional[str] input_path: typing.Optional[str] column: typing.Optional[str] overwrite: bool = False  )   Parameters   output_path (str) — Where to save the outgoing data.   input_path (str) — Where to look for the input data.   column (str) — The column to read.   overwrite (bool, optional, defaults to False) —
Whether or not to overwrite the output_path.    Read data from piped input to the python process. For multi columns data, columns should separated by If columns are provided, then the output will be a dictionary with {column_x: value_x}  save  < source > ( data: dict  )   Parameters   data (dict) — The data to store.    Print the data.


Utilities

class transformers.pipelines.PipelineException  < source > ( task: str model: str reason: str  )   Parameters   task (str) — The task of the pipeline.   model (str) — The model used by the pipeline.   reason (str) — The error message to display.    Raised by a Pipeline when handling call.


Utilities for Trainer

This page lists all the utility functions used by Trainer.
Most of those are only useful if you are studying the code of the Trainer in the library.


Utilities

class transformers.EvalPrediction  < source > ( predictions: typing.Union[numpy.ndarray, typing.Tuple[numpy.ndarray]] label_ids: typing.Union[numpy.ndarray, typing.Tuple[numpy.ndarray]] inputs: typing.Union[numpy.ndarray, typing.Tuple[numpy.ndarray], NoneType] = None  )   Parameters   predictions (np.ndarray) — Predictions of the model.   label_ids (np.ndarray) — Targets to be matched.   inputs (np.ndarray, optional) —    Evaluation output (always contains labels), to be used to compute metrics.
 class transformers.IntervalStrategy  < source > ( value names = None module = None qualname = None type = None start = 1  )    An enumeration.
 transformers.enable_full_determinism  < source > ( seed: int warn_onl


Utilities

y: bool = False  )    Helper function for reproducible behavior during distributed training. See https://pytorch.org/docs/stable/notes/randomness.html for pytorch https://www.tensorflow.org/api_docs/python/tf/config/experimental/enable_op_determinism for tensorflow
 transformers.set_seed  < source > ( seed: int  )   Parameters   seed (int) — The seed to set.    Helper function for reproducible behavior to set the seed in random, numpy, torch and/or tf (if installed).
 transformers.torch_distributed_zero_first  < source > ( local_rank: int  )   Parameters   local_rank (int) — The rank of the local process.    Decorator to make all processes in distributed training wait for each local_master to do something.


Callbacks internals

class transformers.trainer_callback.CallbackHandler  < source > ( callbacks model tokenizer optimizer lr_scheduler  )    Internal class that just calls the list of callbacks in order.


Distributed Evaluation

class transformers.trainer_pt_utils.DistributedTensorGatherer  < source > ( world_size num_samples make_multiple_of = None padding_index = -100  )   Parameters   world_size (int) —
The number of processes used in the distributed training.   num_samples (int) —
The number of samples in our dataset.   make_multiple_of (int, optional) —
If passed, the class assumes the datasets passed to each process are made to be a multiple of this argument
(by adding samples).   padding_index (int, optional, defaults to -100) —
The padding index to use if the arrays don’t all have the same sequence length.    A class responsible for properly gathering tensors (or nested list/tuple of tensors) on the CPU by chunks. If our dataset has


Distributed Evaluation

 16 samples with a batch size of 2 on 3 processes and we gather then transfer on CPU at every
step, our sampler will generate the following indices: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 0, 1] to get something of size a multiple of 3 (so that each process gets the same dataset length). Then process 0, 1 and
2 will be responsible of making predictions for the following samples: P0: [0, 1, 2, 3, 4, 5] P1: [6, 7, 8, 9, 10, 11] P2: [12, 13, 14, 15, 0, 1] The first batch treated on each process will be P0: [0, 1] P1: [6, 7] P2: [12, 13] So if we gather at the end of the first batch, we will get a tensor (nested list/tuple of tensor) corresponding to
the following indices: [0, 1, 6, 7, 12, 13] If we direc


Distributed Evaluation

tly concatenate our results without taking any precautions, the user will then get the predictions for
the indices in this order at the end of the prediction loop: [0, 1, 6, 7, 12, 13, 2, 3, 8, 9, 14, 15, 4, 5, 10, 11, 0, 1] For some reason, that’s not going to roll their boat. This class is there to solve that problem.  add_arrays  < source > ( arrays  )    Add arrays to the internal storage, Will initialize the storage to the full size at the first arrays passed
so that if we’re bound to get an OOM, it happens at the beginning.  finalize  < source > ( )    Return the properly gathered arrays and truncate to the number of samples (since the sampler added some extras
to get each process a dataset of the same length)


Distributed Evaluation

class transformers.HfArgumentParser  < source > ( dataclass_types: typing.Union[DataClassType, typing.Iterable[DataClassType]] **kwargs  )    This subclass of argparse.ArgumentParser uses type hints on dataclasses to generate arguments. The class is designed to play well with the native argparse. In particular, you can add more (non-dataclass backed)
arguments to the parser after initialization and you’ll get the output back after parsing as an additional
namespace. Optional: To create sub argument groups use the _argument_group_name attribute in the dataclass.  parse_args_into_dataclasses  < source > ( args = None return_remaining_strings = False look_for_args_file = True args_filename = None args_file_flag = None  ) → Tuple consisting of  Returns
Tuple c


Distributed Evaluation

onsisting of

the dataclass instances in the same order as they were passed to the initializer.abspath
if applicable, an additional namespace for more (non-dataclass backed) arguments added to the parser
after initialization.
The potential list of remaining argument strings. (same as argparse.ArgumentParser.parse_known_args)
  Parse command-line args into instances of the specified dataclass types. This relies on argparse’s ArgumentParser.parse_known_args. See the doc at:
docs.python.org/3.7/library/argparse.html#argparse.ArgumentParser.parse_args  parse_dict  < source > ( args: typing.Dict[str, typing.Any] allow_extra_keys: bool = False  ) → Tuple consisting of  Parameters   args (dict) —
dict containing config values   allow_extra_keys (bool, optional,


Distributed Evaluation

defaults to False) —
Defaults to False. If False, will raise an exception if the dict contains keys that are not parsed.   Returns
Tuple consisting of

the dataclass instances in the same order as they were passed to the initializer.
  Alternative helper method that does not use argparse at all, instead uses a dict and populating the dataclass
types.  parse_json_file  < source > ( json_file: str allow_extra_keys: bool = False  ) → Tuple consisting of  Parameters   json_file (str or os.PathLike) —
File name of the json file to parse   allow_extra_keys (bool, optional, defaults to False) —
Defaults to False. If False, will raise an exception if the json file contains keys that are not
parsed.   Returns
Tuple consisting of

the dataclass instances in the


Distributed Evaluation

 same order as they were passed to the initializer.
  Alternative helper method that does not use argparse at all, instead loading a json file and populating the
dataclass types.  parse_yaml_file  < source > ( yaml_file: str allow_extra_keys: bool = False  ) → Tuple consisting of  Parameters   yaml_file (str or os.PathLike) —
File name of the yaml file to parse   allow_extra_keys (bool, optional, defaults to False) —
Defaults to False. If False, will raise an exception if the json file contains keys that are not
parsed.   Returns
Tuple consisting of

the dataclass instances in the same order as they were passed to the initializer.
  Alternative helper method that does not use argparse at all, instead loading a yaml file and populating the
dataclass types


Debug Utilities

class transformers.debug_utils.DebugUnderflowOverflow  < source > ( model max_frames_to_save = 21 trace_batch_nums = [] abort_after_batch_num = None  )   Parameters   model (nn.Module) —
The model to debug.   max_frames_to_save (int, optional, defaults to 21) —
How many frames back to record   trace_batch_nums(List[int], optional, defaults to []) —
Which batch numbers to trace (turns detection off)   abort_after_batch_num  (`int“, optional) —
Whether to abort after a certain batch number has finished    This debug class helps detect and understand where the model starts getting very large or very small, and more
importantly nan or inf weight and activation elements. There are 2 working modes: Underflow/overflow detection (default) Specific batch absolute min/max tracing without detection Mode 1: Underflow/overflow detection  To activate the underflow/overflow detection, initialize the ob


Debug Utilities

ject with the model :   Copied debug_overflow = DebugUnderflowOverflow(model) then run the training as normal and if nan or inf gets detected in at least one of the weight, input or output
elements this module will throw an exception and will print max_frames_to_save frames that lead to this event,
each frame reporting the fully qualified module name plus the class name whose forward was run the absolute min and max value of all elements for each module weights, and the inputs and output For example, here is the header and the last few frames in detection report for google/mt5-small run in fp16  mixed precision :   Copied Detected inf/nan during batch_number=0
Last 21 forward frames:
abs min  abs max  metadata
[...]
                  encoder.block.2.layer.1.DenseReluDense.wi_0 Linear
2.17e-07 4.50e+00 weight
1.79e-06 4.65e+00 input[0]
2.68e-06 3.70e+01 output
                  encoder.bl


Debug Utilities

ock.2.layer.1.DenseReluDense.wi_1 Linear
8.08e-07 2.66e+01 weight
1.79e-06 4.65e+00 input[0]
1.27e-04 2.37e+02 output
                  encoder.block.2.layer.1.DenseReluDense.wo Linear
1.01e-06 6.44e+00 weight
0.00e+00 9.74e+03 input[0]
3.18e-04 6.27e+04 output
                  encoder.block.2.layer.1.DenseReluDense T5DenseGatedGeluDense
1.79e-06 4.65e+00 input[0]
3.18e-04 6.27e+04 output
                  encoder.block.2.layer.1.dropout Dropout
3.18e-04 6.27e+04 input[0]
0.00e+00      inf output You can see here, that T5DenseGatedGeluDense.forward resulted in output activations, whose absolute max value was
around 62.7K, which is very close to fp16’s top limit of 64K. In the next frame we have Dropout which
renormalizes the weights, after it zeroed some of the elements, which pushes the absolute max value to more than
64K, and we get an overlow. As you can see it’s the previous frames


Debug Utilities

that we need to look into when the numbers start going into very large for
fp16 numbers. The tracking is done in a forward hook, which gets invoked immediately after forward has completed.  By default the last 21 frames are printed. You can change the default to adjust for your needs. For example :   Copied debug_overflow = DebugUnderflowOverflow(model, max_frames_to_save=100) To validate that you have set up this debugging feature correctly, and you intend to use it in a training that
may take hours to complete, first run it with normal tracing enabled for one of a few batches as explained in
the next section. Mode 2. Specific batch absolute min/max tracing without detection The second work mode is per-batch tracing with the underflow/overflow detection feature turned off. Let’s say you want to watch the absolute min and max values for all the ingredients of each forward call of a  give


Debug Utilities

n batch, and only do that for batches 1 and 3. Then you instantiate this class as :   Copied debug_overflow = DebugUnderflowOverflow(model, trace_batch_nums=[1, 3]) And now full batches 1 and 3 will be traced using the same format as explained above. Batches are 0-indexed. This is helpful if you know that the program starts misbehaving after a certain batch number, so you can
fast-forward right to that area. Early stopping:  You can also specify the batch number after which to stop the training, with :   Copied debug_overflow = DebugUnderflowOverflow(model, trace_batch_nums=[1, 3], abort_after_batch_num=3) This feature is mainly useful in the tracing mode, but you can use it for any mode. Performance: As this module measures absolute min/`max of each weight of the model on every forward it’ll slow the training
down. Therefore remember to turn it off once the debugging needs have been met


Utilities for Image Processors

This page lists all the utility functions used by the image processors, mainly the functional
transformations used to process the images.
Most of those are only useful if you are studying the code of the image processors in the library.


Image Transformations

transformers.image_transforms.center_crop  < source > ( image: ndarray size: typing.Tuple[int, int] data_format: typing.Union[transformers.image_utils.ChannelDimension, str, NoneType] = None input_data_format: typing.Union[transformers.image_utils.ChannelDimension, str, NoneType] = None return_numpy: typing.Optional[bool] = None  ) → np.ndarray  Parameters   image (np.ndarray) —
The image to crop.   size (Tuple[int, int]) —
The target size for the cropped image.   data_format (str or ChannelDimension, optional) —
The channel dimension format for the output image. Can be one of:
"channels_first" or ChannelDimension.FIRST: image in (num_channels, height, width) format.
"channels_last" or ChannelDimension.LAST: image in (height, width, num_channels) format.
If unset, will use the inferred format of the input image.
   input_data_format (str or ChannelDimension, optional) —
The channel dimension format for the input image. Can be one of:
"channels_fi


Image Transformations

rst" or ChannelDimension.FIRST: image in (num_channels, height, width) format.
"channels_last" or ChannelDimension.LAST: image in (height, width, num_channels) format.
If unset, will use the inferred format of the input image.
   return_numpy (bool, optional) —
Whether or not to return the cropped image as a numpy array. Used for backwards compatibility with the
previous ImageFeatureExtractionMixin method.
Unset: will return the same type as the input image.
True: will return a numpy array.
False: will return a PIL.Image.Image object.
   Returns
np.ndarray

The cropped image.
  Crops the image to the specified size using a center crop. Note that if the image is too small to be cropped to
the size given, it will be padded (so the returned result will always be of size size).
 transformers.image_transforms.center_to_corners_format  < source > ( bboxes_center: TensorType  )    Converts bounding boxes from center format to corners format. center f


Image Transformations

ormat: contains the coordinate for the center of the box and its width, height dimensions
(center_x, center_y, width, height)
corners format: contains the coodinates for the top-left and bottom-right corners of the box
(top_left_x, top_left_y, bottom_right_x, bottom_right_y)
 transformers.image_transforms.corners_to_center_format  < source > ( bboxes_corners: TensorType  )    Converts bounding boxes from corners format to center format. corners format: contains the coodinates for the top-left and bottom-right corners of the box
(top_left_x, top_left_y, bottom_right_x, bottom_right_y)
center format: contains the coordinate for the center of the box and its the width, height dimensions
(center_x, center_y, width, height)
 transformers.image_transforms.id_to_rgb  < source > ( id_map  )    Converts unique ID to RGB color.
 transformers.image_transforms.normalize  < source > ( image: ndarray mean: typing.Union[float, typing.Iterable[float]] std: typin


Image Transformations

g.Union[float, typing.Iterable[float]] data_format: typing.Optional[transformers.image_utils.ChannelDimension] = None input_data_format: typing.Union[transformers.image_utils.ChannelDimension, str, NoneType] = None  )   Parameters   image (np.ndarray) —
The image to normalize.   mean (float or Iterable[float]) —
The mean to use for normalization.   std (float or Iterable[float]) —
The standard deviation to use for normalization.   data_format (ChannelDimension, optional) —
The channel dimension format of the output image. If unset, will use the inferred format from the input.   input_data_format (ChannelDimension, optional) —
The channel dimension format of the input image. If unset, will use the inferred format from the input.    Normalizes image using the mean and standard deviation specified by mean and std. image = (image - mean) / std
 transformers.image_transforms.pad  < source > ( image: ndarray padding: typing.Union[int, typing.Tuple[int,


Image Transformations

 int], typing.Iterable[typing.Tuple[int, int]]] mode: PaddingMode = <PaddingMode.CONSTANT: 'constant'> constant_values: typing.Union[float, typing.Iterable[float]] = 0.0 data_format: typing.Union[transformers.image_utils.ChannelDimension, str, NoneType] = None input_data_format: typing.Union[transformers.image_utils.ChannelDimension, str, NoneType] = None  ) → np.ndarray  Parameters   image (np.ndarray) —
The image to pad.   padding (int or Tuple[int, int] or Iterable[Tuple[int, int]]) —
Padding to apply to the edges of the height, width axes. Can be one of three formats:
((before_height, after_height), (before_width, after_width)) unique pad widths for each axis.
((before, after),) yields same before and after pad for height and width.
(pad,) or int is a shortcut for before = after = pad width for all axes.
   mode (PaddingMode) —
The padding mode to use. Can be one of:
"constant": pads with a constant value.
"reflect": pads with the reflection


Image Transformations

of the vector mirrored on the first and last values of the
vector along each axis.
"replicate": pads with the replication of the last value on the edge of the array along each axis.
"symmetric": pads with the reflection of the vector mirrored along the edge of the array.
   constant_values (float or Iterable[float], optional) —
The value to use for the padding if mode is "constant".   data_format (str or ChannelDimension, optional) —
The channel dimension format for the output image. Can be one of:
"channels_first" or ChannelDimension.FIRST: image in (num_channels, height, width) format.
"channels_last" or ChannelDimension.LAST: image in (height, width, num_channels) format.
If unset, will use same as the input image.
   input_data_format (str or ChannelDimension, optional) —
The channel dimension format for the input image. Can be one of:
"channels_first" or ChannelDimension.FIRST: image in (num_channels, height, width) format.
"channels_last" o


Image Transformations

r ChannelDimension.LAST: image in (height, width, num_channels) format.
If unset, will use the inferred format of the input image.
   Returns
np.ndarray

The padded image.
  Pads the image with the specified (height, width) padding and mode.
 transformers.image_transforms.rgb_to_id  < source > ( color  )    Converts RGB color to unique ID.
 transformers.image_transforms.rescale  < source > ( image: ndarray scale: float data_format: typing.Optional[transformers.image_utils.ChannelDimension] = None dtype: dtype = <class 'numpy.float32'> input_data_format: typing.Union[transformers.image_utils.ChannelDimension, str, NoneType] = None  ) → np.ndarray  Parameters   image (np.ndarray) —
The image to rescale.   scale (float) —
The scale to use for rescaling the image.   data_format (ChannelDimension, optional) —
The channel dimension format of the image. If not provided, it will be the same as the input image.   dtype (np.dtype, optional, defaults to


Image Transformations

np.float32) —
The dtype of the output image. Defaults to np.float32. Used for backwards compatibility with feature
extractors.   input_data_format (ChannelDimension, optional) —
The channel dimension format of the input image. If not provided, it will be inferred from the input image.   Returns
np.ndarray

The rescaled image.
  Rescales image by scale.
 transformers.image_transforms.resize  < source > ( image size: typing.Tuple[int, int] resample: PILImageResampling = None reducing_gap: typing.Optional[int] = None data_format: typing.Optional[transformers.image_utils.ChannelDimension] = None return_numpy: bool = True input_data_format: typing.Union[transformers.image_utils.ChannelDimension, str, NoneType] = None  ) → np.ndarray  Parameters   image (PIL.Image.Image or np.ndarray or torch.Tensor) —
The image to resize.   size (Tuple[int, int]) —
The size to use for resizing the image.   resample (int, optional, defaults to PILImageResampling.BIL


Image Transformations

INEAR) —
The filter to user for resampling.   reducing_gap (int, optional) —
Apply optimization by resizing the image in two steps. The bigger reducing_gap, the closer the result to
the fair resampling. See corresponding Pillow documentation for more details.   data_format (ChannelDimension, optional) —
The channel dimension format of the output image. If unset, will use the inferred format from the input.   return_numpy (bool, optional, defaults to True) —
Whether or not to return the resized image as a numpy array. If False a PIL.Image.Image object is
returned.   input_data_format (ChannelDimension, optional) —
The channel dimension format of the input image. If unset, will use the inferred format from the input.   Returns
np.ndarray

The resized image.
  Resizes image to (height, width) specified by size using the PIL library.
 transformers.image_transforms.to_pil_image  < source > ( image: typing.Union[numpy.ndarray, ForwardRef('PIL.Image.


Image Transformations

Image'), ForwardRef('torch.Tensor'), ForwardRef('tf.Tensor'), ForwardRef('jnp.ndarray')] do_rescale: typing.Optional[bool] = None input_data_format: typing.Union[transformers.image_utils.ChannelDimension, str, NoneType] = None  ) → PIL.Image.Image  Parameters   image (PIL.Image.Image or numpy.ndarray or torch.Tensor or tf.Tensor) —
The image to convert to the PIL.Image format.   do_rescale (bool, optional) —
Whether or not to apply the scaling factor (to make pixel values integers between 0 and 255). Will default
to True if the image type is a floating type and casting to int would result in a loss of precision,
and False otherwise.   input_data_format (ChannelDimension, optional) —
The channel dimension format of the input image. If unset, will use the inferred format from the input.   Returns
PIL.Image.Image

The converted image.
  Converts image to a PIL Image. Optionally rescales it and puts the channel dimension back as the last axis if
n


ImageProcessingMixin

class transformers.ImageProcessingMixin  < source > ( **kwargs  )    This is an image processor mixin used to provide saving/loading functionality for sequential and image feature
extractors.  fetch_images  < source > ( image_url_or_urls: typing.Union[str, typing.List[str]]  )    Convert a single or a list of urls into the corresponding PIL.Image objects. If a single url is passed, the return value will be a single object. If a list is passed a list of objects is
returned.  from_dict  < source > ( image_processor_dict: typing.Dict[str, typing.Any] **kwargs  ) → ImageProcessingMixin  Parameters   image_processor_dict (Dict[str, Any]) —
Dictionary that will be used to instantiate the image processor object. Such a dictionary can be
retrieved from a pretrained checkpoint by leveraging the
to_dict() method.   kwargs (Dict[str, Any]) —
Additional parameters from which to initialize the image processor object.   Returns
ImageProcessingMixin

The image processor


ImageProcessingMixin

 object instantiated from those
parameters.
  Instantiates a type of ImageProcessingMixin from a Python dictionary of parameters.  from_json_file  < source > ( json_file: typing.Union[str, os.PathLike]  ) → A image processor of type ImageProcessingMixin  Parameters   json_file (str or os.PathLike) —
Path to the JSON file containing the parameters.   Returns
A image processor of type ImageProcessingMixin

The image_processor object
instantiated from that JSON file.
  Instantiates a image processor of type ImageProcessingMixin from the path to a JSON
file of parameters.  from_pretrained  < source > ( pretrained_model_name_or_path: typing.Union[str, os.PathLike] cache_dir: typing.Union[str, os.PathLike, NoneType] = None force_download: bool = False local_files_only: bool = False token: typing.Union[bool, str, NoneType] = None revision: str = 'main' **kwargs  )   Parameters   pretrained_model_name_or_path (str or os.PathLike) —
This can be either:
a string, t


ImageProcessingMixin

he model id of a pretrained image_processor hosted inside a model repo on
huggingface.co. Valid model ids can be located at the root-level, like bert-base-uncased, or
namespaced under a user or organization name, like dbmdz/bert-base-german-cased.
a path to a directory containing a image processor file saved using the
save_pretrained() method, e.g.,
./my_model_directory/.
a path or url to a saved image processor JSON file, e.g.,
./my_model_directory/preprocessor_config.json.
   cache_dir (str or os.PathLike, optional) —
Path to a directory in which a downloaded pretrained model image processor should be cached if the
standard cache should not be used.   force_download (bool, optional, defaults to False) —
Whether or not to force to (re-)download the image processor files and override the cached versions if
they exist.   resume_download (bool, optional, defaults to False) —
Whether or not to delete incompletely received file. Attempts to resume the download i


ImageProcessingMixin

f such a file
exists.   proxies (Dict[str, str], optional) —
A dictionary of proxy servers to use by protocol or endpoint, e.g., {'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}. The proxies are used on each request.   token (str or bool, optional) —
The token to use as HTTP bearer authorization for remote files. If True, or not specified, will use
the token generated when running huggingface-cli login (stored in ~/.huggingface).   revision (str, optional, defaults to "main") —
The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
git-based system for storing models and other artifacts on huggingface.co, so revision can be any
identifier allowed by git.    Instantiate a type of ImageProcessingMixin from an image processor.  Examples:   Copied # We can't instantiate directly the base class *ImageProcessingMixin* so let's show the examples on a
# derived class: *CLIPImageProcessor*
image_processor = C


ImageProcessingMixin

LIPImageProcessor.from_pretrained(
    "openai/clip-vit-base-patch32"
)  # Download image_processing_config from huggingface.co and cache.
image_processor = CLIPImageProcessor.from_pretrained(
    "./test/saved_model/"
)  # E.g. image processor (or model) was saved using *save_pretrained('./test/saved_model/')*
image_processor = CLIPImageProcessor.from_pretrained("./test/saved_model/preprocessor_config.json")
image_processor = CLIPImageProcessor.from_pretrained(
    "openai/clip-vit-base-patch32", do_normalize=False, foo=False
)
assert image_processor.do_normalize is False
image_processor, unused_kwargs = CLIPImageProcessor.from_pretrained(
    "openai/clip-vit-base-patch32", do_normalize=False, foo=False, return_unused_kwargs=True
)
assert image_processor.do_normalize is False
assert unused_kwargs == {"foo": False}  get_image_processor_dict  < source > ( pretrained_model_name_or_path: typing.Union[str, os.PathLike] **kwargs  ) → Tuple[Dict, Dict]  Parameter


ImageProcessingMixin

s   pretrained_model_name_or_path (str or os.PathLike) —
The identifier of the pre-trained checkpoint from which we want the dictionary of parameters.   subfolder (str, optional, defaults to "") —
In case the relevant files are located inside a subfolder of the model repo on huggingface.co, you can
specify the folder name here.   Returns
Tuple[Dict, Dict]

The dictionary(ies) that will be used to instantiate the image processor object.
  From a pretrained_model_name_or_path, resolve to a dictionary of parameters, to be used for instantiating a
image processor of type ~image_processor_utils.ImageProcessingMixin using from_dict.  push_to_hub  < source > ( repo_id: str use_temp_dir: typing.Optional[bool] = None commit_message: typing.Optional[str] = None private: typing.Optional[bool] = None token: typing.Union[bool, str, NoneType] = None max_shard_size: typing.Union[int, str, NoneType] = '10GB' create_pr: bool = False safe_serialization: bool = False revisi


ImageProcessingMixin

on: str = None **deprecated_kwargs  )   Parameters   repo_id (str) —
The name of the repository you want to push your image processor to. It should contain your organization name
when pushing to a given organization.   use_temp_dir (bool, optional) —
Whether or not to use a temporary directory to store the files saved before they are pushed to the Hub.
Will default to True if there is no directory named like repo_id, False otherwise.   commit_message (str, optional) —
Message to commit while pushing. Will default to "Upload image processor".   private (bool, optional) —
Whether or not the repository created should be private.   token (bool or str, optional) —
The token to use as HTTP bearer authorization for remote files. If True, will use the token generated
when running huggingface-cli login (stored in ~/.huggingface). Will default to True if repo_url
is not specified.   max_shard_size (int or str, optional, defaults to "10GB") —
Only applicable for models


ImageProcessingMixin

. The maximum size for a checkpoint before being sharded. Checkpoints shard
will then be each of size lower than this size. If expressed as a string, needs to be digits followed
by a unit (like "5MB").   create_pr (bool, optional, defaults to False) —
Whether or not to create a PR with the uploaded files or directly commit.   safe_serialization (bool, optional, defaults to False) —
Whether or not to convert the model weights in safetensors format for safer serialization.   revision (str, optional) —
Branch to push the uploaded files to.    Upload the image processor file to the 🤗 Model Hub.  Examples:   Copied from transformers import AutoImageProcessor
image processor = AutoImageProcessor.from_pretrained("bert-base-cased")
# Push the image processor to your namespace with the name "my-finetuned-bert".
image processor.push_to_hub("my-finetuned-bert")
# Push the image processor to an organization with the name "my-finetuned-bert".
image processor.push_to_hub(


ImageProcessingMixin

"huggingface/my-finetuned-bert")  register_for_auto_class  < source > ( auto_class = 'AutoImageProcessor'  )   Parameters   auto_class (str or type, optional, defaults to "AutoImageProcessor ") —
The auto class to register this new image processor with.    Register this class with a given auto class. This should only be used for custom image processors as the ones
in the library are already mapped with AutoImageProcessor . This API is experimental and may have some slight breaking changes in the next releases.  save_pretrained  < source > ( save_directory: typing.Union[str, os.PathLike] push_to_hub: bool = False **kwargs  )   Parameters   save_directory (str or os.PathLike) —
Directory where the image processor JSON file will be saved (will be created if it does not exist).   push_to_hub (bool, optional, defaults to False) —
Whether or not to push your model to the Hugging Face model hub after saving it. You can specify the
repository you want to push to wit


ImageProcessingMixin

h repo_id (will default to the name of save_directory in your
namespace).   kwargs (Dict[str, Any], optional) —
Additional key word arguments passed along to the push_to_hub() method.    Save an image processor object to the directory save_directory, so that it can be re-loaded using the
from_pretrained() class method.  to_dict  < source > ( ) → Dict[str, Any]  Returns
Dict[str, Any]

Dictionary of all the attributes that make up this image processor instance.
  Serializes this instance to a Python dictionary.  to_json_file  < source > ( json_file_path: typing.Union[str, os.PathLike]  )   Parameters   json_file_path (str or os.PathLike) —
Path to the JSON file in which this image_processor instance’s parameters will be saved.    Save this instance to a JSON file.  to_json_string  < source > ( ) → str  Returns
str

String containing all the attributes that make up this feature_extractor instance in JSON format.
  Serializes this instance to a JSON strin


Utilities for FeatureExtractors

This page lists all the utility functions that can be used by the audio FeatureExtractor in order to compute special features from a raw audio using common algorithms such as Short Time Fourier Transform or log mel spectrogram.
Most of those are only useful if you are studying the code of the audio processors in the library.


Audio Transformations

transformers.audio_utils.hertz_to_mel  < source > ( freq: typing.Union[float, numpy.ndarray] mel_scale: str = 'htk'  ) → float or np.ndarray  Parameters   freq (float or np.ndarray) —
The frequency, or multiple frequencies, in hertz (Hz).   mel_scale (str, optional, defaults to "htk") —
The mel frequency scale to use, "htk", "kaldi" or "slaney".   Returns
float or np.ndarray

The frequencies on the mel scale.
  Convert frequency from hertz to mels.
 transformers.audio_utils.mel_to_hertz  < source > ( mels: typing.Union[float, numpy.ndarray] mel_scale: str = 'htk'  ) → float or np.ndarray  Parameters   mels (float or np.ndarray) —
The frequency, or multiple frequencies, in mels.   mel_scale (str, optional, "htk") —
The mel frequency scale to use, "htk", "kaldi" or "slaney".   Returns
float or np.ndarray

The frequencies in hertz.
  Convert frequency from mels to hertz.
 transformers.audio_utils.mel_filter_bank  < source > ( num_frequenc


Audio Transformations

y_bins: int num_mel_filters: int min_frequency: float max_frequency: float sampling_rate: int norm: typing.Optional[str] = None mel_scale: str = 'htk' triangularize_in_mel_space: bool = False  ) → np.ndarray of shape (num_frequency_bins, num_mel_filters)  Parameters   num_frequency_bins (int) —
Number of frequencies used to compute the spectrogram (should be the same as in stft).   num_mel_filters (int) —
Number of mel filters to generate.   min_frequency (float) —
Lowest frequency of interest in Hz.   max_frequency (float) —
Highest frequency of interest in Hz. This should not exceed sampling_rate / 2.   sampling_rate (int) —
Sample rate of the audio waveform.   norm (str, optional) —
If "slaney", divide the triangular mel weights by the width of the mel band (area normalization).   mel_scale (str, optional, defaults to "htk") —
The mel frequency scale to use, "htk", "kaldi" or "slaney".   triangularize_in_mel_space (bool, optional, default


Audio Transformations

s to False) —
If this option is enabled, the triangular filter is applied in mel space rather than frequency space. This
should be set to true in order to get the same results as torchaudio when computing mel filters.   Returns
np.ndarray of shape (num_frequency_bins, num_mel_filters)

Triangular filter bank matrix. This is a
projection matrix to go from a spectrogram to a mel spectrogram.
  Creates a frequency bin conversion matrix used to obtain a mel spectrogram. This is called a mel filter bank, and
various implementation exist, which differ in the number of filters, the shape of the filters, the way the filters
are spaced, the bandwidth of the filters, and the manner in which the spectrum is warped. The goal of these
features is to approximate the non-linear human perception of the variation in pitch with respect to the frequency. Different banks of mel filters were introduced in the literature. The following variations are supported


Audio Transformations

: MFCC FB-20: introduced in 1980 by Davis and Mermelstein, it assumes a sampling frequency of 10 kHz and a speech
bandwidth of [0, 4600] Hz. MFCC FB-24 HTK: from the Cambridge HMM Toolkit (HTK) (1995) uses a filter bank of 24 filters for a speech
bandwidth of [0, 8000] Hz. This assumes sampling rate ≥ 16 kHz. MFCC FB-40: from the Auditory Toolbox for MATLAB written by Slaney in 1998, assumes a sampling rate of 16 kHz and
speech bandwidth of [133, 6854] Hz. This version also includes area normalization. HFCC-E FB-29 (Human Factor Cepstral Coefficients) of Skowronski and Harris (2004), assumes a sampling rate of
12.5 kHz and speech bandwidth of [0, 6250] Hz. This code is adapted from torchaudio and librosa. Note that the default parameters of torchaudio’s
melscale_fbanks implement the "htk" filters while librosa uses the "slaney" implementation.
 transformers.audio_utils.optimal_fft_length  < source > ( window_length: int  )    Finds the best


Audio Transformations

FFT input size for a given window_length. This function takes a given window length and, if not
already a power of two, rounds it up to the next power or two. The FFT algorithm works fastest when the length of the input is a power of two, which may be larger than the size
of the window or analysis frame. For example, if the window is 400 samples, using an FFT input size of 512 samples
is more optimal than an FFT size of 400 samples. Using a larger FFT size does not affect the detected frequencies,
it simply gives a higher frequency resolution (i.e. the frequency bins are smaller).
 transformers.audio_utils.window_function  < source > ( window_length: int name: str = 'hann' periodic: bool = True frame_length: typing.Optional[int] = None center: bool = True  )   Parameters   window_length (int) —
The length of the window in samples.   name (str, optional, defaults to "hann") —
The name of the window function.   periodic (bool, optional, defaul


Audio Transformations

ts to True) —
Whether the window is periodic or symmetric.   frame_length (int, optional) —
The length of the analysis frames in samples. Provide a value for frame_length if the window is smaller
than the frame length, so that it will be zero-padded.   center (bool, optional, defaults to True) —
Whether to center the window inside the FFT buffer. Only used when frame_length is provided.    Returns an array containing the specified window. This window is intended to be used with stft. The following window types are supported: "boxcar": a rectangular window "hamming": the Hamming window "hann": the Hann window "povey": the Povey window
 transformers.audio_utils.spectrogram  < source > ( waveform: ndarray window: ndarray frame_length: int hop_length: int fft_length: typing.Optional[int] = None power: typing.Optional[float] = 1.0 center: bool = True pad_mode: str = 'reflect' onesided: bool = True preemphasis: typing.Optional[float] = None mel_fi


Audio Transformations

lters: typing.Optional[numpy.ndarray] = None mel_floor: float = 1e-10 log_mel: typing.Optional[str] = None reference: float = 1.0 min_value: float = 1e-10 db_range: typing.Optional[float] = None remove_dc_offset: typing.Optional[bool] = None dtype: dtype = <class 'numpy.float32'>  )   Parameters   waveform (np.ndarray of shape (length,)) —
The input waveform. This must be a single real-valued, mono waveform.   window (np.ndarray of shape (frame_length,)) —
The windowing function to apply, including zero-padding if necessary. The actual window length may be
shorter than frame_length, but we’re assuming the array has already been zero-padded.   frame_length (int) —
The length of the analysis frames in samples. With librosa this is always equal to fft_length but we also
allow smaller sizes.   hop_length (int) —
The stride between successive analysis frames in samples.   fft_length (int, optional) —
The size of the FFT buffer in samples. This de


Audio Transformations

termines how many frequency bins the spectrogram will have.
For optimal speed, this should be a power of two. If None, uses frame_length.   power (float, optional, defaults to 1.0) —
If 1.0, returns the amplitude spectrogram. If 2.0, returns the power spectrogram. If None, returns
complex numbers.   center (bool, optional, defaults to True) —
Whether to pad the waveform so that frame t is centered around time t * hop_length. If False, frame
t will start at time t * hop_length.   pad_mode (str, optional, defaults to "reflect") —
Padding mode used when center is True. Possible values are: "constant" (pad with zeros), "edge"
(pad with edge values), "reflect" (pads with mirrored values).   onesided (bool, optional, defaults to True) —
If True, only computes the positive frequencies and returns a spectrogram containing fft_length // 2 + 1
frequency bins. If False, also computes the negative frequencies and returns fft_length frequency bins.   pre


Audio Transformations

emphasis (float, optional) —
Coefficient for a low-pass filter that applies pre-emphasis before the DFT.   mel_filters (np.ndarray of shape (num_freq_bins, num_mel_filters), optional) —
The mel filter bank. If supplied, applies a this filter bank to create a mel spectrogram.   mel_floor (float, optional, defaults to 1e-10) —
Minimum value of mel frequency banks.   log_mel (str, optional) —
How to convert the spectrogram to log scale. Possible options are: None (don’t convert), "log" (take
the natural logarithm) "log10" (take the base-10 logarithm), "dB" (convert to decibels). Can only be
used when power is not None.   reference (float, optional, defaults to 1.0) —
Sets the input spectrogram value that corresponds to 0 dB. For example, use np.max(spectrogram) to set
the loudest part to 0 dB. Must be greater than zero.   min_value (float, optional, defaults to 1e-10) —
The spectrogram will be clipped to this minimum value before conversion to


Audio Transformations

decibels, to avoid taking
log(0). For a power spectrogram, the default of 1e-10 corresponds to a minimum of -100 dB. For an
amplitude spectrogram, the value 1e-5 corresponds to -100 dB. Must be greater than zero.   db_range (float, optional) —
Sets the maximum dynamic range in decibels. For example, if db_range = 80, the difference between the
peak value and the smallest value will never be more than 80 dB. Must be greater than zero.   remove_dc_offset (bool, optional) —
Subtract mean from waveform on each frame, applied before pre-emphasis. This should be set to true in
order to get the same results as torchaudio.compliance.kaldi.fbank when computing mel filters.   dtype (np.dtype, optional, defaults to np.float32) —
Data type of the spectrogram tensor. If power is None, this argument is ignored and the dtype will be
np.complex64.    Calculates a spectrogram over one waveform using the Short-Time Fourier Transform. This function can create


Audio Transformations

the following kinds of spectrograms: amplitude spectrogram (power = 1.0) power spectrogram (power = 2.0) complex-valued spectrogram (power = None) log spectrogram (use log_mel argument) mel spectrogram (provide mel_filters) log-mel spectrogram (provide mel_filters and log_mel) How this works: The input waveform is split into frames of size frame_length that are partially overlapping by `frame_lengthhop_length` samples. Each frame is multiplied by the window and placed into a buffer of size fft_length. The DFT is taken of each windowed frame. The results are stacked into a spectrogram. We make a distinction between the following “blocks” of sample data, each of which may have a different lengths: The analysis frame. This is the size of the time slices that the input waveform is split into. The window. Each analysis frame is multiplied by the window to avoid spectral leakage. The FFT input buffer. The length of this determines how many frequen


Audio Transformations

cy bins are in the spectrogram. In this implementation, the window is assumed to be zero-padded to have the same size as the analysis frame. A
padded window can be obtained from window_function(). The FFT input buffer may be larger than the analysis frame,
typically the next power of two. Note: This function is not optimized for speed yet. It should be mostly compatible with librosa.stft and
torchaudio.functional.transforms.Spectrogram, although it is more flexible due to the different ways spectrograms
can be constructed.
 transformers.audio_utils.power_to_db  < source > ( spectrogram: ndarray reference: float = 1.0 min_value: float = 1e-10 db_range: typing.Optional[float] = None  ) → np.ndarray  Parameters   spectrogram (np.ndarray) —
The input power (mel) spectrogram. Note that a power spectrogram has the amplitudes squared!   reference (float, optional, defaults to 1.0) —
Sets the input spectrogram value that corresponds to 0 dB. For exa


Audio Transformations

mple, use np.max(spectrogram) to set
the loudest part to 0 dB. Must be greater than zero.   min_value (float, optional, defaults to 1e-10) —
The spectrogram will be clipped to this minimum value before conversion to decibels, to avoid taking
log(0). The default of 1e-10 corresponds to a minimum of -100 dB. Must be greater than zero.   db_range (float, optional) —
Sets the maximum dynamic range in decibels. For example, if db_range = 80, the difference between the
peak value and the smallest value will never be more than 80 dB. Must be greater than zero.   Returns
np.ndarray

the spectrogram in decibels
  Converts a power spectrogram to the decibel scale. This computes 10 * log10(spectrogram / reference), using basic
logarithm properties for numerical stability. The motivation behind applying the log function on the (mel) spectrogram is that humans do not hear loudness on a
linear scale. Generally to double the perceived volume of a sound


Audio Transformations

we need to put 8 times as much energy into it.
This means that large variations in energy may not sound all that different if the sound is loud to begin with.
This compression operation makes the (mel) spectrogram features match more closely what humans actually hear. Based on the implementation of librosa.power_to_db.
 transformers.audio_utils.amplitude_to_db  < source > ( spectrogram: ndarray reference: float = 1.0 min_value: float = 1e-05 db_range: typing.Optional[float] = None  ) → np.ndarray  Parameters   spectrogram (np.ndarray) —
The input amplitude (mel) spectrogram.   reference (float, optional, defaults to 1.0) —
Sets the input spectrogram value that corresponds to 0 dB. For example, use np.max(spectrogram) to set
the loudest part to 0 dB. Must be greater than zero.   min_value (float, optional, defaults to 1e-5) —
The spectrogram will be clipped to this minimum value before conversion to decibels, to avoid taking
log(0). The defau


Audio Transformations

lt of 1e-5 corresponds to a minimum of -100 dB. Must be greater than zero.   db_range (float, optional) —
Sets the maximum dynamic range in decibels. For example, if db_range = 80, the difference between the
peak value and the smallest value will never be more than 80 dB. Must be greater than zero.   Returns
np.ndarray

the spectrogram in decibels
  Converts an amplitude spectrogram to the decibel scale. This computes 20 * log10(spectrogram / reference), using
basic logarithm properties for numerical stability. The motivation behind applying the log function on the (mel) spectrogram is that humans do not hear loudness on a
linear scale. Generally to double the perceived volume of a sound we need to put 8 times as much energy into it.
This means that large variations in energy may not sound all that different if the sound is loud to begin with.
This compression operation makes the (mel) spectrogram features match more closely what humans a


Data Collator

Data collators are objects that will form a batch by using a list of dataset elements as input. These elements are of
the same type as the elements of train_dataset or eval_dataset.
To be able to build batches, data collators may apply some processing (like padding). Some of them (like
DataCollatorForLanguageModeling) also apply some random data augmentation (like random masking)
on the formed batch.
Examples of use can be found in the example scripts or example notebooks.


Default data collator

transformers.default_data_collator  < source > ( features: typing.List[InputDataClass] return_tensors = 'pt'  )    Very simple data collator that simply collates batches of dict-like objects and performs special handling for
potential keys named: label: handles a single value (int or float) per object label_ids: handles a list of values per object Does not do any additional preprocessing: property names of the input object will be used as corresponding inputs
to the model. See glue and ner for example of how it’s useful.


DefaultDataCollator

class transformers.DefaultDataCollator  < source > ( return_tensors: str = 'pt'  )   Parameters   return_tensors (str, optional, defaults to "pt") —
The type of Tensor to return. Allowable values are “np”, “pt” and “tf”.    Very simple data collator that simply collates batches of dict-like objects and performs special handling for
potential keys named: label: handles a single value (int or float) per object label_ids: handles a list of values per object Does not do any additional preprocessing: property names of the input object will be used as corresponding inputs
to the model. See glue and ner for example of how it’s useful. This is an object (like other data collators) rather than a pure function like default_data_collator. This can be
helpful if you need to set a return_tensors value at initialization.


DataCollatorWithPadding

class transformers.DataCollatorWithPadding  < source > ( tokenizer: PreTrainedTokenizerBase padding: typing.Union[bool, str, transformers.utils.generic.PaddingStrategy] = True max_length: typing.Optional[int] = None pad_to_multiple_of: typing.Optional[int] = None return_tensors: str = 'pt'  )   Parameters   tokenizer (PreTrainedTokenizer or PreTrainedTokenizerFast) —
The tokenizer used for encoding the data.   padding (bool, str or PaddingStrategy, optional, defaults to True) —
Select a strategy to pad the returned sequences (according to the model’s padding side and padding index)
among:
True or 'longest' (default): Pad to the longest sequence in the batch (or no padding if only a single
sequence is provided).
'max_length': Pad to a maximum le


DataCollatorWithPadding

ngth specified with the argument max_length or to the maximum
acceptable input length for the model if that argument is not provided.
False or 'do_not_pad': No padding (i.e., can output a batch with sequences of different lengths).
   max_length (int, optional) —
Maximum length of the returned list and optionally padding length (see above).   pad_to_multiple_of (int, optional) —
If set will pad the sequence to a multiple of the provided value.
This is especially useful to enable the use of Tensor Cores on NVIDIA hardware with compute capability >=
7.5 (Volta).   return_tensors (str, optional, defaults to "pt") —
The type of Tensor to return. Allowable values are “np”, “pt” and “tf”.    Data collator that will dynamically pad the inputs received


DataCollatorForTokenClassification

class transformers.DataCollatorForTokenClassification  < source > ( tokenizer: PreTrainedTokenizerBase padding: typing.Union[bool, str, transformers.utils.generic.PaddingStrategy] = True max_length: typing.Optional[int] = None pad_to_multiple_of: typing.Optional[int] = None label_pad_token_id: int = -100 return_tensors: str = 'pt'  )   Parameters   tokenizer (PreTrainedTokenizer or PreTrainedTokenizerFast) —
The tokenizer used for encoding the data.   padding (bool, str or PaddingStrategy, optional, defaults to True) —
Select a strategy to pad the returned sequences (according to the model’s padding side and padding index)
among:
True or 'longest' (default): Pad to the longest sequence in the batch (or no padding if only a single
sequence is provided).
'max_length': Pad to a maximum length specified with the argument max_length or to the maximum
acceptab


DataCollatorForTokenClassification

le input length for the model if that argument is not provided.
False or 'do_not_pad': No padding (i.e., can output a batch with sequences of different lengths).
   max_length (int, optional) —
Maximum length of the returned list and optionally padding length (see above).   pad_to_multiple_of (int, optional) —
If set will pad the sequence to a multiple of the provided value.
This is especially useful to enable the use of Tensor Cores on NVIDIA hardware with compute capability >=
7.5 (Volta).   label_pad_token_id (int, optional, defaults to -100) —
The id to use when padding the labels (-100 will be automatically ignore by PyTorch loss functions).   return_tensors (str, optional, defaults to "pt") —
The type of Tensor to return. Allowable values are “np”, “pt” and “tf”.    Data collator that will dynamically pad the inputs received, as well as the labels.


class transformers.DataCollatorForSeq2Seq  < source > ( tokenizer: PreTrainedTokenizerBase model: typing.Optional[typing.Any] = None padding: typing.Union[bool, str, transformers.utils.generic.PaddingStrategy] = True max_length: typing.Optional[int] = None pad_to_multiple_of: typing.Optional[int] = None label_pad_token_id: int = -100 return_tensors: str = 'pt'  )   Parameters   tokenizer (PreTrainedTokenizer or PreTrainedTokenizerFast) —
The tokenizer used for encoding the data.   model (PreTrainedModel, optional) —
The model that is being trained. If set and has the prepare_decoder_input_ids_from_labels, use it to
prepare the decoder_input_ids
This is useful w


hen using label_smoothing to avoid calculating loss twice.   padding (bool, str or PaddingStrategy, optional, defaults to True) —
Select a strategy to pad the returned sequences (according to the model’s padding side and padding index)
among:
True or 'longest' (default): Pad to the longest sequence in the batch (or no padding if only a single
sequence is provided).
'max_length': Pad to a maximum length specified with the argument max_length or to the maximum
acceptable input length for the model if that argument is not provided.
False or 'do_not_pad': No padding (i.e., can output a batch with sequences of different lengths).
   max_length (int, optional) —
Maxi


mum length of the returned list and optionally padding length (see above).   pad_to_multiple_of (int, optional) —
If set will pad the sequence to a multiple of the provided value.
This is especially useful to enable the use of Tensor Cores on NVIDIA hardware with compute capability >=
7.5 (Volta).   label_pad_token_id (int, optional, defaults to -100) —
The id to use when padding the labels (-100 will be automatically ignored by PyTorch loss functions).   return_tensors (str, optional, defaults to "pt") —
The type of Tensor to return. Allowable values are “np”, “pt” and “tf”.    Data collator that will dynamically pad the inputs received, as well as the labels.


DataCollatorForLanguageModeling

class transformers.DataCollatorForLanguageModeling  < source > ( tokenizer: PreTrainedTokenizerBase mlm: bool = True mlm_probability: float = 0.15 pad_to_multiple_of: typing.Optional[int] = None tf_experimental_compile: bool = False return_tensors: str = 'pt'  )   Parameters   tokenizer (PreTrainedTokenizer or PreTrainedTokenizerFast) —
The tokenizer used for encoding the data.   mlm (bool, optional, defaults to True) —
Whether or not to use masked language modeling. If set to False, the labels are the same as the inputs
with the padding tokens ignored (by setting them to -100). Otherwise, the labels are -100 for non-masked
tokens and the value to predict for the masked token.   mlm_probability (float


DataCollatorForLanguageModeling

, optional, defaults to 0.15) —
The probability with which to (randomly) mask tokens in the input, when mlm is set to True.   pad_to_multiple_of (int, optional) —
If set will pad the sequence to a multiple of the provided value.   return_tensors (str) —
The type of Tensor to return. Allowable values are “np”, “pt” and “tf”.    Data collator used for language modeling. Inputs are dynamically padded to the maximum length of a batch if they
are not all of the same length. For best performance, this data collator should be used with a dataset having items that are dictionaries or
BatchEncoding, with the "special_tokens_mask" key, as returned by a PreTrainedTokenizer or a
PreTrainedTokenizerFast with the a


DataCollatorForLanguageModeling

rgument return_special_tokens_mask=True.  numpy_mask_tokens  < source > ( inputs: typing.Any special_tokens_mask: typing.Optional[typing.Any] = None  )    Prepare masked tokens inputs/labels for masked language modeling: 80% MASK, 10% random, 10% original.  tf_mask_tokens  < source > ( inputs: typing.Any vocab_size mask_token_id special_tokens_mask: typing.Optional[typing.Any] = None  )    Prepare masked tokens inputs/labels for masked language modeling: 80% MASK, 10% random, 10% original.  torch_mask_tokens  < source > ( inputs: typing.Any special_tokens_mask: typing.Optional[typing.Any] = None  )    Prepare masked tokens inputs/labels for masked language modeling: 80% MASK, 10% random, 10% original.


DataCollatorForWholeWordMask

class transformers.DataCollatorForWholeWordMask  < source > ( tokenizer: PreTrainedTokenizerBase mlm: bool = True mlm_probability: float = 0.15 pad_to_multiple_of: typing.Optional[int] = None tf_experimental_compile: bool = False return_tensors: str = 'pt'  )    Data collator used for language modeling that masks entire words. collates batches of tensors, honoring their tokenizer’s pad_token preprocesses batches for masked language modeling This collator relies on details of the implementation of subword tokenization by BertTokenizer, specifically
that subword tokens are prefixed with ##. For tokenizers that do not adhere to this scheme, this collator will
produce an output that is roughly equivalent to .DataCollatorForLanguageModeling.  numpy_mask_tokens  < source > ( inputs: typing.A


DataCollatorForWholeWordMask

ny mask_labels: typing.Any  )    Prepare masked tokens inputs/labels for masked language modeling: 80% MASK, 10% random, 10% original. Set
‘mask_labels’ means we use whole word mask (wwm), we directly mask idxs according to it’s ref.  tf_mask_tokens  < source > ( inputs: typing.Any mask_labels: typing.Any  )    Prepare masked tokens inputs/labels for masked language modeling: 80% MASK, 10% random, 10% original. Set
‘mask_labels’ means we use whole word mask (wwm), we directly mask idxs according to it’s ref.  torch_mask_tokens  < source > ( inputs: typing.Any mask_labels: typing.Any  )    Prepare masked tokens inputs/labels for masked language modeling: 80% MASK, 10% random, 10% original. Set
‘mask_labels’ means we use whole word mask (wwm), we directly mask idxs according to it’s ref.


class transformers.DataCollatorForPermutationLanguageModeling  < source > ( tokenizer: PreTrainedTokenizerBase plm_probability: float = 0.16666666666666666 max_span_length: int = 5 return_tensors: str = 'pt'  )    Data collator used for permutation language modeling. collates batches of tensors, honoring their tokenizer’s pad_token preprocesses batches for permutation language modeling with procedures specific to XLNet  numpy_mask_tokens  < source > ( inputs: typing.Any  )    The masked tokens to be predicted for a particular sequence are determined by the following algorithm: Start from the beginning of the sequence by setting cur_len = 0 (number of tokens processed so far). Sample a span_length from the interval [1, max_span_length] (length of span of tokens to be masked) Reserve a context of length context_length = span_length / plm_probability to surround span to be
masked Sample a starting poin


t start_index from the interval [cur_len, cur_len + context_length - span_length] and mask tokens start_index:start_index + span_length Set cur_len = cur_len + context_length. If cur_len < max_len (i.e. there are tokens remaining in the
sequence to be processed), repeat from Step 1.  tf_mask_tokens  < source > ( inputs: typing.Any  )    The masked tokens to be predicted for a particular sequence are determined by the following algorithm: Start from the beginning of the sequence by setting cur_len = 0 (number of tokens processed so far). Sample a span_length from the interval [1, max_span_length] (length of span of tokens to be masked) Reserve a context of length context_length = span_length / plm_probability to surround span to be
masked Sample a starting point start_index from the interval [cur_len, cur_len + context_length - span_length] and mask tokens start_index:start_index + span_length Set cu


r_len = cur_len + context_length. If cur_len < max_len (i.e. there are tokens remaining in the
sequence to be processed), repeat from Step 1.  torch_mask_tokens  < source > ( inputs: typing.Any  )    The masked tokens to be predicted for a particular sequence are determined by the following algorithm: Start from the beginning of the sequence by setting cur_len = 0 (number of tokens processed so far). Sample a span_length from the interval [1, max_span_length] (length of span of tokens to be masked) Reserve a context of length context_length = span_length / plm_probability to surround span to be
masked Sample a starting point start_index from the interval [cur_len, cur_len + context_length - span_length] and mask tokens start_index:start_index + span_length Set cur_len = cur_len + context_length. If cur_len < max_len (i.e. there are tokens remaining in the
sequence to be processed), repeat from Step


Feature Extractor

A feature extractor is in charge of preparing input features for audio or vision models. This includes feature extraction
from sequences, e.g., pre-processing audio files to Log-Mel Spectrogram features, feature extraction from images
e.g. cropping image image files, but also padding, normalization, and conversion to Numpy, PyTorch, and TensorFlow
tensors.


FeatureExtractionMixin

class transformers.FeatureExtractionMixin  < source > ( **kwargs  )    This is a feature extraction mixin used to provide saving/loading functionality for sequential and image feature
extractors.  from_pretrained  < source > ( pretrained_model_name_or_path: typing.Union[str, os.PathLike] cache_dir: typing.Union[str, os.PathLike, NoneType] = None force_download: bool = False local_files_only: bool = False token: typing.Union[bool, str, NoneType] = None revision: str = 'main' **kwargs  )   Parameters   pretrained_model_name_or_path (str or os.PathLike) —
This can be either:
a string, the model id of a pretrained feature_extractor hosted inside a model repo on
huggingface.co. Valid model ids can be located at the root-level, like bert-base-uncased, or
namespaced under a user or organization name, like dbmdz/bert-base-german-cased.
a path to a directory co


FeatureExtractionMixin

ntaining a feature extractor file saved using the
save_pretrained() method, e.g.,
./my_model_directory/.
a path or url to a saved feature extractor JSON file, e.g.,
./my_model_directory/preprocessor_config.json.
   cache_dir (str or os.PathLike, optional) —
Path to a directory in which a downloaded pretrained model feature extractor should be cached if the
standard cache should not be used.   force_download (bool, optional, defaults to False) —
Whether or not to force to (re-)download the feature extractor files and override the cached versions
if they exist.   resume_download (bool, optional, defaults to False) —
Whether or not to delete incompletely received file. Attempts to resume the download if such a file
exists.   proxies (Dict[str, str], optional) —
A dictionary of proxy servers to use by protocol or endpoint, e.g., {'http': 'foo.bar:3128', 'h


FeatureExtractionMixin

ttp://hostname': 'foo.bar:4012'}. The proxies are used on each request.   token (str or bool, optional) —
The token to use as HTTP bearer authorization for remote files. If True, or not specified, will use
the token generated when running huggingface-cli login (stored in ~/.huggingface).   revision (str, optional, defaults to "main") —
The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
git-based system for storing models and other artifacts on huggingface.co, so revision can be any
identifier allowed by git.    Instantiate a type of FeatureExtractionMixin from a feature extractor, e.g. a
derived class of SequenceFeatureExtractor.  Examples:   Copied # We can't instantiate directly the base class *FeatureExtractionMixin* nor *SequenceFeatureExtractor* so let's show the examples on a
# derived class: *W


FeatureExtractionMixin

av2Vec2FeatureExtractor*
feature_extractor = Wav2Vec2FeatureExtractor.from_pretrained(
    "facebook/wav2vec2-base-960h"
)  # Download feature_extraction_config from huggingface.co and cache.
feature_extractor = Wav2Vec2FeatureExtractor.from_pretrained(
    "./test/saved_model/"
)  # E.g. feature_extractor (or model) was saved using *save_pretrained('./test/saved_model/')*
feature_extractor = Wav2Vec2FeatureExtractor.from_pretrained("./test/saved_model/preprocessor_config.json")
feature_extractor = Wav2Vec2FeatureExtractor.from_pretrained(
    "facebook/wav2vec2-base-960h", return_attention_mask=False, foo=False
)
assert feature_extractor.return_attention_mask is False
feature_extractor, unused_kwargs = Wav2Vec2FeatureExtractor.from_pretrained(
    "facebook/wav2vec2-base-960h", return_attention_mask=False, foo=False, return_unused_kwargs=True
)
assert


FeatureExtractionMixin

 feature_extractor.return_attention_mask is False
assert unused_kwargs == {"foo": False}  save_pretrained  < source > ( save_directory: typing.Union[str, os.PathLike] push_to_hub: bool = False **kwargs  )   Parameters   save_directory (str or os.PathLike) —
Directory where the feature extractor JSON file will be saved (will be created if it does not exist).   push_to_hub (bool, optional, defaults to False) —
Whether or not to push your model to the Hugging Face model hub after saving it. You can specify the
repository you want to push to with repo_id (will default to the name of save_directory in your
namespace).   kwargs (Dict[str, Any], optional) —
Additional key word arguments passed along to the push_to_hub() method.    Save a feature_extractor object to the directory save_directory, so that it can be re-loaded using the
from_pretrained() class met


SequenceFeatureExtractor

class transformers.SequenceFeatureExtractor  < source > ( feature_size: int sampling_rate: int padding_value: float **kwargs  )   Parameters   feature_size (int) —
The feature dimension of the extracted features.   sampling_rate (int) —
The sampling rate at which the audio files should be digitalized expressed in hertz (Hz).   padding_value (float) —
The value that is used to fill the padding values / vectors.    This is a general feature extraction class for speech recognition.  pad  < source > ( processed_features: typing.Union[transformers.feature_extraction_utils.BatchFeature, typing.List[transformers.feature_extraction_utils.BatchFeature], typing.Dict[str, transformers.feature_extraction_utils.BatchFeature], typing.Dict[str, typing.List[transformers.feature_extraction_utils.BatchFeature]], typing.List[typing.Dict[str, transformers.feature_extraction_utils.BatchFeature]]] padding: typing.Union[bool, str, transformers.utils.generic.PaddingStrategy] = True max_length: typing.O


SequenceFeatureExtractor

ptional[int] = None truncation: bool = False pad_to_multiple_of: typing.Optional[int] = None return_attention_mask: typing.Optional[bool] = None return_tensors: typing.Union[str, transformers.utils.generic.TensorType, NoneType] = None  )   Parameters   processed_features (BatchFeature, list of BatchFeature, Dict[str, List[float]], Dict[str, List[List[float]] or List[Dict[str, List[float]]]) —
Processed inputs. Can represent one input (BatchFeature or Dict[str, List[float]]) or a batch of
input values / vectors (list of BatchFeature, Dict[str, List[List[float]]] or List[Dict[str,
List[float]]]) so you can use this method during preprocessing as well as in a PyTorch Dataloader
collate function.
Instead of List[float] you can have tensors (numpy arrays, PyTorch tensors or TensorFlow tensors),
see the note above for the return type.   padding (bool, str or PaddingStrategy, optional, defaults to True) —
Select a strategy to pad the returned sequences (according to the model’s padding


SequenceFeatureExtractor

 side and padding
index) among:
True or 'longest': Pad to the longest sequence in the batch (or no padding if only a single
sequence if provided).
'max_length': Pad to a maximum length specified with the argument max_length or to the maximum
acceptable input length for the model if that argument is not provided.
False or 'do_not_pad' (default): No padding (i.e., can output a batch with sequences of different
lengths).
   max_length (int, optional) —
Maximum length of the returned list and optionally padding length (see above).   truncation (bool) —
Activates truncation to cut input sequences longer than max_length to max_length.   pad_to_multiple_of (int, optional) —
If set will pad the sequence to a multiple of the provided value.
This is especially useful to enable the use of Tensor Cores on NVIDIA hardware with compute capability
>= 7.5 (Volta), or on TPUs which benefit from having sequence lengths be a multiple of 128.   return_attention_mask (bool, optional) —
Whether to re


SequenceFeatureExtractor

turn the attention mask. If left to the default, will return the attention mask according
to the specific feature_extractor’s default.
What are attention masks?   return_tensors (str or TensorType, optional) —
If set, will return tensors instead of list of python integers. Acceptable values are:
'tf': Return TensorFlow tf.constant objects.
'pt': Return PyTorch torch.Tensor objects.
'np': Return Numpy np.ndarray objects.
    Pad input values / input vectors or a batch of input values / input vectors up to predefined length or to the
max sequence length in the batch. Padding side (left/right) padding values are defined at the feature extractor level (with self.padding_side,
self.padding_value) If the processed_features passed are dictionary of numpy arrays, PyTorch tensors or TensorFlow tensors, the
result will use the same type unless you provide a different tensor type with return_tensors. In the case of
PyTorch tensors, you will lose the specific device of your tensors however.


BatchFeature

class transformers.BatchFeature  < source > ( data: typing.Union[typing.Dict[str, typing.Any], NoneType] = None tensor_type: typing.Union[NoneType, str, transformers.utils.generic.TensorType] = None  )   Parameters   data (dict, optional) —
Dictionary of lists/arrays/tensors returned by the call/pad methods (‘input_values’, ‘attention_mask’,
etc.).   tensor_type (Union[None, str, TensorType], optional) —
You can give a tensor_type here to convert the lists of integers in PyTorch/TensorFlow/Numpy Tensors at
initialization.    Holds the output of the pad() and feature extractor specific __call__ methods. This class is derived from a python dictionary and can be used as a dictionary.  convert_to_tensors  < source > ( tensor_type: typing.Uni


BatchFeature

on[str, transformers.utils.generic.TensorType, NoneType] = None  )   Parameters   tensor_type (str or TensorType, optional) —
The type of tensors to use. If str, should be one of the values of the enum TensorType. If
None, no modification is done.    Convert the inner content to tensors.  to  < source > ( *args **kwargs  ) → BatchFeature  Parameters   args (Tuple) —
Will be passed to the to(...) function of the tensors.   kwargs (Dict, optional) —
Will be passed to the to(...) function of the tensors.   Returns
BatchFeature

The same instance after modification.
  Send all values to device by calling v.to(*args, **kwargs) (PyTorch only). This should support casting in
different dtypes and sending the BatchFeature to a different device


ImageFeatureExtractionMixin

class transformers.ImageFeatureExtractionMixin  < source > ( )    Mixin that contain utilities for preparing image features.  center_crop  < source > ( image size  ) → new_image  Parameters   image (PIL.Image.Image or np.ndarray or torch.Tensor of shape (n_channels, height, width) or (height, width, n_channels)) —
The image to resize.   size (int or Tuple[int, int]) —
The size to which crop the image.   Returns
new_image

A center cropped PIL.Image.Image or np.ndarray or torch.Tensor of shape: (n_channels,
height, width).
  Crops image to the given size using a center crop. Note that if the image is too small to be cropped to the
size given, it will be padded (so the returned result has the size asked).  convert_rgb  < source > ( image  )   Parameters   image (PIL.Image.Image) —
The image to convert.    Converts PIL.Image.Image to RGB format.  expand_dims  < source > ( image  )   Para


ImageFeatureExtractionMixin

meters   image (PIL.Image.Image or np.ndarray or torch.Tensor) —
The image to expand.    Expands 2-dimensional image to 3 dimensions.  flip_channel_order  < source > ( image  )   Parameters   image (PIL.Image.Image or np.ndarray or torch.Tensor) —
The image whose color channels to flip. If np.ndarray or torch.Tensor, the channel dimension should
be first.    Flips the channel order of image from RGB to BGR, or vice versa. Note that this will trigger a conversion of
image to a NumPy array if it’s a PIL Image.  normalize  < source > ( image mean std rescale = False  )   Parameters   image (PIL.Image.Image or np.ndarray or torch.Tensor) —
The image to normalize.   mean (List[float] or np.ndarray or torch.Tensor) —
The mean (per channel) to use for normalization.   std (List[float] or np.ndarray or torch.Tensor) —
The standard deviation (per channel) to use for normalization.   rescale (bool


ImageFeatureExtractionMixin

, optional, defaults to False) —
Whether or not to rescale the image to be between 0 and 1. If a PIL image is provided, scaling will
happen automatically.    Normalizes image with mean and std. Note that this will trigger a conversion of image to a NumPy array
if it’s a PIL Image.  rescale  < source > ( image: ndarray scale: typing.Union[float, int]  )    Rescale a numpy image by scale amount  resize  < source > ( image size resample = None default_to_square = True max_size = None  ) → image  Parameters   image (PIL.Image.Image or np.ndarray or torch.Tensor) —
The image to resize.   size (int or Tuple[int, int]) —
The size to use for resizing the image. If size is a sequence like (h, w), output size will be
matched to this.
If size is an int and default_to_square is True, then image will be resized to (size, size). If
size is an int and default_to_square is False, then smaller edge of th


ImageFeatureExtractionMixin

e image will be matched to
this number. i.e, if height > width, then image will be rescaled to (size * height / width, size).   resample (int, optional, defaults to PILImageResampling.BILINEAR) —
The filter to user for resampling.   default_to_square (bool, optional, defaults to True) —
How to convert size when it is a single int. If set to True, the size will be converted to a
square (size,size). If set to False, will replicate
torchvision.transforms.Resize
with support for resizing only the smallest edge and providing an optional max_size.   max_size (int, optional, defaults to None) —
The maximum allowed for the longer edge of the resized image: if the longer edge of the image is
greater than max_size after being resized according to size, then the image is resized again so
that the longer edge is equal to max_size. As a result, size might be overruled, i.e the smaller
edge may be sho


ImageFeatureExtractionMixin

rter than size. Only used if default_to_square is False.   Returns
image

A resized PIL.Image.Image.
  Resizes image. Enforces conversion of input to PIL.Image.  rotate  < source > ( image angle resample = None expand = 0 center = None translate = None fillcolor = None  ) → image  Parameters   image (PIL.Image.Image or np.ndarray or torch.Tensor) —
The image to rotate. If np.ndarray or torch.Tensor, will be converted to PIL.Image.Image before
rotating.   Returns
image

A rotated PIL.Image.Image.
  Returns a rotated copy of image. This method returns a copy of image, rotated the given number of degrees
counter clockwise around its centre.  to_numpy_array  < source > ( image rescale = None channel_first = True  )   Parameters   image (PIL.Image.Image or np.ndarray or torch.Tensor) —
The image to convert to a NumPy array.   rescale (bool, optional) —
Whether or not to apply the scalin


ImageFeatureExtractionMixin

g factor (to make pixel values floats between 0. and 1.). Will
default to True if the image is a PIL Image or an array/tensor of integers, False otherwise.   channel_first (bool, optional, defaults to True) —
Whether or not to permute the dimensions of the image to put the channel dimension first.    Converts image to a numpy array. Optionally rescales it and puts the channel dimension as the first
dimension.  to_pil_image  < source > ( image rescale = None  )   Parameters   image (PIL.Image.Image or numpy.ndarray or torch.Tensor) —
The image to convert to the PIL Image format.   rescale (bool, optional) —
Whether or not to apply the scaling factor (to make pixel values integers between 0 and 255). Will
default to True if the image type is a floating type, False otherwise.    Converts image to a PIL Image. Optionally rescales it and puts the channel dimension back as the last axis if
nee


Configuration

The base class PretrainedConfig implements the common methods for loading/saving a configuration
either from a local file or directory, or from a pretrained model configuration provided by the library (downloaded
from HuggingFace’s AWS S3 repository).
Each derived config class implements model specific attributes. Common attributes present in all config classes are:
hidden_size, num_attention_heads, and num_hidden_layers. Text models further implement:
vocab_size.


PretrainedConfig

class transformers.PretrainedConfig  < source > ( **kwargs  )   Parameters   name_or_path (str, optional, defaults to "") —
Store the string that was passed to PreTrainedModel.from_pretrained() or
TFPreTrainedModel.from_pretrained() as pretrained_model_name_or_path if the configuration was created
with such a method.   output_hidden_states (bool, optional, defaults to False) —
Whether or not the model should return all hidden-states.   output_attentions (bool, optional, defaults to False) —
Whether or not the model should returns all attentions.   return_dict (bool, optional, defaults to True) —
Whether or not the model should return a ModelOutput instead of a plain tuple.   is_encoder_decoder (bool, optional, defaults to False) —
Whether the model is used as an encoder/decoder or not.   is_decoder (bool, optional, defaults to False) —
Whether the model is used as decoder or not (in which case it’s used as an encoder).   cross_attention_hidden_size** (bool, option


PretrainedConfig

al) —
The hidden size of the cross-attention layer in case the model is used as a decoder in an encoder-decoder
setting and the cross-attention hidden dimension differs from self.config.hidden_size.   add_cross_attention (bool, optional, defaults to False) —
Whether cross-attention layers should be added to the model. Note, this option is only relevant for models
that can be used as decoder models within the EncoderDecoderModel class, which consists of all models
in AUTO_MODELS_FOR_CAUSAL_LM.   tie_encoder_decoder (bool, optional, defaults to False) —
Whether all encoder weights should be tied to their equivalent decoder weights. This requires the encoder
and decoder model to have the exact same parameter names.   prune_heads (Dict[int, List[int]], optional, defaults to {}) —
Pruned heads of the model. The keys are the selected layer indices and the associated values, the list of
heads to prune in said layer.
For instance {1: [0, 2], 2: [2, 3]} will prune heads 0


PretrainedConfig

and 2 on layer 1 and heads 2 and 3 on layer 2.   chunk_size_feed_forward (int, optional, defaults to 0) —
The chunk size of all feed forward layers in the residual attention blocks. A chunk size of 0 means that
the feed forward layer is not chunked. A chunk size of n means that the feed forward layer processes n <
sequence_length embeddings at a time. For more information on feed forward chunking, see How does Feed
Forward Chunking work?.   Parameters for sequence generation   max_length (int, optional, defaults to 20) —
Maximum length that will be used by default in the generate method of the model.   min_length (int, optional, defaults to 0) —
Minimum length that will be used by default in the generate method of the model.   do_sample (bool, optional, defaults to False) —
Flag that will be used by default in the generate method of the model. Whether or not to use sampling ;
use greedy decoding otherwise.   early_stopping (bool, optional, defaults to False) —
Fla


PretrainedConfig

g that will be used by default in the generate method of the model. Whether to stop the beam search
when at least num_beams sentences are finished per batch or not.   num_beams (int, optional, defaults to 1) —
Number of beams for beam search that will be used by default in the generate method of the model. 1 means
no beam search.   num_beam_groups (int, optional, defaults to 1) —
Number of groups to divide num_beams into in order to ensure diversity among different groups of beams
that will be used by default in the generate method of the model. 1 means no group beam search.   diversity_penalty (float, optional, defaults to 0.0) —
Value to control diversity for group beam search. that will be used by default in the generate method of
the model. 0 means no diversity penalty. The higher the penalty, the more diverse are the outputs.   temperature (float, optional, defaults to 1.0) —
The value used to module the next token probabilities that will be used by default i


PretrainedConfig

n the generate method
of the model. Must be strictly positive.   top_k (int, optional, defaults to 50) —
Number of highest probability vocabulary tokens to keep for top-k-filtering that will be used by default in
the generate method of the model.   top_p (float, optional, defaults to 1) —
Value that will be used by default in the generate method of the model for top_p. If set to float < 1,
only the most probable tokens with probabilities that add up to top_p or higher are kept for generation.   typical_p (float, optional, defaults to 1) —
Local typicality measures how similar the conditional probability of predicting a target token next is to
the expected conditional probability of predicting a random token next, given the partial text already
generated. If set to float < 1, the smallest set of the most locally typical tokens with probabilities that
add up to typical_p or higher are kept for generation. See this
paper for more details.   repetition_penalty (float,


PretrainedConfig

 optional, defaults to 1) —
Parameter for repetition penalty that will be used by default in the generate method of the model. 1.0
means no penalty.   length_penalty (float, optional, defaults to 1) —
Exponential penalty to the length that is used with beam-based generation. It is applied as an exponent to
the sequence length, which in turn is used to divide the score of the sequence. Since the score is the log
likelihood of the sequence (i.e. negative), length_penalty > 0.0 promotes longer sequences, while
length_penalty < 0.0 encourages shorter sequences.   no_repeat_ngram_size (int, optional, defaults to 0) — Value that will be used by default in the —
generate method of the model for no_repeat_ngram_size. If set to int > 0, all ngrams of that size can
only occur once.   encoder_no_repeat_ngram_size (int, optional, defaults to 0) — Value that will be used by —
default in the generate method of the model for encoder_no_repeat_ngram_size. If set to int > 0, all
n


PretrainedConfig

grams of that size that occur in the encoder_input_ids cannot occur in the decoder_input_ids.   bad_words_ids (List[int], optional) —
List of token ids that are not allowed to be generated that will be used by default in the generate
method of the model. In order to get the tokens of the words that should not appear in the generated text,
use tokenizer.encode(bad_word, add_prefix_space=True).   num_return_sequences (int, optional, defaults to 1) —
Number of independently computed returned sequences for each element in the batch that will be used by
default in the generate method of the model.   output_scores (bool, optional, defaults to False) —
Whether the model should return the logits when used for generation.   return_dict_in_generate (bool, optional, defaults to False) —
Whether the model should return a ModelOutput instead of a torch.LongTensor.   forced_bos_token_id (int, optional) —
The id of the token to force as the first generated token after the decode


PretrainedConfig

r_start_token_id. Useful for
multilingual models like mBART where the first generated token needs to be the target
language token.   forced_eos_token_id (int, optional) —
The id of the token to force as the last generated token when max_length is reached.   remove_invalid_values (bool, optional) —
Whether to remove possible nan and inf outputs of the model to prevent the generation method to crash.
Note that using remove_invalid_values can slow down generation.   Parameters for fine-tuning tasks   architectures (List[str], optional) —
Model architectures that can be used with the model pretrained weights.   finetuning_task (str, optional) —
Name of the task used to fine-tune the model. This can be used when converting from an original (TensorFlow
or PyTorch) checkpoint.   id2label (Dict[int, str], optional) —
A map from index (for instance prediction index, or target index) to label.   label2id (Dict[str, int], optional) — A map from label to index for the model.


PretrainedConfig

  num_labels (int, optional) —
Number of labels to use in the last layer added to the model, typically for a classification task.   task_specific_params (Dict[str, Any], optional) —
Additional keyword arguments to store for the current task.   problem_type (str, optional) —
Problem type for XxxForSequenceClassification models. Can be one of "regression",
"single_label_classification" or "multi_label_classification".   Parameters linked to the tokenizer   tokenizer_class (str, optional) —
The name of the associated tokenizer class to use (if none is set, will use the tokenizer associated to the
model by default).   prefix (str, optional) —
A specific prompt that should be added at the beginning of each text before calling the model.   bos_token_id (int, optional) — The id of the beginning-of-stream token.   pad_token_id (int, optional) — The id of the padding token.   eos_token_id (int, optional) — The id of the end-of-stream token.   decoder_start_token_id (int, o


PretrainedConfig

ptional) —
If an encoder-decoder model starts decoding with a different token than bos, the id of that token.   sep_token_id (int, optional) — The id of the separation token.   PyTorch specific parameters   torchscript (bool, optional, defaults to False) —
Whether or not the model should be used with Torchscript.   tie_word_embeddings (bool, optional, defaults to True) —
Whether the model’s input and output word embeddings should be tied. Note that this is only relevant if the
model has a output word embedding layer.   torch_dtype (str, optional) —
The dtype of the weights. This attribute can be used to initialize the model to a non-default dtype
(which is normally float32) and thus allow for optimal storage allocation. For example, if the saved
model is float16, ideally we want to load it back using the minimal amount of memory needed to load
float16 weights. Since the config object is stored in plain text, this attribute contains just the
floating type string wi


PretrainedConfig

thout the torch. prefix. For example, for torch.float16 `torch_dtype is the
"float16" string.
This attribute is currently not being used during model loading time, but this may change in the future
versions. But we can already start preparing for the future by saving the dtype with save_pretrained.   TensorFlow specific parameters   use_bfloat16 (bool, optional, defaults to False) —
Whether or not the model should use BFloat16 scalars (only used by some TensorFlow models).   tf_legacy_loss (bool, optional, defaults to False) —
Whether the model should use legacy TensorFlow losses. Legacy losses have variable output shapes and may
not be XLA-compatible. This option is here for backward compatibility and will be removed in Transformers
v5.     Base class for all configuration classes. Handles a few parameters common to all models’ configurations as well as
methods for loading/downloading/saving configurations. A configuration file can be loaded and saved to disk. Lo


PretrainedConfig

ading the configuration file and using this file to
initialize a model does not load the model weights. It only affects the model’s configuration. Class attributes (overridden by derived classes): model_type (str) — An identifier for the model type, serialized into the JSON file, and used to recreate
the correct object in AutoConfig. is_composition (bool) — Whether the config class is composed of multiple sub-configs. In this case the
config has to be initialized from two or more configs of type PretrainedConfig like:
EncoderDecoderConfig or ~RagConfig. keys_to_ignore_at_inference (List[str]) — A list of keys to ignore by default when looking at dictionary
outputs of the model during inference. attribute_map (Dict[str, str]) — A dict that maps model specific attribute names to the standardized
naming of attributes. Common attributes (present in all subclasses): vocab_size (int) — The number of tokens in the vocabulary, which is also the first dimension of the
embe


PretrainedConfig

ddings matrix (this attribute may be missing for models that don’t have a text modality like ViT). hidden_size (int) — The hidden size of the model. num_attention_heads (int) — The number of attention heads used in the multi-head attention layers of the
model. num_hidden_layers (int) — The number of blocks in the model.  push_to_hub  < source > ( repo_id: str use_temp_dir: typing.Optional[bool] = None commit_message: typing.Optional[str] = None private: typing.Optional[bool] = None token: typing.Union[bool, str, NoneType] = None max_shard_size: typing.Union[int, str, NoneType] = '10GB' create_pr: bool = False safe_serialization: bool = False revision: str = None **deprecated_kwargs  )   Parameters   repo_id (str) —
The name of the repository you want to push your config to. It should contain your organization name
when pushing to a given organization.   use_temp_dir (bool, optional) —
Whether or not to use a temporary directory to store the files saved before they


PretrainedConfig

 are pushed to the Hub.
Will default to True if there is no directory named like repo_id, False otherwise.   commit_message (str, optional) —
Message to commit while pushing. Will default to "Upload config".   private (bool, optional) —
Whether or not the repository created should be private.   token (bool or str, optional) —
The token to use as HTTP bearer authorization for remote files. If True, will use the token generated
when running huggingface-cli login (stored in ~/.huggingface). Will default to True if repo_url
is not specified.   max_shard_size (int or str, optional, defaults to "10GB") —
Only applicable for models. The maximum size for a checkpoint before being sharded. Checkpoints shard
will then be each of size lower than this size. If expressed as a string, needs to be digits followed
by a unit (like "5MB").   create_pr (bool, optional, defaults to False) —
Whether or not to create a PR with the uploaded files or directly commit.   safe_serialization


PretrainedConfig

 (bool, optional, defaults to False) —
Whether or not to convert the model weights in safetensors format for safer serialization.   revision (str, optional) —
Branch to push the uploaded files to.    Upload the configuration file to the 🤗 Model Hub.  Examples:   Copied from transformers import AutoConfig
config = AutoConfig.from_pretrained("bert-base-cased")
# Push the config to your namespace with the name "my-finetuned-bert".
config.push_to_hub("my-finetuned-bert")
# Push the config to an organization with the name "my-finetuned-bert".
config.push_to_hub("huggingface/my-finetuned-bert")  dict_torch_dtype_to_str  < source > ( d: typing.Dict[str, typing.Any]  )    Checks whether the passed dictionary and its nested dicts have a torch_dtype key and if it’s not None,
converts torch.dtype to a string of just the type. For example, torch.float32 get converted into “float32”
string, which can then be stored in the json format.  from_dict  < source > ( config_dict: typi


PretrainedConfig

ng.Dict[str, typing.Any] **kwargs  ) → PretrainedConfig  Parameters   config_dict (Dict[str, Any]) —
Dictionary that will be used to instantiate the configuration object. Such a dictionary can be
retrieved from a pretrained checkpoint by leveraging the get_config_dict() method.   kwargs (Dict[str, Any]) —
Additional parameters from which to initialize the configuration object.   Returns
PretrainedConfig

The configuration object instantiated from those parameters.
  Instantiates a PretrainedConfig from a Python dictionary of parameters.  from_json_file  < source > ( json_file: typing.Union[str, os.PathLike]  ) → PretrainedConfig  Parameters   json_file (str or os.PathLike) —
Path to the JSON file containing the parameters.   Returns
PretrainedConfig

The configuration object instantiated from that JSON file.
  Instantiates a PretrainedConfig from the path to a JSON file of parameters.  from_pretrained  < source > ( pretrained_model_name_or_path: typing.Union


PretrainedConfig

[str, os.PathLike] cache_dir: typing.Union[str, os.PathLike, NoneType] = None force_download: bool = False local_files_only: bool = False token: typing.Union[bool, str, NoneType] = None revision: str = 'main' **kwargs  ) → PretrainedConfig  Parameters   pretrained_model_name_or_path (str or os.PathLike) —
This can be either:
a string, the model id of a pretrained model configuration hosted inside a model repo on
huggingface.co. Valid model ids can be located at the root-level, like bert-base-uncased, or
namespaced under a user or organization name, like dbmdz/bert-base-german-cased.
a path to a directory containing a configuration file saved using the
save_pretrained() method, e.g., ./my_model_directory/.
a path or url to a saved configuration JSON file, e.g., ./my_model_directory/configuration.json.
   cache_dir (str or os.PathLike, optional) —
Path to a directory in which a downloaded pretrained model configuration should be cached if the
standard cache should n