PreTrainedTokenizer

tional) —
If set, will return tensors instead of list of python integers. Acceptable values are:
'tf': Return TensorFlow tf.constant objects.
'pt': Return PyTorch torch.Tensor objects.
'np': Return Numpy np.ndarray objects.
**kwargs — Passed along to the .tokenize() method.   Returns
List[int], torch.Tensor, tf.Tensor or np.ndarray

The tokenized ids of the text.
  Converts a string to a sequence of ids (integer), using the tokenizer and vocabulary. Same as doing self.convert_tokens_to_ids(self.tokenize(text)).  push_to_hub  < source > ( repo_id: str use_temp_dir: typing.Optional[bool] = None commit_message: typing.Optional[str] = None private: typing.Optional[bool] = None token: typing.Union[bool, str, NoneType] = None max_shard_size: typing.Union[int, str, NoneType] = '10GB' create_pr: bool = False safe_serialization: bool = False revision: str = None **deprecated_kwargs  )   Parameters   repo_id (str) —
The name of the repository you want to push your tokenizer to. It sh


PreTrainedTokenizer

ould contain your organization name
when pushing to a given organization.   use_temp_dir (bool, optional) —
Whether or not to use a temporary directory to store the files saved before they are pushed to the Hub.
Will default to True if there is no directory named like repo_id, False otherwise.   commit_message (str, optional) —
Message to commit while pushing. Will default to "Upload tokenizer".   private (bool, optional) —
Whether or not the repository created should be private.   token (bool or str, optional) —
The token to use as HTTP bearer authorization for remote files. If True, will use the token generated
when running huggingface-cli login (stored in ~/.huggingface). Will default to True if repo_url
is not specified.   max_shard_size (int or str, optional, defaults to "10GB") —
Only applicable for models. The maximum size for a checkpoint before being sharded. Checkpoints shard
will then be each of size lower than this size. If expressed as a string, needs to be digits


PreTrainedTokenizer

 followed
by a unit (like "5MB").   create_pr (bool, optional, defaults to False) —
Whether or not to create a PR with the uploaded files or directly commit.   safe_serialization (bool, optional, defaults to False) —
Whether or not to convert the model weights in safetensors format for safer serialization.   revision (str, optional) —
Branch to push the uploaded files to.    Upload the tokenizer files to the 🤗 Model Hub.  Examples:   Copied from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("bert-base-cased")
# Push the tokenizer to your namespace with the name "my-finetuned-bert".
tokenizer.push_to_hub("my-finetuned-bert")
# Push the tokenizer to an organization with the name "my-finetuned-bert".
tokenizer.push_to_hub("huggingface/my-finetuned-bert")  convert_ids_to_tokens  < source > ( ids: typing.Union[int, typing.List[int]] skip_special_tokens: bool = False  ) → str or List[str]  Parameters   ids (int or List[int]) —
The token id (or token ids


PreTrainedTokenizer

) to convert to tokens.   skip_special_tokens (bool, optional, defaults to False) —
Whether or not to remove special tokens in the decoding.   Returns
str or List[str]

The decoded token(s).
  Converts a single index or a sequence of indices in a token or a sequence of tokens, using the vocabulary and
added tokens.  convert_tokens_to_ids  < source > ( tokens: typing.Union[str, typing.List[str]]  ) → int or List[int]  Parameters   tokens (str or List[str]) — One or several token(s) to convert to token id(s).   Returns
int or List[int]

The token id or list of token ids.
  Converts a token string (or a sequence of tokens) in a single integer id (or a sequence of ids), using the
vocabulary.  get_added_vocab  < source > ( ) → Dict[str, int]  Returns
Dict[str, int]

The added tokens.
  Returns the added tokens in the vocabulary as a dictionary of token to index. Results might be different from
the fast call because for now we always add the tokens even if they are already


PreTrainedTokenizer

in the vocabulary. This is
something we should change.  num_special_tokens_to_add  < source > ( pair: bool = False  ) → int  Parameters   pair (bool, optional, defaults to False) —
Whether the number of added tokens should be computed in the case of a sequence pair or a single
sequence.   Returns
int

Number of special tokens added to sequences.
  Returns the number of added tokens when encoding a sequence with special tokens. This encodes a dummy input and checks the number of added tokens, and is therefore not efficient. Do not put
this inside your training loop.  prepare_for_tokenization  < source > ( text: str is_split_into_words: bool = False **kwargs  ) → Tuple[str, Dict[str, Any]]  Parameters   text (str) —
The text to prepare.   is_split_into_words (bool, optional, defaults to False) —
Whether or not the input is already pre-tokenized (e.g., split into words). If set to True, the
tokenizer assumes the input is already split into words (for instance, by splitting it


PreTrainedTokenizer

on whitespace)
which it will tokenize. This is useful for NER or token classification.   kwargs (Dict[str, Any], optional) —
Keyword arguments to use for the tokenization.   Returns
Tuple[str, Dict[str, Any]]

The prepared text and the unused kwargs.
  Performs any necessary transformations before tokenization. This method should pop the arguments from kwargs and return the remaining kwargs as well. We test the
kwargs at the end of the encoding process to be sure all the arguments have been used.  tokenize  < source > ( text: str **kwargs  ) → List[str]  Parameters   text (str) —
The sequence to be encoded.   **kwargs (additional keyword arguments) —
Passed along to the model-specific prepare_for_tokenization preprocessing method.   Returns
List[str]

The list of tokens.
  Converts a string in a sequence of tokens, using the tokenizer. Split in words for word-based vocabulary or sub-words for sub-word-based vocabularies
(BPE/SentencePieces/WordPieces). Takes care of adde


PreTrainedTokenizerFast

The PreTrainedTokenizerFast depend on the tokenizers library. The tokenizers obtained from the 🤗 tokenizers library can be
loaded very simply into 🤗 transformers. Take a look at the Using tokenizers from 🤗 tokenizers page to understand how this is done.
 class transformers.PreTrainedTokenizerFast  < source > ( *args **kwargs  )   Parameters   model_max_length (int, optional) —
The maximum length (in number of tokens) for the inputs to the transformer model. When the tokenizer is
loaded with from_pretrained(), this will be set to the
value stored for the associated model in max_model_input_sizes (see above). If no value is provided, will
default to VERY_LARGE_INTEGER (int(1e30)).   padding_side (str, optional) —
The side on which the model should have padding applied. Should be selected between [‘right’, ‘left’].
Default value is picked from the class attribute of the same name.   truncation_side (str, optional) —
The side on which the model should have truncat


PreTrainedTokenizerFast

ion applied. Should be selected between [‘right’, ‘left’].
Default value is picked from the class attribute of the same name.   chat_template (str, optional) —
A Jinja template string that will be used to format lists of chat messages. See
https://huggingface.co/docs/transformers/chat_templating for a full description.   model_input_names (List[string], optional) —
The list of inputs accepted by the forward pass of the model (like "token_type_ids" or
"attention_mask"). Default value is picked from the class attribute of the same name.   bos_token (str or tokenizers.AddedToken, optional) —
A special token representing the beginning of a sentence. Will be associated to self.bos_token and
self.bos_token_id.   eos_token (str or tokenizers.AddedToken, optional) —
A special token representing the end of a sentence. Will be associated to self.eos_token and
self.eos_token_id.   unk_token (str or tokenizers.AddedToken, optional) —
A special token representing an out-of


PreTrainedTokenizerFast

-vocabulary token. Will be associated to self.unk_token and
self.unk_token_id.   sep_token (str or tokenizers.AddedToken, optional) —
A special token separating two different sentences in the same input (used by BERT for instance). Will be
associated to self.sep_token and self.sep_token_id.   pad_token (str or tokenizers.AddedToken, optional) —
A special token used to make arrays of tokens the same size for batching purpose. Will then be ignored by
attention mechanisms or loss computation. Will be associated to self.pad_token and self.pad_token_id.   cls_token (str or tokenizers.AddedToken, optional) —
A special token representing the class of the input (used by BERT for instance). Will be associated to
self.cls_token and self.cls_token_id.   mask_token (str or tokenizers.AddedToken, optional) —
A special token representing a masked token (used by masked-language modeling pretraining objectives, like
BERT). Will be associated to self.mask_token and self.mask_t


PreTrainedTokenizerFast

oken_id.   additional_special_tokens (tuple or list of str or tokenizers.AddedToken, optional) —
A tuple or a list of additional special tokens. Add them here to ensure they are skipped when decoding with
skip_special_tokens is set to True. If they are not part of the vocabulary, they will be added at the end
of the vocabulary.   clean_up_tokenization_spaces (bool, optional, defaults to True) —
Whether or not the model should cleanup the spaces that were added when splitting the input text during the
tokenization process.   split_special_tokens (bool, optional, defaults to False) —
Whether or not the special tokens should be split during the tokenization process. The default behavior is
to not split special tokens. This means that if <s> is the bos_token, then tokenizer.tokenize("<s>") = ['<s>]. Otherwise, if split_special_tokens=True, then tokenizer.tokenize("<s>") will be give ['<', 's', '>']. This argument is only supported for slow tokenizers for the momen


PreTrainedTokenizerFast

t.   tokenizer_object (tokenizers.Tokenizer) —
A tokenizers.Tokenizer object from 🤗 tokenizers to instantiate from. See Using tokenizers from 🤗
tokenizers for more information.   tokenizer_file (str) —
A path to a local JSON file representing a previously serialized tokenizers.Tokenizer object from 🤗
tokenizers.    Base class for all fast tokenizers (wrapping HuggingFace tokenizers library). Inherits from PreTrainedTokenizerBase. Handles all the shared methods for tokenization and special tokens, as well as methods for
downloading/caching/loading pretrained tokenizers, as well as adding tokens to the vocabulary. This class also contains the added tokens in a unified way on top of all tokenizers so we don’t have to handle the
specific vocabulary augmentation methods of the various underlying dictionary structures (BPE, sentencepiece…). Class attributes (overridden by derived classes) vocab_files_names (Dict[str, str]) — A dictionary with, as keys, the __init__


PreTrainedTokenizerFast

keyword name of each
vocabulary file required by the model, and as associated values, the filename for saving the associated file
(string). pretrained_vocab_files_map (Dict[str, Dict[str, str]]) — A dictionary of dictionaries, with the
high-level keys being the __init__ keyword name of each vocabulary file required by the model, the
low-level being the short-cut-names of the pretrained models with, as associated values, the url to the
associated pretrained vocabulary file. max_model_input_sizes (Dict[str, Optional[int]]) — A dictionary with, as keys, the short-cut-names
of the pretrained models, and as associated values, the maximum length of the sequence inputs of this model,
or None if the model has no maximum input size. pretrained_init_configuration (Dict[str, Dict[str, Any]]) — A dictionary with, as keys, the
short-cut-names of the pretrained models, and as associated values, a dictionary of specific arguments to
pass to the __init__ method of the tokeniz


PreTrainedTokenizerFast

er class for this pretrained model when loading the tokenizer
with the from_pretrained() method. model_input_names (List[str]) — A list of inputs expected in the forward pass of the model. padding_side (str) — The default value for the side on which the model should have padding applied.
Should be 'right' or 'left'. truncation_side (str) — The default value for the side on which the model should have truncation
applied. Should be 'right' or 'left'.  __call__  < source > ( text: typing.Union[str, typing.List[str], typing.List[typing.List[str]]] = None text_pair: typing.Union[str, typing.List[str], typing.List[typing.List[str]], NoneType] = None text_target: typing.Union[str, typing.List[str], typing.List[typing.List[str]]] = None text_pair_target: typing.Union[str, typing.List[str], typing.List[typing.List[str]], NoneType] = None add_special_tokens: bool = True padding: typing.Union[bool, str, transformers.utils.generic.PaddingStrategy] = False truncation: typi


PreTrainedTokenizerFast

ng.Union[bool, str, transformers.tokenization_utils_base.TruncationStrategy] = None max_length: typing.Optional[int] = None stride: int = 0 is_split_into_words: bool = False pad_to_multiple_of: typing.Optional[int] = None return_tensors: typing.Union[str, transformers.utils.generic.TensorType, NoneType] = None return_token_type_ids: typing.Optional[bool] = None return_attention_mask: typing.Optional[bool] = None return_overflowing_tokens: bool = False return_special_tokens_mask: bool = False return_offsets_mapping: bool = False return_length: bool = False verbose: bool = True **kwargs  ) → BatchEncoding  Parameters   text (str, List[str], List[List[str]], optional) —
The sequence or batch of sequences to be encoded. Each sequence can be a string or a list of strings
(pretokenized string). If the sequences are provided as list of strings (pretokenized), you must set
is_split_into_words=True (to lift the ambiguity with a batch of sequences).   text_pair (str, Li


PreTrainedTokenizerFast

st[str], List[List[str]], optional) —
The sequence or batch of sequences to be encoded. Each sequence can be a string or a list of strings
(pretokenized string). If the sequences are provided as list of strings (pretokenized), you must set
is_split_into_words=True (to lift the ambiguity with a batch of sequences).   text_target (str, List[str], List[List[str]], optional) —
The sequence or batch of sequences to be encoded as target texts. Each sequence can be a string or a
list of strings (pretokenized string). If the sequences are provided as list of strings (pretokenized),
you must set is_split_into_words=True (to lift the ambiguity with a batch of sequences).   text_pair_target (str, List[str], List[List[str]], optional) —
The sequence or batch of sequences to be encoded as target texts. Each sequence can be a string or a
list of strings (pretokenized string). If the sequences are provided as list of strings (pretokenized),
you must set is_split_into_words=T


PreTrainedTokenizerFast

rue (to lift the ambiguity with a batch of sequences).   add_special_tokens (bool, optional, defaults to True) —
Whether or not to add special tokens when encoding the sequences. This will use the underlying
PretrainedTokenizerBase.build_inputs_with_special_tokens function, which defines which tokens are
automatically added to the input ids. This is usefull if you want to add bos or eos tokens
automatically.   padding (bool, str or PaddingStrategy, optional, defaults to False) —
Activates and controls padding. Accepts the following values:
True or 'longest': Pad to the longest sequence in the batch (or no padding if only a single
sequence if provided).
'max_length': Pad to a maximum length specified with the argument max_length or to the maximum
acceptable input length for the model if that argument is not provided.
False or 'do_not_pad' (default): No padding (i.e., can output a batch with sequences of different
lengths).
   truncation (bool, str or Truncation


PreTrainedTokenizerFast

Strategy, optional, defaults to False) —
Activates and controls truncation. Accepts the following values:
True or 'longest_first': Truncate to a maximum length specified with the argument max_length or
to the maximum acceptable input length for the model if that argument is not provided. This will
truncate token by token, removing a token from the longest sequence in the pair if a pair of
sequences (or a batch of pairs) is provided.
'only_first': Truncate to a maximum length specified with the argument max_length or to the
maximum acceptable input length for the model if that argument is not provided. This will only
truncate the first sequence of a pair if a pair of sequences (or a batch of pairs) is provided.
'only_second': Truncate to a maximum length specified with the argument max_length or to the
maximum acceptable input length for the model if that argument is not provided. This will only
truncate the second sequence of a pair if a pair of sequences (or


PreTrainedTokenizerFast

a batch of pairs) is provided.
False or 'do_not_truncate' (default): No truncation (i.e., can output batch with sequence lengths
greater than the model maximum admissible input size).
   max_length (int, optional) —
Controls the maximum length to use by one of the truncation/padding parameters.
If left unset or set to None, this will use the predefined model maximum length if a maximum length
is required by one of the truncation/padding parameters. If the model has no specific maximum input
length (like XLNet) truncation/padding to a maximum length will be deactivated.   stride (int, optional, defaults to 0) —
If set to a number along with max_length, the overflowing tokens returned when
return_overflowing_tokens=True will contain some tokens from the end of the truncated sequence
returned to provide some overlap between truncated and overflowing sequences. The value of this
argument defines the number of overlapping tokens.   is_split_into_words (bool, option


PreTrainedTokenizerFast

al, defaults to False) —
Whether or not the input is already pre-tokenized (e.g., split into words). If set to True, the
tokenizer assumes the input is already split into words (for instance, by splitting it on whitespace)
which it will tokenize. This is useful for NER or token classification.   pad_to_multiple_of (int, optional) —
If set will pad the sequence to a multiple of the provided value. Requires padding to be activated.
This is especially useful to enable the use of Tensor Cores on NVIDIA hardware with compute capability
>= 7.5 (Volta).   return_tensors (str or TensorType, optional) —
If set, will return tensors instead of list of python integers. Acceptable values are:
'tf': Return TensorFlow tf.constant objects.
'pt': Return PyTorch torch.Tensor objects.
'np': Return Numpy np.ndarray objects.
   return_token_type_ids (bool, optional) —
Whether to return token type IDs. If left to the default, will return the token type IDs according to
the specific


PreTrainedTokenizerFast

 tokenizer’s default, defined by the return_outputs attribute.
What are token type IDs?   return_attention_mask (bool, optional) —
Whether to return the attention mask. If left to the default, will return the attention mask according
to the specific tokenizer’s default, defined by the return_outputs attribute.
What are attention masks?   return_overflowing_tokens (bool, optional, defaults to False) —
Whether or not to return overflowing token sequences. If a pair of sequences of input ids (or a batch
of pairs) is provided with truncation_strategy = longest_first or True, an error is raised instead
of returning overflowing tokens.   return_special_tokens_mask (bool, optional, defaults to False) —
Whether or not to return special tokens mask information.   return_offsets_mapping (bool, optional, defaults to False) —
Whether or not to return (char_start, char_end) for each token.
This is only available on fast tokenizers inheriting from PreTrainedTokenizerFast, i


PreTrainedTokenizerFast

f using
Python’s tokenizer, this method will raise NotImplementedError.   return_length  (bool, optional, defaults to False) —
Whether or not to return the lengths of the encoded inputs.   verbose (bool, optional, defaults to True) —
Whether or not to print more information and warnings.
**kwargs — passed to the self.tokenize() method   Returns
BatchEncoding

A BatchEncoding with the following fields:
input_ids — List of token ids to be fed to a model.
What are input IDs?
token_type_ids — List of token type ids to be fed to a model (when return_token_type_ids=True or
if “token_type_ids” is in self.model_input_names).
What are token type IDs?
attention_mask — List of indices specifying which tokens should be attended to by the model (when
return_attention_mask=True or if “attention_mask” is in self.model_input_names).
What are attention masks?
overflowing_tokens — List of overflowing tokens sequences (when a max_length is specified and
return_overflowing_tok


PreTrainedTokenizerFast

ens=True).
num_truncated_tokens — Number of tokens truncated (when a max_length is specified and
return_overflowing_tokens=True).
special_tokens_mask — List of 0s and 1s, with 1 specifying added special tokens and 0 specifying
regular sequence tokens (when add_special_tokens=True and return_special_tokens_mask=True).
length — The length of the inputs (when return_length=True)
  Main method to tokenize and prepare for the model one or several sequence(s) or one or several pair(s) of
sequences.  apply_chat_template  < source > ( conversation: typing.Union[typing.List[typing.Dict[str, str]], ForwardRef('Conversation')] chat_template: typing.Optional[str] = None add_generation_prompt: bool = False tokenize: bool = True padding: bool = False truncation: bool = False max_length: typing.Optional[int] = None return_tensors: typing.Union[str, transformers.utils.generic.TensorType, NoneType] = None **tokenizer_kwargs  ) → List[int]  Parameters   conversation (Union[List


PreTrainedTokenizerFast

[Dict[str, str]], “Conversation”]) — A Conversation object or list of dicts
with “role” and “content” keys, representing the chat history so far.   chat_template (str, optional) — A Jinja template to use for this conversion. If
this is not passed, the model’s default chat template will be used instead.   add_generation_prompt (bool, optional) — Whether to end the prompt with the token(s) that indicate
the start of an assistant message. This is useful when you want to generate a response from the model.
Note that this argument will be passed to the chat template, and so it must be supported in the
template for this argument to have any effect.   tokenize (bool, defaults to True) —
Whether to tokenize the output. If False, the output will be a string.   padding (bool, defaults to False) —
Whether to pad sequences to the maximum length. Has no effect if tokenize is False.   truncation (bool, defaults to False) —
Whether to truncate sequences at the maximum length


PreTrainedTokenizerFast

. Has no effect if tokenize is False.   max_length (int, optional) —
Maximum length (in tokens) to use for padding or truncation. Has no effect if tokenize is False. If
not specified, the tokenizer’s max_length attribute will be used as a default.   return_tensors (str or TensorType, optional) —
If set, will return tensors of a particular framework. Has no effect if tokenize is False. Acceptable
values are:
'tf': Return TensorFlow tf.Tensor objects.
'pt': Return PyTorch torch.Tensor objects.
'np': Return NumPy np.ndarray objects.
'jax': Return JAX jnp.ndarray objects.
**tokenizer_kwargs — Additional kwargs to pass to the tokenizer.
   Returns
List[int]

A list of token ids representing the tokenized chat so far, including control tokens. This
output is ready to pass to the model, either directly or via methods like generate().
  Converts a Conversation object or a list of dictionaries with "role" and "content" keys to a list of token
ids. This method is int


PreTrainedTokenizerFast

ended for use with chat models, and will read the tokenizer’s chat_template attribute to
determine the format and control tokens to use when converting. When chat_template is None, it will fall back
to the default_chat_template specified at the class level.  batch_decode  < source > ( sequences: typing.Union[typing.List[int], typing.List[typing.List[int]], ForwardRef('np.ndarray'), ForwardRef('torch.Tensor'), ForwardRef('tf.Tensor')] skip_special_tokens: bool = False clean_up_tokenization_spaces: bool = None **kwargs  ) → List[str]  Parameters   sequences (Union[List[int], List[List[int]], np.ndarray, torch.Tensor, tf.Tensor]) —
List of tokenized input ids. Can be obtained using the __call__ method.   skip_special_tokens (bool, optional, defaults to False) —
Whether or not to remove special tokens in the decoding.   clean_up_tokenization_spaces (bool, optional) —
Whether or not to clean up the tokenization spaces. If None, will default to
self.clean_up_tokeniz


PreTrainedTokenizerFast

ation_spaces.   kwargs (additional keyword arguments, optional) —
Will be passed to the underlying model specific decode method.   Returns
List[str]

The list of decoded sentences.
  Convert a list of lists of token ids into a list of strings by calling decode.  decode  < source > ( token_ids: typing.Union[int, typing.List[int], ForwardRef('np.ndarray'), ForwardRef('torch.Tensor'), ForwardRef('tf.Tensor')] skip_special_tokens: bool = False clean_up_tokenization_spaces: bool = None **kwargs  ) → str  Parameters   token_ids (Union[int, List[int], np.ndarray, torch.Tensor, tf.Tensor]) —
List of tokenized input ids. Can be obtained using the __call__ method.   skip_special_tokens (bool, optional, defaults to False) —
Whether or not to remove special tokens in the decoding.   clean_up_tokenization_spaces (bool, optional) —
Whether or not to clean up the tokenization spaces. If None, will default to
self.clean_up_tokenization_spaces.   kwargs (additional keyword


PreTrainedTokenizerFast

arguments, optional) —
Will be passed to the underlying model specific decode method.   Returns
str

The decoded sentence.
  Converts a sequence of ids in a string, using the tokenizer and vocabulary with options to remove special
tokens and clean up tokenization spaces. Similar to doing self.convert_tokens_to_string(self.convert_ids_to_tokens(token_ids)).  encode  < source > ( text: typing.Union[str, typing.List[str], typing.List[int]] text_pair: typing.Union[str, typing.List[str], typing.List[int], NoneType] = None add_special_tokens: bool = True padding: typing.Union[bool, str, transformers.utils.generic.PaddingStrategy] = False truncation: typing.Union[bool, str, transformers.tokenization_utils_base.TruncationStrategy] = None max_length: typing.Optional[int] = None stride: int = 0 return_tensors: typing.Union[str, transformers.utils.generic.TensorType, NoneType] = None **kwargs  ) → List[int], torch.Tensor, tf.Tensor or np.ndarray  Parameters   text (st


PreTrainedTokenizerFast

r, List[str] or List[int]) —
The first sequence to be encoded. This can be a string, a list of strings (tokenized string using the
tokenize method) or a list of integers (tokenized string ids using the convert_tokens_to_ids
method).   text_pair (str, List[str] or List[int], optional) —
Optional second sequence to be encoded. This can be a string, a list of strings (tokenized string using
the tokenize method) or a list of integers (tokenized string ids using the convert_tokens_to_ids
method).   add_special_tokens (bool, optional, defaults to True) —
Whether or not to add special tokens when encoding the sequences. This will use the underlying
PretrainedTokenizerBase.build_inputs_with_special_tokens function, which defines which tokens are
automatically added to the input ids. This is usefull if you want to add bos or eos tokens
automatically.   padding (bool, str or PaddingStrategy, optional, defaults to False) —
Activates and controls padding. Accepts the foll


PreTrainedTokenizerFast

owing values:
True or 'longest': Pad to the longest sequence in the batch (or no padding if only a single
sequence if provided).
'max_length': Pad to a maximum length specified with the argument max_length or to the maximum
acceptable input length for the model if that argument is not provided.
False or 'do_not_pad' (default): No padding (i.e., can output a batch with sequences of different
lengths).
   truncation (bool, str or TruncationStrategy, optional, defaults to False) —
Activates and controls truncation. Accepts the following values:
True or 'longest_first': Truncate to a maximum length specified with the argument max_length or
to the maximum acceptable input length for the model if that argument is not provided. This will
truncate token by token, removing a token from the longest sequence in the pair if a pair of
sequences (or a batch of pairs) is provided.
'only_first': Truncate to a maximum length specified with the argument max_length or to the
max


PreTrainedTokenizerFast

imum acceptable input length for the model if that argument is not provided. This will only
truncate the first sequence of a pair if a pair of sequences (or a batch of pairs) is provided.
'only_second': Truncate to a maximum length specified with the argument max_length or to the
maximum acceptable input length for the model if that argument is not provided. This will only
truncate the second sequence of a pair if a pair of sequences (or a batch of pairs) is provided.
False or 'do_not_truncate' (default): No truncation (i.e., can output batch with sequence lengths
greater than the model maximum admissible input size).
   max_length (int, optional) —
Controls the maximum length to use by one of the truncation/padding parameters.
If left unset or set to None, this will use the predefined model maximum length if a maximum length
is required by one of the truncation/padding parameters. If the model has no specific maximum input
length (like XLNet) truncation/paddi


PreTrainedTokenizerFast

ng to a maximum length will be deactivated.   stride (int, optional, defaults to 0) —
If set to a number along with max_length, the overflowing tokens returned when
return_overflowing_tokens=True will contain some tokens from the end of the truncated sequence
returned to provide some overlap between truncated and overflowing sequences. The value of this
argument defines the number of overlapping tokens.   is_split_into_words (bool, optional, defaults to False) —
Whether or not the input is already pre-tokenized (e.g., split into words). If set to True, the
tokenizer assumes the input is already split into words (for instance, by splitting it on whitespace)
which it will tokenize. This is useful for NER or token classification.   pad_to_multiple_of (int, optional) —
If set will pad the sequence to a multiple of the provided value. Requires padding to be activated.
This is especially useful to enable the use of Tensor Cores on NVIDIA hardware with compute capabi


PreTrainedTokenizerFast

lity
>= 7.5 (Volta).   return_tensors (str or TensorType, optional) —
If set, will return tensors instead of list of python integers. Acceptable values are:
'tf': Return TensorFlow tf.constant objects.
'pt': Return PyTorch torch.Tensor objects.
'np': Return Numpy np.ndarray objects.
**kwargs — Passed along to the .tokenize() method.   Returns
List[int], torch.Tensor, tf.Tensor or np.ndarray

The tokenized ids of the text.
  Converts a string to a sequence of ids (integer), using the tokenizer and vocabulary. Same as doing self.convert_tokens_to_ids(self.tokenize(text)).  push_to_hub  < source > ( repo_id: str use_temp_dir: typing.Optional[bool] = None commit_message: typing.Optional[str] = None private: typing.Optional[bool] = None token: typing.Union[bool, str, NoneType] = None max_shard_size: typing.Union[int, str, NoneType] = '10GB' create_pr: bool = False safe_serialization: bool = False revision: str = None **deprecated_kwargs  )   Parameters   repo_id


PreTrainedTokenizerFast

 (str) —
The name of the repository you want to push your tokenizer to. It should contain your organization name
when pushing to a given organization.   use_temp_dir (bool, optional) —
Whether or not to use a temporary directory to store the files saved before they are pushed to the Hub.
Will default to True if there is no directory named like repo_id, False otherwise.   commit_message (str, optional) —
Message to commit while pushing. Will default to "Upload tokenizer".   private (bool, optional) —
Whether or not the repository created should be private.   token (bool or str, optional) —
The token to use as HTTP bearer authorization for remote files. If True, will use the token generated
when running huggingface-cli login (stored in ~/.huggingface). Will default to True if repo_url
is not specified.   max_shard_size (int or str, optional, defaults to "10GB") —
Only applicable for models. The maximum size for a checkpoint before being sharded. Checkpoints shar


PreTrainedTokenizerFast

d
will then be each of size lower than this size. If expressed as a string, needs to be digits followed
by a unit (like "5MB").   create_pr (bool, optional, defaults to False) —
Whether or not to create a PR with the uploaded files or directly commit.   safe_serialization (bool, optional, defaults to False) —
Whether or not to convert the model weights in safetensors format for safer serialization.   revision (str, optional) —
Branch to push the uploaded files to.    Upload the tokenizer files to the 🤗 Model Hub.  Examples:   Copied from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("bert-base-cased")
# Push the tokenizer to your namespace with the name "my-finetuned-bert".
tokenizer.push_to_hub("my-finetuned-bert")
# Push the tokenizer to an organization with the name "my-finetuned-bert".
tokenizer.push_to_hub("huggingface/my-finetuned-bert")  convert_ids_to_tokens  < source > ( ids: typing.Union[int, typing.List[int]] skip_speci


PreTrainedTokenizerFast

al_tokens: bool = False  ) → str or List[str]  Parameters   ids (int or List[int]) —
The token id (or token ids) to convert to tokens.   skip_special_tokens (bool, optional, defaults to False) —
Whether or not to remove special tokens in the decoding.   Returns
str or List[str]

The decoded token(s).
  Converts a single index or a sequence of indices in a token or a sequence of tokens, using the vocabulary and
added tokens.  convert_tokens_to_ids  < source > ( tokens: typing.Union[str, typing.List[str]]  ) → int or List[int]  Parameters   tokens (str or List[str]) — One or several token(s) to convert to token id(s).   Returns
int or List[int]

The token id or list of token ids.
  Converts a token string (or a sequence of tokens) in a single integer id (or a sequence of ids), using the
vocabulary.  get_added_vocab  < source > ( ) → Dict[str, int]  Returns
Dict[str, int]

The added tokens.
  Returns the added tokens in the vocabulary as a dictionary of


PreTrainedTokenizerFast

token to index.  num_special_tokens_to_add  < source > ( pair: bool = False  ) → int  Parameters   pair (bool, optional, defaults to False) —
Whether the number of added tokens should be computed in the case of a sequence pair or a single
sequence.   Returns
int

Number of special tokens added to sequences.
  Returns the number of added tokens when encoding a sequence with special tokens. This encodes a dummy input and checks the number of added tokens, and is therefore not efficient. Do not put
this inside your training loop.  set_truncation_and_padding  < source > ( padding_strategy: PaddingStrategy truncation_strategy: TruncationStrategy max_length: int stride: int pad_to_multiple_of: typing.Optional[int]  )   Parameters   padding_strategy (PaddingStrategy) —
The kind of padding that will be applied to the input   truncation_strategy (TruncationStrategy) —
The kind of truncation that will be applied to the input   max_length (int) —
The maximum size of a


PreTrainedTokenizerFast

 sequence.   stride (int) —
The stride to use when handling overflow.   pad_to_multiple_of (int, optional) —
If set will pad the sequence to a multiple of the provided value. This is especially useful to enable
the use of Tensor Cores on NVIDIA hardware with compute capability >= 7.5 (Volta).    Define the truncation and the padding strategies for fast tokenizers (provided by HuggingFace tokenizers
library) and restore the tokenizer settings afterwards. The provided tokenizer has no padding / truncation strategy before the managed section. If your tokenizer set a
padding / truncation strategy before, then it will be reset to no padding / truncation when exiting the managed
section.  train_new_from_iterator  < source > ( text_iterator vocab_size length = None new_special_tokens = None special_tokens_map = None **kwargs  ) → PreTrainedTokenizerFast  Parameters   text_iterator (generator of List[str]) —
The training corpus. Should be a generator of batches of tex


PreTrainedTokenizerFast

ts, for instance a list of lists of texts
if you have everything in memory.   vocab_size (int) —
The size of the vocabulary you want for your tokenizer.   length (int, optional) —
The total number of sequences in the iterator. This is used to provide meaningful progress tracking   new_special_tokens (list of str or AddedToken, optional) —
A list of new special tokens to add to the tokenizer you are training.   special_tokens_map (Dict[str, str], optional) —
If you want to rename some of the special tokens this tokenizer uses, pass along a mapping old special
token name to new special token name in this argument.   kwargs (Dict[str, Any], optional) —
Additional keyword arguments passed along to the trainer from the 🤗 Tokenizers library.   Returns
PreTrainedTokenizerFast

A new tokenizer of the same type as the original one, trained on
text_iterator.
  Trains a tokenizer on a new corpus with the same defaults (in terms of special tokens or tokenization pipeli


BatchEncoding

class transformers.BatchEncoding  < source > ( data: typing.Union[typing.Dict[str, typing.Any], NoneType] = None encoding: typing.Union[tokenizers.Encoding, typing.Sequence[tokenizers.Encoding], NoneType] = None tensor_type: typing.Union[NoneType, str, transformers.utils.generic.TensorType] = None prepend_batch_axis: bool = False n_sequences: typing.Optional[int] = None  )   Parameters   data (dict, optional) —
Dictionary of lists/arrays/tensors returned by the __call__/encode_plus/batch_encode_plus methods
(‘input_ids’, ‘attention_mask’, etc.).   encoding (tokenizers.Encoding or Sequence[tokenizers.Encoding], optional) —
If the tokenizer is a fast tokenizer which outputs additional information like mapping from word/character
space to token space the tokenizers.Encoding instance or list of instance (for batches) hold this
information.   tensor_type (Union[None, str, TensorType], optional) —
You can give a tensor_type here to con


BatchEncoding

vert the lists of integers in PyTorch/TensorFlow/Numpy Tensors at
initialization.   prepend_batch_axis (bool, optional, defaults to False) —
Whether or not to add a batch axis when converting to tensors (see tensor_type above).   n_sequences (Optional[int], optional) —
You can give a tensor_type here to convert the lists of integers in PyTorch/TensorFlow/Numpy Tensors at
initialization.    Holds the output of the call(),
encode_plus() and
batch_encode_plus() methods (tokens, attention_masks, etc). This class is derived from a python dictionary and can be used as a dictionary. In addition, this class exposes
utility methods to map from word/character space to token space.  char_to_token  < source > ( batch_or_char_index: int char_index: typing.Optional[int] = None sequence_index: int = 0  ) → int  Parameters   batch_or_char_index (int) —
Index of the sequence in the batch. If the batch only comprise one sequence, this can be the i


BatchEncoding

ndex of
the word in the sequence   char_index (int, optional) —
If a batch index is provided in batch_or_token_index, this can be the index of the word in the
sequence.   sequence_index (int, optional, defaults to 0) —
If pair of sequences are encoded in the batch this can be used to specify which sequence in the pair (0
or 1) the provided character index belongs to.   Returns
int

Index of the token.
  Get the index of the token in the encoded output comprising a character in the original string for a sequence
of the batch. Can be called as: self.char_to_token(char_index) if batch size is 1 self.char_to_token(batch_index, char_index) if batch size is greater or equal to 1 This method is particularly suited when the input sequences are provided as pre-tokenized sequences (i.e. words
are defined by the user). In this case it allows to easily associate encoded tokens with provided tokenized
words.  char_to_word  < source > ( bat


BatchEncoding

ch_or_char_index: int char_index: typing.Optional[int] = None sequence_index: int = 0  ) → int or List[int]  Parameters   batch_or_char_index (int) —
Index of the sequence in the batch. If the batch only comprise one sequence, this can be the index of
the character in the original string.   char_index (int, optional) —
If a batch index is provided in batch_or_token_index, this can be the index of the character in the
original string.   sequence_index (int, optional, defaults to 0) —
If pair of sequences are encoded in the batch this can be used to specify which sequence in the pair (0
or 1) the provided character index belongs to.   Returns
int or List[int]

Index or indices of the associated encoded token(s).
  Get the word in the original string corresponding to a character in the original string of a sequence of the
batch. Can be called as: self.char_to_word(char_index) if batch size is 1 self.char_to_word(batch_index, char


BatchEncoding

_index) if batch size is greater than 1 This method is particularly suited when the input sequences are provided as pre-tokenized sequences (i.e. words
are defined by the user). In this case it allows to easily associate encoded tokens with provided tokenized
words.  convert_to_tensors  < source > ( tensor_type: typing.Union[str, transformers.utils.generic.TensorType, NoneType] = None prepend_batch_axis: bool = False  )   Parameters   tensor_type (str or TensorType, optional) —
The type of tensors to use. If str, should be one of the values of the enum TensorType. If
None, no modification is done.   prepend_batch_axis (int, optional, defaults to False) —
Whether or not to add the batch dimension during the conversion.    Convert the inner content to tensors.  sequence_ids  < source > ( batch_index: int = 0  ) → List[Optional[int]]  Parameters   batch_index (int, optional, defaults to 0) — The index to access in the batch.   Retur


BatchEncoding

ns
List[Optional[int]]

A list indicating the sequence id corresponding to each token. Special tokens added
by the tokenizer are mapped to None and other tokens are mapped to the index of their corresponding
sequence.
  Return a list mapping the tokens to the id of their original sentences: None for special tokens added around or between sequences, 0 for tokens corresponding to words in the first sequence, 1 for tokens corresponding to words in the second sequence when a pair of sequences was jointly
encoded.  to  < source > ( device: typing.Union[str, ForwardRef('torch.device')]  ) → BatchEncoding  Parameters   device (str or torch.device) — The device to put the tensors on.   Returns
BatchEncoding

The same instance after modification.
  Send all values to device by calling v.to(device) (PyTorch only).  token_to_chars  < source > ( batch_or_token_index: int token_index: typing.Optional[int] = None  ) → CharSpan  Parameter


BatchEncoding

s   batch_or_token_index (int) —
Index of the sequence in the batch. If the batch only comprise one sequence, this can be the index of
the token in the sequence.   token_index (int, optional) —
If a batch index is provided in batch_or_token_index, this can be the index of the token or tokens in
the sequence.   Returns
CharSpan

Span of characters in the original string, or None, if the token
(e.g. , ) doesn’t correspond to any chars in the origin string.
  Get the character span corresponding to an encoded token in a sequence of the batch. Character spans are returned as a CharSpan with: start — Index of the first character in the original string associated to the token. end — Index of the character following the last character in the original string associated to the
token. Can be called as: self.token_to_chars(token_index) if batch size is 1 self.token_to_chars(batch_index, token_index) if batch size is greater or equal to 1


BatchEncoding

  token_to_sequence  < source > ( batch_or_token_index: int token_index: typing.Optional[int] = None  ) → int  Parameters   batch_or_token_index (int) —
Index of the sequence in the batch. If the batch only comprises one sequence, this can be the index of
the token in the sequence.   token_index (int, optional) —
If a batch index is provided in batch_or_token_index, this can be the index of the token in the
sequence.   Returns
int

Index of the word in the input sequence.
  Get the index of the sequence represented by the given token. In the general use case, this method returns 0
for a single sequence or the first sequence of a pair, and 1 for the second sequence of a pair Can be called as: self.token_to_sequence(token_index) if batch size is 1 self.token_to_sequence(batch_index, token_index) if batch size is greater than 1 This method is particularly suited when the input sequences are provided as pre-tokenized sequences (i.


BatchEncoding

e.,
words are defined by the user). In this case it allows to easily associate encoded tokens with provided
tokenized words.  token_to_word  < source > ( batch_or_token_index: int token_index: typing.Optional[int] = None  ) → int  Parameters   batch_or_token_index (int) —
Index of the sequence in the batch. If the batch only comprise one sequence, this can be the index of
the token in the sequence.   token_index (int, optional) —
If a batch index is provided in batch_or_token_index, this can be the index of the token in the
sequence.   Returns
int

Index of the word in the input sequence.
  Get the index of the word corresponding (i.e. comprising) to an encoded token in a sequence of the batch. Can be called as: self.token_to_word(token_index) if batch size is 1 self.token_to_word(batch_index, token_index) if batch size is greater than 1 This method is particularly suited when the input sequences are provided as pre-tokenized


BatchEncoding

sequences (i.e.,
words are defined by the user). In this case it allows to easily associate encoded tokens with provided
tokenized words.  tokens  < source > ( batch_index: int = 0  ) → List[str]  Parameters   batch_index (int, optional, defaults to 0) — The index to access in the batch.   Returns
List[str]

The list of tokens at that index.
  Return the list of tokens (sub-parts of the input strings after word/subword splitting and before conversion to
integer indices) at a given batch index (only works for the output of a fast tokenizer).  word_ids  < source > ( batch_index: int = 0  ) → List[Optional[int]]  Parameters   batch_index (int, optional, defaults to 0) — The index to access in the batch.   Returns
List[Optional[int]]

A list indicating the word corresponding to each token. Special tokens added by the
tokenizer are mapped to None and other tokens are mapped to the index of their corresponding word
(several token


BatchEncoding

s will be mapped to the same word index if they are parts of that word).
  Return a list mapping the tokens to their actual word in the initial sentence for a fast tokenizer.  word_to_chars  < source > ( batch_or_word_index: int word_index: typing.Optional[int] = None sequence_index: int = 0  ) → CharSpan or List[CharSpan]  Parameters   batch_or_word_index (int) —
Index of the sequence in the batch. If the batch only comprise one sequence, this can be the index of
the word in the sequence   word_index (int, optional) —
If a batch index is provided in batch_or_token_index, this can be the index of the word in the
sequence.   sequence_index (int, optional, defaults to 0) —
If pair of sequences are encoded in the batch this can be used to specify which sequence in the pair (0
or 1) the provided word index belongs to.   Returns
CharSpan or List[CharSpan]

Span(s) of the associated character or characters in the string. CharSpan
ar


BatchEncoding

e NamedTuple with:
start: index of the first character associated to the token in the original string
end: index of the character following the last character associated to the token in the original
string
  Get the character span in the original string corresponding to given word in a sequence of the batch. Character spans are returned as a CharSpan NamedTuple with: start: index of the first character in the original string end: index of the character following the last character in the original string Can be called as: self.word_to_chars(word_index) if batch size is 1 self.word_to_chars(batch_index, word_index) if batch size is greater or equal to 1  word_to_tokens  < source > ( batch_or_word_index: int word_index: typing.Optional[int] = None sequence_index: int = 0  ) → (TokenSpan, optional)  Parameters   batch_or_word_index (int) —
Index of the sequence in the batch. If the batch only comprises one sequence, this can be the i


BatchEncoding

ndex of
the word in the sequence.   word_index (int, optional) —
If a batch index is provided in batch_or_token_index, this can be the index of the word in the
sequence.   sequence_index (int, optional, defaults to 0) —
If pair of sequences are encoded in the batch this can be used to specify which sequence in the pair (0
or 1) the provided word index belongs to.   Returns
(TokenSpan, optional)

Span of tokens in the encoded sequence. Returns
None if no tokens correspond to the word. This can happen especially when the token is a special token
that has been used to format the tokenization. For example when we add a class token at the very beginning
of the tokenization.
  Get the encoded token span corresponding to a word in a sequence of the batch. Token spans are returned as a TokenSpan with: start — Index of the first token. end — Index of the token following the last token. Can be called as: self.word_to_tokens(word_index,


BatchEncoding

sequence_index: int = 0) if batch size is 1 self.word_to_tokens(batch_index, word_index, sequence_index: int = 0) if batch size is greater or equal to
1 This method is particularly suited when the input sequences are provided as pre-tokenized sequences (i.e. words
are defined by the user). In this case it allows to easily associate encoded tokens with provided tokenized
words.  words  < source > ( batch_index: int = 0  ) → List[Optional[int]]  Parameters   batch_index (int, optional, defaults to 0) — The index to access in the batch.   Returns
List[Optional[int]]

A list indicating the word corresponding to each token. Special tokens added by the
tokenizer are mapped to None and other tokens are mapped to the index of their corresponding word
(several tokens will be mapped to the same word index if they are parts of that word).
  Return a list mapping the tokens to their actual word in the initial sentence for a fast tokenizer


Model outputs

All models have outputs that are instances of subclasses of ModelOutput. Those are
data structures containing all the information returned by the model, but that can also be used as tuples or
dictionaries.
Let’s see how this looks in an example:
  Copied from transformers import BertTokenizer, BertForSequenceClassification
import torch
tokenizer = BertTokenizer.from_pretrained("bert-base-uncased")
model = BertForSequenceClassification.from_pretrained("bert-base-uncased")
inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
labels = torch.tensor([1]).unsqueeze(0)  # Batch size 1
outputs = model(**inputs, labels=labels)
The outputs object is a SequenceClassifierOutput, as we can see i


Model outputs

n the
documentation of that class below, it means it has an optional loss, a logits, an optional hidden_states and
an optional attentions attribute. Here we have the loss since we passed along labels, but we don’t have
hidden_states and attentions because we didn’t pass output_hidden_states=True or
output_attentions=True.
When passing output_hidden_states=True you may expect the outputs.hidden_states[-1] to match outputs.last_hidden_states exactly.
However, this is not always the case. Some models apply normalization or subsequent process to the last hidden state when it’s returned.
You can access each attribute as you would usually do, and if that attribute has not been returned by the model,


Model outputs

you
will get None. Here for instance outputs.loss is the loss computed by the model, and outputs.attentions is
None.
When considering our outputs object as tuple, it only considers the attributes that don’t have None values.
Here for instance, it has two elements, loss then logits, so
  Copied outputs[:2]
will return the tuple (outputs.loss, outputs.logits) for instance.
When considering our outputs object as dictionary, it only considers the attributes that don’t have None
values. Here for instance, it has two keys that are loss and logits.
We document here the generic model outputs that are used by more than one model type. Specific output types are
documented on their corresponding model pag


ModelOutput

class transformers.utils.ModelOutput  < source > ( *args **kwargs  )    Base class for all model outputs as dataclass. Has a __getitem__ that allows indexing by integer or slice (like a
tuple) or strings (like a dictionary) that will ignore the None attributes. Otherwise behaves like a regular
python dictionary. You can’t unpack a ModelOutput directly. Use the to_tuple() method to convert it to a tuple
before.  to_tuple  < source > ( )    Convert self to a tuple containing all the attributes/keys that are not None.


BaseModelOutput

class transformers.modeling_outputs.BaseModelOutput  < source > ( last_hidden_state: FloatTensor = None hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None  )   Parameters   last_hidden_state (torch.FloatTensor of shape (batch_size, sequence_length, hidden_size)) —
Sequence of hidden-states at the output of the last layer of the model.   hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of torch.FloatTensor (one for the output of the embeddings, if the model has


BaseModelOutput

an embedding layer, +
one for the output of each layer) of shape (batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.   attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for model’s outputs, with potential hidden states and attentions


BaseModelOutputWithPooling

class transformers.modeling_outputs.BaseModelOutputWithPooling  < source > ( last_hidden_state: FloatTensor = None pooler_output: FloatTensor = None hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None  )   Parameters   last_hidden_state (torch.FloatTensor of shape (batch_size, sequence_length, hidden_size)) —
Sequence of hidden-states at the output of the last layer of the model.   pooler_output (torch.FloatTensor of shape (batch_size, hidden_size)) —
Last layer hidden-state of the first token of the sequence (classification token) after further processing
through the layers used for the auxiliary pretraining task. E.g. for BERT-family of models, this returns
the classification token after processing through a linear layer and a tanh activation function. The linear
layer weights are trained from the next sentence prediction (classific


BaseModelOutputWithPooling

ation) objective during pretraining.   hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of torch.FloatTensor (one for the output of the embeddings, if the model has an embedding layer, +
one for the output of each layer) of shape (batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.   attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for model’s outputs that also contains a pooling of the last hidden states.


BaseModelOutputWithCrossAttentions

class transformers.modeling_outputs.BaseModelOutputWithCrossAttentions  < source > ( last_hidden_state: FloatTensor = None hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None cross_attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None  )   Parameters   last_hidden_state (torch.FloatTensor of shape (batch_size, sequence_length, hidden_size)) —
Sequence of hidden-states at the output of the last layer of the model.   hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of torch.FloatTensor (one for the output of the embeddings, if the model has an embedding layer, +
one for the output of each layer) of shape (batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the option


BaseModelOutputWithCrossAttentions

al initial embedding outputs.   attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.   cross_attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True and config.add_cross_attention=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the decoder’s cross-attention layer, after the attention softmax, used to compute the
weighted average in the cross-attention heads.    Base class for model’s outputs, with potential hidden states and attentions


class transformers.modeling_outputs.BaseModelOutputWithPoolingAndCrossAttentions  < source > ( last_hidden_state: FloatTensor = None pooler_output: FloatTensor = None hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None past_key_values: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None cross_attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None  )   Parameters   last_hidden_state (torch.FloatTensor of shape (batch_size, sequence_length, hidden_size)) —
Sequence of hidden-states at the output of the last layer of the model.   pooler_output (torch.FloatTensor of shape (batch_size, hidden_size)) —
Last layer hidden-state of the first token of the sequence (classification


 token) after further processing
through the layers used for the auxiliary pretraining task. E.g. for BERT-family of models, this returns
the classification token after processing through a linear layer and a tanh activation function. The linear
layer weights are trained from the next sentence prediction (classification) objective during pretraining.   hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of torch.FloatTensor (one for the output of the embeddings, if the model has an embedding layer, +
one for the output of each layer) of shape (batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.   at


tentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.   cross_attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True and config.add_cross_attention=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the decoder’s cross-attention layer, after the attention softmax, used to compute the
weighted average in the cros


s-attention heads.   past_key_values (tuple(tuple(torch.FloatTensor)), optional, returned when use_cache=True is passed or when config.use_cache=True) —
Tuple of tuple(torch.FloatTensor) of length config.n_layers, with each tuple having 2 tensors of shape
(batch_size, num_heads, sequence_length, embed_size_per_head)) and optionally if
config.is_encoder_decoder=True 2 additional tensors of shape (batch_size, num_heads, encoder_sequence_length, embed_size_per_head).
Contains pre-computed hidden-states (key and values in the self-attention blocks and optionally if
config.is_encoder_decoder=True in the cross-attention blocks) that can be used (see past_key_values
input) to speed up sequential decoding.    Base class for model’s outputs that also contains a pooling of the last hidden states


BaseModelOutputWithPast

class transformers.modeling_outputs.BaseModelOutputWithPast  < source > ( last_hidden_state: FloatTensor = None past_key_values: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None  )   Parameters   last_hidden_state (torch.FloatTensor of shape (batch_size, sequence_length, hidden_size)) —
Sequence of hidden-states at the output of the last layer of the model.
If past_key_values is used only the last hidden-state of the sequences of shape (batch_size, 1, hidden_size) is output.   past_key_values (tuple(tuple(torch.FloatTensor)), optional, returned when use_cache=True is passed or when


BaseModelOutputWithPast

 config.use_cache=True) —
Tuple of tuple(torch.FloatTensor) of length config.n_layers, with each tuple having 2 tensors of shape
(batch_size, num_heads, sequence_length, embed_size_per_head)) and optionally if
config.is_encoder_decoder=True 2 additional tensors of shape (batch_size, num_heads, encoder_sequence_length, embed_size_per_head).
Contains pre-computed hidden-states (key and values in the self-attention blocks and optionally if
config.is_encoder_decoder=True in the cross-attention blocks) that can be used (see past_key_values
input) to speed up sequential decoding.   hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of torch.Float


BaseModelOutputWithPast

Tensor (one for the output of the embeddings, if the model has an embedding layer, +
one for the output of each layer) of shape (batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.   attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for model’s outputs that may also contain a past key/values (to speed up sequential decoding).


class transformers.modeling_outputs.BaseModelOutputWithPastAndCrossAttentions  < source > ( last_hidden_state: FloatTensor = None past_key_values: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None cross_attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None  )   Parameters   last_hidden_state (torch.FloatTensor of shape (batch_size, sequence_length, hidden_size)) —
Sequence of hidden-states at the output of the last layer of the model.
If past_key_values is used only the last hidden-state of the sequences of shape (batch_size, 1, hidden_size) is output.   past_key_values (tuple(tuple(torch.FloatTensor)), optional, returned when use_cache=True is passed or when config.use_cache=True) —
Tuple of tuple(torch.FloatTensor) of length config.n_layers, wi


th each tuple having 2 tensors of shape
(batch_size, num_heads, sequence_length, embed_size_per_head)) and optionally if
config.is_encoder_decoder=True 2 additional tensors of shape (batch_size, num_heads, encoder_sequence_length, embed_size_per_head).
Contains pre-computed hidden-states (key and values in the self-attention blocks and optionally if
config.is_encoder_decoder=True in the cross-attention blocks) that can be used (see past_key_values
input) to speed up sequential decoding.   hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of torch.FloatTensor (one for the output of the embeddings, if the model has an embedding layer, +
one for the output of each layer) of shape (batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the optional initial embedding outputs


.   attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.   cross_attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True and config.add_cross_attention=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the decoder’s cross-attention layer, after the attention softmax, used to compute the
weighted average in the cross-attention heads.    Base class for model’s outputs that may also contain a past key/values (to speed up sequential decoding).


Seq2SeqModelOutput

class transformers.modeling_outputs.Seq2SeqModelOutput  < source > ( last_hidden_state: FloatTensor = None past_key_values: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None decoder_hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None decoder_attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None cross_attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None encoder_last_hidden_state: typing.Optional[torch.FloatTensor] = None encoder_hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None encoder_attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None  )   Parameters   last_hidden_state (torch.FloatTensor of shape (batch_size, sequence_length, hidden_size)) —
Sequence of hidden-states at the output of the last layer of the decoder of the model.
If past_key_values is used only the last hidden-state of the sequences of shape (batch_size, 1, hidden_size) is output.   past_key_values (tuple(tuple(t


Seq2SeqModelOutput

orch.FloatTensor)), optional, returned when use_cache=True is passed or when config.use_cache=True) —
Tuple of tuple(torch.FloatTensor) of length config.n_layers, with each tuple having 2 tensors of shape
(batch_size, num_heads, sequence_length, embed_size_per_head)) and 2 additional tensors of shape
(batch_size, num_heads, encoder_sequence_length, embed_size_per_head).
Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention
blocks) that can be used (see past_key_values input) to speed up sequential decoding.   decoder_hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of torch.FloatTensor (one for the output of the embeddings, if the model has an embedding layer, +
one for the output of each layer) of shape (batch_size, sequence_length, hidden_size).
Hidden-states of the decoder at the output of each layer plus the optional initial


Seq2SeqModelOutput

 embedding outputs.   decoder_attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the decoder, after the attention softmax, used to compute the weighted average in the
self-attention heads.   cross_attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the decoder’s cross-attention layer, after the attention softmax, used to compute the
weighted average in the cross-attention heads.   encoder_last_hidden_state (torch.FloatTensor of shape (batch_size, sequence_length, hidden_size), optional) —
Sequence of hidden-states at the output of the last layer


Seq2SeqModelOutput

 of the encoder of the model.   encoder_hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of torch.FloatTensor (one for the output of the embeddings, if the model has an embedding layer, +
one for the output of each layer) of shape (batch_size, sequence_length, hidden_size).
Hidden-states of the encoder at the output of each layer plus the optional initial embedding outputs.   encoder_attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the encoder, after the attention softmax, used to compute the weighted average in the
self-attention heads.    Base class for model encoder’s outputs that also contains : pre-computed hidden states that can speed up sequential
decodi


CausalLMOutput

class transformers.modeling_outputs.CausalLMOutput  < source > ( loss: typing.Optional[torch.FloatTensor] = None logits: FloatTensor = None hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None  )   Parameters   loss (torch.FloatTensor of shape (1,), optional, returned when labels is provided) —
Language modeling loss (for next-token prediction).   logits (torch.FloatTensor of shape (batch_size, sequence_length, config.vocab_size)) —
Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax).   hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_h


CausalLMOutput

idden_states=True) —
Tuple of torch.FloatTensor (one for the output of the embeddings, if the model has an embedding layer, +
one for the output of each layer) of shape (batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.   attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for causal language model (or autoregressive) outputs


CausalLMOutputWithCrossAttentions

class transformers.modeling_outputs.CausalLMOutputWithCrossAttentions  < source > ( loss: typing.Optional[torch.FloatTensor] = None logits: FloatTensor = None past_key_values: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None cross_attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None  )   Parameters   loss (torch.FloatTensor of shape (1,), optional, returned when labels is provided) —
Language modeling loss (for next-token prediction).   logits (torch.FloatTensor of shape (batch_size, sequence_length, config.vocab_size)) —
Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax).   hidden_states (tuple(torch.FloatTensor), optional, returned when


CausalLMOutputWithCrossAttentions

 output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of torch.FloatTensor (one for the output of the embeddings, if the model has an embedding layer, +
one for the output of each layer) of shape (batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.   attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.   cross_attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_at


CausalLMOutputWithCrossAttentions

tentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Cross attentions weights after the attention softmax, used to compute the weighted average in the
cross-attention heads.   past_key_values (tuple(tuple(torch.FloatTensor)), optional, returned when use_cache=True is passed or when config.use_cache=True) —
Tuple of torch.FloatTensor tuples of length config.n_layers, with each tuple containing the cached key,
value states of the self-attention and the cross-attention layers if model is used in encoder-decoder
setting. Only relevant if config.is_decoder = True.
Contains pre-computed hidden-states (key and values in the attention blocks) that can be used (see
past_key_values input) to speed up sequential decoding.    Base class for causal language model (or autoregressive) outputs


CausalLMOutputWithPast

class transformers.modeling_outputs.CausalLMOutputWithPast  < source > ( loss: typing.Optional[torch.FloatTensor] = None logits: FloatTensor = None past_key_values: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None  )   Parameters   loss (torch.FloatTensor of shape (1,), optional, returned when labels is provided) —
Language modeling loss (for next-token prediction).   logits (torch.FloatTensor of shape (batch_size, sequence_length, config.vocab_size)) —
Prediction scores of the language modeling head (scores for each v


CausalLMOutputWithPast

ocabulary token before SoftMax).   past_key_values (tuple(tuple(torch.FloatTensor)), optional, returned when use_cache=True is passed or when config.use_cache=True) —
Tuple of tuple(torch.FloatTensor) of length config.n_layers, with each tuple having 2 tensors of shape
(batch_size, num_heads, sequence_length, embed_size_per_head))
Contains pre-computed hidden-states (key and values in the self-attention blocks) that can be used (see
past_key_values input) to speed up sequential decoding.   hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of torch.FloatTensor (one for the outp


CausalLMOutputWithPast

ut of the embeddings, if the model has an embedding layer, +
one for the output of each layer) of shape (batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.   attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for causal language model (or autoregressive) outputs.


MaskedLMOutput

class transformers.modeling_outputs.MaskedLMOutput  < source > ( loss: typing.Optional[torch.FloatTensor] = None logits: FloatTensor = None hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None  )   Parameters   loss (torch.FloatTensor of shape (1,), optional, returned when labels is provided) —
Masked language modeling (MLM) loss.   logits (torch.FloatTensor of shape (batch_size, sequence_length, config.vocab_size)) —
Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax).   hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output


MaskedLMOutput

_hidden_states=True) —
Tuple of torch.FloatTensor (one for the output of the embeddings, if the model has an embedding layer, +
one for the output of each layer) of shape (batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.   attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for masked language models outputs


Seq2SeqLMOutput

class transformers.modeling_outputs.Seq2SeqLMOutput  < source > ( loss: typing.Optional[torch.FloatTensor] = None logits: FloatTensor = None past_key_values: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None decoder_hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None decoder_attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None cross_attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None encoder_last_hidden_state: typing.Optional[torch.FloatTensor] = None encoder_hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None encoder_attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None  )   Parameters   loss (torch.FloatTensor of shape (1,), optional, returned when labels is provided) —
Language modeling loss.   logits (torch.FloatTensor of shape (batch_size, sequence_length, config.vocab_size)) —
Prediction scores of the language modeling head (scores for each vocabulary token befor


Seq2SeqLMOutput

e SoftMax).   past_key_values (tuple(tuple(torch.FloatTensor)), optional, returned when use_cache=True is passed or when config.use_cache=True) —
Tuple of tuple(torch.FloatTensor) of length config.n_layers, with each tuple having 2 tensors of shape
(batch_size, num_heads, sequence_length, embed_size_per_head)) and 2 additional tensors of shape
(batch_size, num_heads, encoder_sequence_length, embed_size_per_head).
Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention
blocks) that can be used (see past_key_values input) to speed up sequential decoding.   decoder_hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of torch.FloatTensor (one for the output of the embeddings, if the model has an embedding layer, +
one for the output of each layer) of shape (batch_size, sequence_length, hidden_size).
Hidden-states of the decod


Seq2SeqLMOutput

er at the output of each layer plus the initial embedding outputs.   decoder_attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the decoder, after the attention softmax, used to compute the weighted average in the
self-attention heads.   cross_attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the decoder’s cross-attention layer, after the attention softmax, used to compute the
weighted average in the cross-attention heads.   encoder_last_hidden_state (torch.FloatTensor of shape (batch_size, sequence_length, hidden_size), optional)


Seq2SeqLMOutput

—
Sequence of hidden-states at the output of the last layer of the encoder of the model.   encoder_hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of torch.FloatTensor (one for the output of the embeddings, if the model has an embedding layer, +
one for the output of each layer) of shape (batch_size, sequence_length, hidden_size).
Hidden-states of the encoder at the output of each layer plus the initial embedding outputs.   encoder_attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the encoder, after the attention softmax, used to compute the weighted average in the
self-attention heads.    Base class for sequence-to-sequence language models outputs.


NextSentencePredictorOutput

class transformers.modeling_outputs.NextSentencePredictorOutput  < source > ( loss: typing.Optional[torch.FloatTensor] = None logits: FloatTensor = None hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None  )   Parameters   loss (torch.FloatTensor of shape (1,), optional, returned when next_sentence_label is provided) —
Next sequence prediction (classification) loss.   logits (torch.FloatTensor of shape (batch_size, 2)) —
Prediction scores of the next sequence prediction (classification) head (scores of True/False continuation
before SoftMax).   hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidd


NextSentencePredictorOutput

en_states=True) —
Tuple of torch.FloatTensor (one for the output of the embeddings, if the model has an embedding layer, +
one for the output of each layer) of shape (batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.   attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for outputs of models predicting if two sentences are consecutive or not.


SequenceClassifierOutput

class transformers.modeling_outputs.SequenceClassifierOutput  < source > ( loss: typing.Optional[torch.FloatTensor] = None logits: FloatTensor = None hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None  )   Parameters   loss (torch.FloatTensor of shape (1,), optional, returned when labels is provided) —
Classification (or regression if config.num_labels==1) loss.   logits (torch.FloatTensor of shape (batch_size, config.num_labels)) —
Classification (or regression if config.num_labels==1) scores (before SoftMax).   hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden


SequenceClassifierOutput

_states=True) —
Tuple of torch.FloatTensor (one for the output of the embeddings, if the model has an embedding layer, +
one for the output of each layer) of shape (batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.   attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for outputs of sentence classification models.


class transformers.modeling_outputs.Seq2SeqSequenceClassifierOutput  < source > ( loss: typing.Optional[torch.FloatTensor] = None logits: FloatTensor = None past_key_values: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None decoder_hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None decoder_attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None cross_attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None encoder_last_hidden_state: typing.Optional[torch.FloatTensor] = None encoder_hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None encoder_attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None  )   Parameters   loss (torch.FloatTensor of shape (1,), optional, returned when label is provided) —
Classification (or regression if config.num_labels==1) loss.   logits (torch.FloatTensor of shape (batch_size, config.num_labels)) —
Classification (or regression if config.num_labels==1) score


s (before SoftMax).   past_key_values (tuple(tuple(torch.FloatTensor)), optional, returned when use_cache=True is passed or when config.use_cache=True) —
Tuple of tuple(torch.FloatTensor) of length config.n_layers, with each tuple having 2 tensors of shape
(batch_size, num_heads, sequence_length, embed_size_per_head)) and 2 additional tensors of shape
(batch_size, num_heads, encoder_sequence_length, embed_size_per_head).
Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention
blocks) that can be used (see past_key_values input) to speed up sequential decoding.   decoder_hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of torch.FloatTensor (one for the output of the embeddings, if the model has an embedding layer, +
one for the output of each layer) of shape (batch_size, sequence_length, hidden_size).
Hidden-states of the decod


er at the output of each layer plus the initial embedding outputs.   decoder_attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the decoder, after the attention softmax, used to compute the weighted average in the
self-attention heads.   cross_attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the decoder’s cross-attention layer, after the attention softmax, used to compute the
weighted average in the cross-attention heads.   encoder_last_hidden_state (torch.FloatTensor of shape (batch_size, sequence_length, hidden_size), optional) —
Sequen


ce of hidden-states at the output of the last layer of the encoder of the model.   encoder_hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of torch.FloatTensor (one for the output of the embeddings, if the model has an embedding layer, +
one for the output of each layer) of shape (batch_size, sequence_length, hidden_size).
Hidden-states of the encoder at the output of each layer plus the initial embedding outputs.   encoder_attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the encoder, after the attention softmax, used to compute the weighted average in the
self-attention heads.    Base class for outputs of sequence-to-sequence sentence classification model


MultipleChoiceModelOutput

class transformers.modeling_outputs.MultipleChoiceModelOutput  < source > ( loss: typing.Optional[torch.FloatTensor] = None logits: FloatTensor = None hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None  )   Parameters   loss (torch.FloatTensor of shape (1,), optional, returned when labels is provided) —
Classification loss.   logits (torch.FloatTensor of shape (batch_size, num_choices)) —
num_choices is the second dimension of the input tensors. (see input_ids above).
Classification scores (before SoftMax).   hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidd


MultipleChoiceModelOutput

en_states=True) —
Tuple of torch.FloatTensor (one for the output of the embeddings, if the model has an embedding layer, +
one for the output of each layer) of shape (batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.   attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for outputs of multiple choice models.


TokenClassifierOutput

class transformers.modeling_outputs.TokenClassifierOutput  < source > ( loss: typing.Optional[torch.FloatTensor] = None logits: FloatTensor = None hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None  )   Parameters   loss (torch.FloatTensor of shape (1,), optional, returned when labels is provided)  —
Classification loss.   logits (torch.FloatTensor of shape (batch_size, sequence_length, config.num_labels)) —
Classification scores (before SoftMax).   hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of torch.


TokenClassifierOutput

FloatTensor (one for the output of the embeddings, if the model has an embedding layer, +
one for the output of each layer) of shape (batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.   attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for outputs of token classification models.


QuestionAnsweringModelOutput

class transformers.modeling_outputs.QuestionAnsweringModelOutput  < source > ( loss: typing.Optional[torch.FloatTensor] = None start_logits: FloatTensor = None end_logits: FloatTensor = None hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None  )   Parameters   loss (torch.FloatTensor of shape (1,), optional, returned when labels is provided) —
Total span extraction loss is the sum of a Cross-Entropy for the start and end positions.   start_logits (torch.FloatTensor of shape (batch_size, sequence_length)) —
Span-start scores (before SoftMax).   end_logits (torch.FloatTensor of shape (batch_size, sequence_length)) —
Span-end scores (before SoftMax).   hidden_states (tuple(torch.FloatTensor), optional, return


QuestionAnsweringModelOutput

ed when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of torch.FloatTensor (one for the output of the embeddings, if the model has an embedding layer, +
one for the output of each layer) of shape (batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.   attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for outputs of question answering models


class transformers.modeling_outputs.Seq2SeqQuestionAnsweringModelOutput  < source > ( loss: typing.Optional[torch.FloatTensor] = None start_logits: FloatTensor = None end_logits: FloatTensor = None past_key_values: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None decoder_hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None decoder_attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None cross_attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None encoder_last_hidden_state: typing.Optional[torch.FloatTensor] = None encoder_hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None encoder_attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None  )   Parameters   loss (torch.FloatTensor of shape (1,), optional, returned when l


abels is provided) —
Total span extraction loss is the sum of a Cross-Entropy for the start and end positions.   start_logits (torch.FloatTensor of shape (batch_size, sequence_length)) —
Span-start scores (before SoftMax).   end_logits (torch.FloatTensor of shape (batch_size, sequence_length)) —
Span-end scores (before SoftMax).   past_key_values (tuple(tuple(torch.FloatTensor)), optional, returned when use_cache=True is passed or when config.use_cache=True) —
Tuple of tuple(torch.FloatTensor) of length config.n_layers, with each tuple having 2 tensors of shape
(batch_size, num_heads, sequence_length, embed_size_per_head)) and 2 additional tensors of shape
(batch_size, num_heads, encoder_sequence_length, embed_size_per_head).
Contains pre-computed hidden-states (key and values in the self-attention blocks and


 in the cross-attention
blocks) that can be used (see past_key_values input) to speed up sequential decoding.   decoder_hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of torch.FloatTensor (one for the output of the embeddings, if the model has an embedding layer, +
one for the output of each layer) of shape (batch_size, sequence_length, hidden_size).
Hidden-states of the decoder at the output of each layer plus the initial embedding outputs.   decoder_attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentio


ns weights of the decoder, after the attention softmax, used to compute the weighted average in the
self-attention heads.   cross_attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the decoder’s cross-attention layer, after the attention softmax, used to compute the
weighted average in the cross-attention heads.   encoder_last_hidden_state (torch.FloatTensor of shape (batch_size, sequence_length, hidden_size), optional) —
Sequence of hidden-states at the output of the last layer of the encoder of the model.   encoder_hidden_states (tuple(torch.FloatTensor), optional, returned when output_hi


dden_states=True is passed or when config.output_hidden_states=True) —
Tuple of torch.FloatTensor (one for the output of the embeddings, if the model has an embedding layer, +
one for the output of each layer) of shape (batch_size, sequence_length, hidden_size).
Hidden-states of the encoder at the output of each layer plus the initial embedding outputs.   encoder_attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the encoder, after the attention softmax, used to compute the weighted average in the
self-attention heads.    Base class for outputs of sequence-to-sequence question answering mod