class transformers.modeling_outputs.Seq2SeqSpectrogramOutput  < source > ( loss: typing.Optional[torch.FloatTensor] = None spectrogram: FloatTensor = None past_key_values: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None decoder_hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None decoder_attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None cross_attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None encoder_last_hidden_state: typing.Optional[torch.FloatTensor] = None encoder_hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None encoder_attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None  )   Parameters   loss (torch.FloatTensor of shape (1,), optional, returned when labels is provided) —
Spectrogram generation loss.   spectrogram (torch.FloatTensor of shape (batch_size, sequence_length, num_bins)) —
The predicted spectrogram.   past_key_values (tuple(tupl


e(torch.FloatTensor)), optional, returned when use_cache=True is passed or when config.use_cache=True) —
Tuple of tuple(torch.FloatTensor) of length config.n_layers, with each tuple having 2 tensors of shape
(batch_size, num_heads, sequence_length, embed_size_per_head)) and 2 additional tensors of shape
(batch_size, num_heads, encoder_sequence_length, embed_size_per_head).
Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention
blocks) that can be used (see past_key_values input) to speed up sequential decoding.   decoder_hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of torch.FloatTensor (one for the output of the embeddings, if the model has an embedding layer, +
one for the output of each layer) of shape (batch_size, sequence_length, hidden_size).
Hidden-states of the decoder at the output of each


layer plus the initial embedding outputs.   decoder_attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the decoder, after the attention softmax, used to compute the weighted average in the
self-attention heads.   cross_attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the decoder’s cross-attention layer, after the attention softmax, used to compute the
weighted average in the cross-attention heads.   encoder_last_hidden_state (torch.FloatTensor of shape (batch_size, sequence_length, hidden_size), optional) —
Sequenc


e of hidden-states at the output of the last layer of the encoder of the model.   encoder_hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of torch.FloatTensor (one for the output of the embeddings, if the model has an embedding layer, +
one for the output of each layer) of shape (batch_size, sequence_length, hidden_size).
Hidden-states of the encoder at the output of each layer plus the initial embedding outputs.   encoder_attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the encoder, after the attention softmax, used to compute the weighted average in the
self-attention heads.    Base class for sequence-to-sequence spectrogram outpu


SemanticSegmenterOutput

class transformers.modeling_outputs.SemanticSegmenterOutput  < source > ( loss: typing.Optional[torch.FloatTensor] = None logits: FloatTensor = None hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None  )   Parameters   loss (torch.FloatTensor of shape (1,), optional, returned when labels is provided) —
Classification (or regression if config.num_labels==1) loss.   logits (torch.FloatTensor of shape (batch_size, config.num_labels, logits_height, logits_width)) —
Classification scores for each pixel.
The logits returned do not necessarily have the same size as the pixel_values passed as inputs. This is
to avoid doing two interpolations and lose some quality when a user needs to resize the logits to the
original image size as post-processing. You should always check your logits shape


SemanticSegmenterOutput

 and resize as needed.
   hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of torch.FloatTensor (one for the output of the embeddings, if the model has an embedding layer, +
one for the output of each layer) of shape (batch_size, patch_size, hidden_size).
Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.   attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, patch_size, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for outputs of semantic segmentation models


ImageClassifierOutput

class transformers.modeling_outputs.ImageClassifierOutput  < source > ( loss: typing.Optional[torch.FloatTensor] = None logits: FloatTensor = None hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None  )   Parameters   loss (torch.FloatTensor of shape (1,), optional, returned when labels is provided) —
Classification (or regression if config.num_labels==1) loss.   logits (torch.FloatTensor of shape (batch_size, config.num_labels)) —
Classification (or regression if config.num_labels==1) scores (before SoftMax).   hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.ou


ImageClassifierOutput

tput_hidden_states=True) —
Tuple of torch.FloatTensor (one for the output of the embeddings, if the model has an embedding layer, +
one for the output of each stage) of shape (batch_size, sequence_length, hidden_size). Hidden-states
(also called feature maps) of the model at the output of each stage.   attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, patch_size, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for outputs of image classification models.


ImageClassifierOutputWithNoAttention

class transformers.modeling_outputs.ImageClassifierOutputWithNoAttention  < source > ( loss: typing.Optional[torch.FloatTensor] = None logits: FloatTensor = None hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None  )   Parameters   loss (torch.FloatTensor of shape (1,), optional, returned when labels is provided) —
Classification (or regression if config.num_labels==1) loss.   logits (torch.FloatTensor of shape (batch_size, config.num_labels)) —
Classification (or regression if config.num_


ImageClassifierOutputWithNoAttention

labels==1) scores (before SoftMax).   hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of torch.FloatTensor (one for the output of the embeddings, if the model has an embedding layer, +
one for the output of each stage) of shape (batch_size, num_channels, height, width). Hidden-states (also
called feature maps) of the model at the output of each stage.    Base class for outputs of image classification models


DepthEstimatorOutput

class transformers.modeling_outputs.DepthEstimatorOutput  < source > ( loss: typing.Optional[torch.FloatTensor] = None predicted_depth: FloatTensor = None hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None  )   Parameters   loss (torch.FloatTensor of shape (1,), optional, returned when labels is provided) —
Classification (or regression if config.num_labels==1) loss.   predicted_depth (torch.FloatTensor of shape (batch_size, height, width)) —
Predicted depth for each pixel.   hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True


DepthEstimatorOutput

) —
Tuple of torch.FloatTensor (one for the output of the embeddings, if the model has an embedding layer, +
one for the output of each layer) of shape (batch_size, num_channels, height, width).
Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.   attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, patch_size, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for outputs of depth estimation models


class transformers.modeling_outputs.Wav2Vec2BaseModelOutput  < source > ( last_hidden_state: FloatTensor = None extract_features: FloatTensor = None hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None  )   Parameters   last_hidden_state (torch.FloatTensor of shape (batch_size, sequence_length, hidden_size)) —
Sequence of hidden-states at the output of the last layer of the model.   extract_features (torch.FloatTensor of shape (batch_size, sequence_length, conv_dim[-1])) —
Sequence of extracted feature vectors of the last convolutional layer of the model.   hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is


passed or when config.output_hidden_states=True) —
Tuple of torch.FloatTensor (one for the output of the embeddings + one for the output of each layer) of
shape (batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the initial embedding outputs.   attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for models that have been trained with the Wav2Vec2 loss objective


XVectorOutput

class transformers.modeling_outputs.XVectorOutput  < source > ( loss: typing.Optional[torch.FloatTensor] = None logits: FloatTensor = None embeddings: FloatTensor = None hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None  )   Parameters   loss (torch.FloatTensor of shape (1,), optional, returned when labels is provided) —
Classification loss.   logits (torch.FloatTensor of shape (batch_size, config.xvector_output_dim)) —
Classification hidden states before AMSoftmax.   embeddings (torch.FloatTensor of shape (batch_size, config.xvector_output_dim)) —
Utterance embeddings used for vector similarity-based retrieval.   hidden_states (tuple(torch.FloatTensor), o


XVectorOutput

ptional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of torch.FloatTensor (one for the output of the embeddings + one for the output of each layer) of
shape (batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the initial embedding outputs.   attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Output type of Wav2Vec2ForXVector.


class transformers.modeling_outputs.Seq2SeqTSModelOutput  < source > ( last_hidden_state: FloatTensor = None past_key_values: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None decoder_hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None decoder_attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None cross_attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None encoder_last_hidden_state: typing.Optional[torch.FloatTensor] = None encoder_hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None encoder_attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None loc: typing.Optional[torch.FloatTensor] = None scale: typing.Optional[torch.FloatTensor] = None static_features: typing.Optional[torch.FloatTensor] = None  )   Parameters   last_hidden_state (torch.FloatTensor of shape (batch_size, sequence_length, hidden_size)) —
Sequence of hidden-states at the output of the last layer


 of the decoder of the model.
If past_key_values is used only the last hidden-state of the sequences of shape (batch_size, 1, hidden_size) is output.   past_key_values (tuple(tuple(torch.FloatTensor)), optional, returned when use_cache=True is passed or when config.use_cache=True) —
Tuple of tuple(torch.FloatTensor) of length config.n_layers, with each tuple having 2 tensors of shape
(batch_size, num_heads, sequence_length, embed_size_per_head)) and 2 additional tensors of shape
(batch_size, num_heads, encoder_sequence_length, embed_size_per_head).
Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention
blocks) that can be used (see past_key_values input) to speed up sequential decoding.   decoder_hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of torch.FloatTensor (one for the output of the embeddings, i


f the model has an embedding layer, +
one for the output of each layer) of shape (batch_size, sequence_length, hidden_size).
Hidden-states of the decoder at the output of each layer plus the optional initial embedding outputs.   decoder_attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the decoder, after the attention softmax, used to compute the weighted average in the
self-attention heads.   cross_attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the decoder’s cross-attention layer, after the attention softmax, used


 to compute the
weighted average in the cross-attention heads.   encoder_last_hidden_state (torch.FloatTensor of shape (batch_size, sequence_length, hidden_size), optional) —
Sequence of hidden-states at the output of the last layer of the encoder of the model.   encoder_hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of torch.FloatTensor (one for the output of the embeddings, if the model has an embedding layer, +
one for the output of each layer) of shape (batch_size, sequence_length, hidden_size).
Hidden-states of the encoder at the output of each layer plus the optional initial embedding outputs.   encoder_attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length)


.
Attentions weights of the encoder, after the attention softmax, used to compute the weighted average in the
self-attention heads.   loc (torch.FloatTensor of shape (batch_size,) or (batch_size, input_size), optional) —
Shift values of each time series’ context window which is used to give the model inputs of the same
magnitude and then used to shift back to the original magnitude.   scale (torch.FloatTensor of shape (batch_size,) or (batch_size, input_size), optional) —
Scaling values of each time series’ context window which is used to give the model inputs of the same
magnitude and then used to rescale back to the original magnitude.   static_features (torch.FloatTensor of shape (batch_size, feature size), optional) —
Static features of each time series’ in a batch which are copied to the covariates at inference time.    Base class for time series model’s encoder outputs that also contains pre-computed hidden states that can speed up
sequential decoding


class transformers.modeling_outputs.Seq2SeqTSPredictionOutput  < source > ( loss: typing.Optional[torch.FloatTensor] = None params: typing.Optional[typing.Tuple[torch.FloatTensor]] = None past_key_values: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None decoder_hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None decoder_attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None cross_attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None encoder_last_hidden_state: typing.Optional[torch.FloatTensor] = None encoder_hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None encoder_attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None loc: typing.Optional[torch.FloatTensor] = None scale: typing.Optional[torch.FloatTensor] = None static_features: typing.Optional[torch.FloatTensor] = None  )   Parameters   loss (torch.FloatTensor of shape (1,), optional, returned when a fut


ure_values is provided) —
Distributional loss.   params (torch.FloatTensor of shape (batch_size, num_samples, num_params)) —
Parameters of the chosen distribution.   past_key_values (tuple(tuple(torch.FloatTensor)), optional, returned when use_cache=True is passed or when config.use_cache=True) —
Tuple of tuple(torch.FloatTensor) of length config.n_layers, with each tuple having 2 tensors of shape
(batch_size, num_heads, sequence_length, embed_size_per_head)) and 2 additional tensors of shape
(batch_size, num_heads, encoder_sequence_length, embed_size_per_head).
Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention
blocks) that can be used (see past_key_values input) to speed up sequential decoding.   decoder_hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of torch.FloatTensor (one for the output of t


he embeddings, if the model has an embedding layer, +
one for the output of each layer) of shape (batch_size, sequence_length, hidden_size).
Hidden-states of the decoder at the output of each layer plus the initial embedding outputs.   decoder_attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the decoder, after the attention softmax, used to compute the weighted average in the
self-attention heads.   cross_attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the decoder’s cross-attention layer, after the attention soft


max, used to compute the
weighted average in the cross-attention heads.   encoder_last_hidden_state (torch.FloatTensor of shape (batch_size, sequence_length, hidden_size), optional) —
Sequence of hidden-states at the output of the last layer of the encoder of the model.   encoder_hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of torch.FloatTensor (one for the output of the embeddings, if the model has an embedding layer, +
one for the output of each layer) of shape (batch_size, sequence_length, hidden_size).
Hidden-states of the encoder at the output of each layer plus the initial embedding outputs.   encoder_attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_lengt


h).
Attentions weights of the encoder, after the attention softmax, used to compute the weighted average in the
self-attention heads.   loc (torch.FloatTensor of shape (batch_size,) or (batch_size, input_size), optional) —
Shift values of each time series’ context window which is used to give the model inputs of the same
magnitude and then used to shift back to the original magnitude.   scale (torch.FloatTensor of shape (batch_size,) or (batch_size, input_size), optional) —
Scaling values of each time series’ context window which is used to give the model inputs of the same
magnitude and then used to rescale back to the original magnitude.   static_features (torch.FloatTensor of shape (batch_size, feature size), optional) —
Static features of each time series’ in a batch which are copied to the covariates at inference time.    Base class for time series model’s decoder outputs that also contain the loss as well as the parameters of the
chosen distribution


SampleTSPredictionOutput

class transformers.modeling_outputs.SampleTSPredictionOutput  < source > ( sequences: FloatTensor = None  )   Parameters   sequences (torch.FloatTensor of shape (batch_size, num_samples, prediction_length) or (batch_size, num_samples, prediction_length, input_size)) —
Sampled values from the chosen distribution.    Base class for time series model’s predictions outputs that contains the sampled values from the chosen
distribution.


TFBaseModelOutput

class transformers.modeling_tf_outputs.TFBaseModelOutput  < source > ( last_hidden_state: tf.Tensor = None hidden_states: Tuple[tf.Tensor] | None = None attentions: Tuple[tf.Tensor] | None = None  )   Parameters   last_hidden_state (tf.Tensor of shape (batch_size, sequence_length, hidden_size)) —
Sequence of hidden-states at the output of the last layer of the model.   hidden_states (tuple(tf.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of tf.Tensor (one for the output of the embeddings + one for the output


TFBaseModelOutput

 of each layer) of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the initial embedding outputs.   attentions (tuple(tf.Tensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of tf.Tensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for model’s outputs, with potential hidden states and attentions.


TFBaseModelOutputWithPooling

class transformers.modeling_tf_outputs.TFBaseModelOutputWithPooling  < source > ( last_hidden_state: tf.Tensor = None pooler_output: tf.Tensor = None hidden_states: Tuple[tf.Tensor] | None = None attentions: Tuple[tf.Tensor] | None = None  )   Parameters   last_hidden_state (tf.Tensor of shape (batch_size, sequence_length, hidden_size)) —
Sequence of hidden-states at the output of the last layer of the model.   pooler_output (tf.Tensor of shape (batch_size, hidden_size)) —
Last layer hidden-state of the first token of the sequence (classification token) further processed by a
Linear layer and a Tanh activation function. The Linear layer weights are trained from the next sentence
prediction (classification) objective during pretraining.
This output is usually not a good summary of the semantic content of the input, you’re often better with
averaging or pooling th


TFBaseModelOutputWithPooling

e sequence of hidden-states for the whole input sequence.   hidden_states (tuple(tf.Tensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of tf.Tensor (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the initial embedding outputs.   attentions (tuple(tf.Tensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of tf.Tensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for model’s outputs that also contains a pooling of the last hidden states


class transformers.modeling_tf_outputs.TFBaseModelOutputWithPoolingAndCrossAttentions  < source > ( last_hidden_state: tf.Tensor = None pooler_output: tf.Tensor = None past_key_values: List[tf.Tensor] | None = None hidden_states: Tuple[tf.Tensor] | None = None attentions: Tuple[tf.Tensor] | None = None cross_attentions: Tuple[tf.Tensor] | None = None  )   Parameters   last_hidden_state (tf.Tensor of shape (batch_size, sequence_length, hidden_size)) —
Sequence of hidden-states at the output of the last layer of the model.   pooler_output (tf.Tensor of shape (batch_size, hidden_size)) —
Last layer hidden-state of the first token of the sequence (classification token) further processed by a
Linear layer and a Tanh activation function. The Linear layer weights are trained from the next sentence
prediction (classification) objective during pretraining.
This output is usually not a go


od summary of the semantic content of the input, you’re often better with
averaging or pooling the sequence of hidden-states for the whole input sequence.   past_key_values (List[tf.Tensor], optional, returned when use_cache=True is passed or when config.use_cache=True) —
List of tf.Tensor of length config.n_layers, with each tensor of shape (2, batch_size, num_heads, sequence_length, embed_size_per_head)).
Contains pre-computed hidden-states (key and values in the attention blocks) that can be used (see
past_key_values input) to speed up sequential decoding.   hidden_states (tuple(tf.Tensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of tf.Tensor (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output


 of each layer plus the initial embedding outputs.   attentions (tuple(tf.Tensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of tf.Tensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.   cross_attentions (tuple(tf.Tensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of tf.Tensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the decoder’s cross-attention layer, after the attention softmax, used to compute the
weighted average in the cross-attention heads.    Base class for model’s outputs that also contains a pooling of the last hidden states.


TFBaseModelOutputWithPast

class transformers.modeling_tf_outputs.TFBaseModelOutputWithPast  < source > ( last_hidden_state: tf.Tensor = None past_key_values: List[tf.Tensor] | None = None hidden_states: Tuple[tf.Tensor] | None = None attentions: Tuple[tf.Tensor] | None = None  )   Parameters   last_hidden_state (tf.Tensor of shape (batch_size, sequence_length, hidden_size)) —
Sequence of hidden-states at the output of the last layer of the model.
If past_key_values is used only the last hidden-state of the sequences of shape (batch_size, 1, hidden_size) is output.   past_key_values (List[tf.Tensor], optional, returned when use_cache=True is passed or when config.use_cache=True) —
List of tf.Tensor of length config.n_layers, with each tensor of shape (2, batch_size, num_heads, sequence_length, embed_size_per_head)).
Contains pre-computed hidden-states (key and values in the attention blocks) that can be used (


TFBaseModelOutputWithPast

see
past_key_values input) to speed up sequential decoding.   hidden_states (tuple(tf.Tensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of tf.Tensor (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the initial embedding outputs.   attentions (tuple(tf.Tensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of tf.Tensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for model’s outputs that may also contain a past key/values (to speed up sequential decoding).


class transformers.modeling_tf_outputs.TFBaseModelOutputWithPastAndCrossAttentions  < source > ( last_hidden_state: tf.Tensor = None past_key_values: List[tf.Tensor] | None = None hidden_states: Tuple[tf.Tensor] | None = None attentions: Tuple[tf.Tensor] | None = None cross_attentions: Tuple[tf.Tensor] | None = None  )   Parameters   last_hidden_state (tf.Tensor of shape (batch_size, sequence_length, hidden_size)) —
Sequence of hidden-states at the output of the last layer of the model.
If past_key_values is used only the last hidden-state of the sequences of shape (batch_size, 1, hidden_size) is output.   past_key_values (List[tf.Tensor], optional, returned when use_cache=True is passed or when config.use_cache=True) —
List of tf.Tensor of len


gth config.n_layers, with each tensor of shape (2, batch_size, num_heads, sequence_length, embed_size_per_head)).
Contains pre-computed hidden-states (key and values in the attention blocks) that can be used (see
past_key_values input) to speed up sequential decoding.   hidden_states (tuple(tf.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of tf.Tensor (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the initial embedding outputs.   attentions (tuple(tf.Tensor), optional, returned when output_attentions=True is passed or when config.


output_attentions=True) —
Tuple of tf.Tensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.   cross_attentions (tuple(tf.Tensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of tf.Tensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the decoder’s cross-attention layer, after the attention softmax, used to compute the
weighted average in the cross-attention heads.    Base class for model’s outputs that may also contain a past key/values (to speed up sequential decoding)


class transformers.modeling_tf_outputs.TFSeq2SeqModelOutput  < source > ( last_hidden_state: tf.Tensor = None past_key_values: List[tf.Tensor] | None = None decoder_hidden_states: Tuple[tf.Tensor] | None = None decoder_attentions: Tuple[tf.Tensor] | None = None cross_attentions: Tuple[tf.Tensor] | None = None encoder_last_hidden_state: tf.Tensor | None = None encoder_hidden_states: Tuple[tf.Tensor] | None = None encoder_attentions: Tuple[tf.Tensor] | None = None  )   Parameters   last_hidden_state (tf.Tensor of shape (batch_size, sequence_length, hidden_size)) —
Sequence of hidden-states at the output of the last layer of the decoder of the model.
If past_key_values is used only the last hidden-state of the sequences of shape (batch_size, 1, hidden_size) is output.   past_key_values (List[tf.Tensor], optional, returned when use_cache=True is passed


or when config.use_cache=True) —
List of tf.Tensor of length config.n_layers, with each tensor of shape (2, batch_size, num_heads, sequence_length, embed_size_per_head)).
Contains pre-computed hidden-states (key and values in the attention blocks) of the decoder that can be
used (see past_key_values input) to speed up sequential decoding.   decoder_hidden_states (tuple(tf.Tensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of tf.Tensor (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the decoder at the output of each layer plus the initial embedding outputs.   decoder_attentions (tuple(tf.Tensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True


) —
Tuple of tf.Tensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the decoder, after the attention softmax, used to compute the weighted average in the
self-attention heads.   cross_attentions (tuple(tf.Tensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of tf.Tensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the decoder’s cross-attention layer, after the attention softmax, used to compute the
weighted average in the cross-attention heads.   encoder_last_hidden_state (tf.Tensor of shape (batch_size, sequence_length, hidden_size), optional) —
Sequence of hidden-states at the output of the last layer of the encoder of the model.   encoder_hidden_states (tupl


e(tf.Tensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of tf.Tensor (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the encoder at the output of each layer plus the initial embedding outputs.   encoder_attentions (tuple(tf.Tensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of tf.Tensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the encoder, after the attention softmax, used to compute the weighted average in the
self-attention heads.    Base class for model encoder’s outputs that also contains : pre-computed hidden states that can speed up sequential
decoding.


TFCausalLMOutput

class transformers.modeling_tf_outputs.TFCausalLMOutput  < source > ( loss: tf.Tensor | None = None logits: tf.Tensor = None hidden_states: Tuple[tf.Tensor] | None = None attentions: Tuple[tf.Tensor] | None = None  )   Parameters   loss (tf.Tensor of shape (n,), optional, where n is the number of non-masked labels, returned when labels is provided) —
Language modeling loss (for next-token prediction).   logits (tf.Tensor of shape (batch_size, sequence_length, config.vocab_size)) —
Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax).   hidden_states (tuple(tf.Tensor), optional, returned when output_hidden_states=True is passed or when co


TFCausalLMOutput

nfig.output_hidden_states=True) —
Tuple of tf.Tensor (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the initial embedding outputs.   attentions (tuple(tf.Tensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of tf.Tensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for causal language model (or autoregressive) outputs


TFCausalLMOutputWithCrossAttentions

class transformers.modeling_tf_outputs.TFCausalLMOutputWithCrossAttentions  < source > ( loss: tf.Tensor | None = None logits: tf.Tensor = None past_key_values: List[tf.Tensor] | None = None hidden_states: Tuple[tf.Tensor] | None = None attentions: Tuple[tf.Tensor] | None = None cross_attentions: Tuple[tf.Tensor] | None = None  )   Parameters   loss (tf.Tensor of shape (n,), optional, where n is the number of non-masked labels, returned when labels is provided) —
Language modeling loss (for next-token prediction).   logits (tf.Tensor of shape (batch_size, sequence_length, config.vocab_size)) —
Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax).   hidden_states (tuple(tf.Tensor), optional, returned when output_hi


TFCausalLMOutputWithCrossAttentions

dden_states=True is passed or when config.output_hidden_states=True) —
Tuple of tf.Tensor (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the initial embedding outputs.   attentions (tuple(tf.Tensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of tf.Tensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.   cross_attentions (tuple(tf.Tensor), optional, returned when output_attentions=True is passed or when confi


TFCausalLMOutputWithCrossAttentions

g.output_attentions=True) —
Tuple of tf.Tensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the decoder’s cross-attention layer, after the attention softmax, used to compute the
weighted average in the cross-attention heads.   past_key_values (List[tf.Tensor], optional, returned when use_cache=True is passed or when config.use_cache=True) —
List of tf.Tensor of length config.n_layers, with each tensor of shape (2, batch_size, num_heads, sequence_length, embed_size_per_head)).
Contains pre-computed hidden-states (key and values in the attention blocks) that can be used (see
past_key_values input) to speed up sequential decoding.    Base class for causal language model (or autoregressive) output


TFCausalLMOutputWithPast

class transformers.modeling_tf_outputs.TFCausalLMOutputWithPast  < source > ( loss: tf.Tensor | None = None logits: tf.Tensor = None past_key_values: List[tf.Tensor] | None = None hidden_states: Tuple[tf.Tensor] | None = None attentions: Tuple[tf.Tensor] | None = None  )   Parameters   loss (tf.Tensor of shape (n,), optional, where n is the number of non-masked labels, returned when labels is provided) —
Language modeling loss (for next-token prediction).   logits (tf.Tensor of shape (batch_size, sequence_length, config.vocab_size)) —
Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax).   past_key_values (List[tf.Tensor], optional, returned when use_cache=True is passed or when config.use_cache=True) —
List of tf.Tensor of length config.n_layers, with each tensor of shape (2, batch_size, num_heads, sequence_length, embed_size_per_head)).
Contains pre-computed hidde


TFCausalLMOutputWithPast

n-states (key and values in the attention blocks) that can be used (see
past_key_values input) to speed up sequential decoding.   hidden_states (tuple(tf.Tensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of tf.Tensor (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the initial embedding outputs.   attentions (tuple(tf.Tensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of tf.Tensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for causal language model (or autoregressive) outputs


TFMaskedLMOutput

class transformers.modeling_tf_outputs.TFMaskedLMOutput  < source > ( loss: tf.Tensor | None = None logits: tf.Tensor = None hidden_states: Tuple[tf.Tensor] | None = None attentions: Tuple[tf.Tensor] | None = None  )   Parameters   loss (tf.Tensor of shape (n,), optional, where n is the number of non-masked labels, returned when labels is provided) —
Masked language modeling (MLM) loss.   logits (tf.Tensor of shape (batch_size, sequence_length, config.vocab_size)) —
Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax).   hidden_states (tuple(tf.Tensor), optional, returned when output_hidden_states=True is passed or when


TFMaskedLMOutput

config.output_hidden_states=True) —
Tuple of tf.Tensor (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the initial embedding outputs.   attentions (tuple(tf.Tensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of tf.Tensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for masked language models outputs


TFSeq2SeqLMOutput

class transformers.modeling_tf_outputs.TFSeq2SeqLMOutput  < source > ( loss: tf.Tensor | None = None logits: tf.Tensor = None past_key_values: List[tf.Tensor] | None = None decoder_hidden_states: Tuple[tf.Tensor] | None = None decoder_attentions: Tuple[tf.Tensor] | None = None cross_attentions: Tuple[tf.Tensor] | None = None encoder_last_hidden_state: tf.Tensor | None = None encoder_hidden_states: Tuple[tf.Tensor] | None = None encoder_attentions: Tuple[tf.Tensor] | None = None  )   Parameters   loss (tf.Tensor of shape (n,), optional, where n is the number of non-masked labels, returned when labels is provided) —
Language modeling loss.   logits (tf.Tensor of shape (batch_size, sequence_length, config.vocab_size)) —
Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax).   past_key_values (List[tf.Tens


TFSeq2SeqLMOutput

or], optional, returned when use_cache=True is passed or when config.use_cache=True) —
List of tf.Tensor of length config.n_layers, with each tensor of shape (2, batch_size, num_heads, sequence_length, embed_size_per_head)).
Contains pre-computed hidden-states (key and values in the attention blocks) of the decoder that can be
used (see past_key_values input) to speed up sequential decoding.   decoder_hidden_states (tuple(tf.Tensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of tf.Tensor (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the decoder at the output of each layer plus the initial embedding outputs.   decoder_attentions (tuple(tf.Tensor), optional, returned when output_atten


TFSeq2SeqLMOutput

tions=True is passed or when config.output_attentions=True) —
Tuple of tf.Tensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the decoder, after the attention softmax, used to compute the weighted average in the
self-attention heads.   cross_attentions (tuple(tf.Tensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of tf.Tensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the decoder’s cross-attention layer, after the attention softmax, used to compute the
weighted average in the cross-attention heads.   encoder_last_hidden_state (tf.Tensor of shape (batch_size, sequence_length, hidden_size), optional) —
Sequence of hidden-states at the output of the last la


TFSeq2SeqLMOutput

yer of the encoder of the model.   encoder_hidden_states (tuple(tf.Tensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of tf.Tensor (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the encoder at the output of each layer plus the initial embedding outputs.   encoder_attentions (tuple(tf.Tensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of tf.Tensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the encoder, after the attention softmax, used to compute the weighted average in the
self-attention heads.    Base class for sequence-to-sequence language models outputs


TFNextSentencePredictorOutput

class transformers.modeling_tf_outputs.TFNextSentencePredictorOutput  < source > ( loss: tf.Tensor | None = None logits: tf.Tensor = None hidden_states: Tuple[tf.Tensor] | None = None attentions: Tuple[tf.Tensor] | None = None  )   Parameters   loss (tf.Tensor of shape (n,), optional, where n is the number of non-masked labels, returned when next_sentence_label is provided) —
Next sentence prediction loss.   logits (tf.Tensor of shape (batch_size, 2)) —
Prediction scores of the next sequence prediction (classification) head (scores of True/False continuation
before SoftMax).   hidden_states (tuple(tf.Tensor), optional, returned when output_hidden_states=True is passed or when config.output


TFNextSentencePredictorOutput

_hidden_states=True) —
Tuple of tf.Tensor (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the initial embedding outputs.   attentions (tuple(tf.Tensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of tf.Tensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for outputs of models predicting if two sentences are consecutive or not


TFSequenceClassifierOutput

class transformers.modeling_tf_outputs.TFSequenceClassifierOutput  < source > ( loss: tf.Tensor | None = None logits: tf.Tensor = None hidden_states: Tuple[tf.Tensor] | None = None attentions: Tuple[tf.Tensor] | None = None  )   Parameters   loss (tf.Tensor of shape (batch_size, ), optional, returned when labels is provided) —
Classification (or regression if config.num_labels==1) loss.   logits (tf.Tensor of shape (batch_size, config.num_labels)) —
Classification (or regression if config.num_labels==1) scores (before SoftMax).   hidden_states (tuple(tf.Tensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_sta


TFSequenceClassifierOutput

tes=True) —
Tuple of tf.Tensor (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the initial embedding outputs.   attentions (tuple(tf.Tensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of tf.Tensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for outputs of sentence classification models.


class transformers.modeling_tf_outputs.TFSeq2SeqSequenceClassifierOutput  < source > ( loss: tf.Tensor | None = None logits: tf.Tensor = None past_key_values: List[tf.Tensor] | None = None decoder_hidden_states: Tuple[tf.Tensor] | None = None decoder_attentions: Tuple[tf.Tensor] | None = None cross_attentions: Tuple[tf.Tensor] | None = None encoder_last_hidden_state: tf.Tensor | None = None encoder_hidden_states: Tuple[tf.Tensor] | None = None encoder_attentions: Tuple[tf.Tensor] | None = None  )   Parameters   loss (tf.Tensor of shape (1,), optional, returned when label is provided) —
Classification (or regression if config.num_labels==1) loss.   logits (tf.Tensor of shape (batch_size, config.num_labels)) —
Classification (or regression if config.num_labels==1) scores (before SoftMax).   past_key_values


 (List[tf.Tensor], optional, returned when use_cache=True is passed or when config.use_cache=True) —
List of tf.Tensor of length config.n_layers, with each tensor of shape (2, batch_size, num_heads, sequence_length, embed_size_per_head)).
Contains pre-computed hidden-states (key and values in the attention blocks) of the decoder that can be
used (see past_key_values input) to speed up sequential decoding.   decoder_hidden_states (tuple(tf.Tensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of tf.Tensor (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the decoder at the output of each layer plus the initial embedding outputs.   decoder_attentions


 (tuple(tf.Tensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of tf.Tensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the decoder, after the attention softmax, used to compute the weighted average in the
self-attention heads.   cross_attentions (tuple(tf.Tensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of tf.Tensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length)   encoder_last_hidden_state (tf.Tensor of shape (batch_size, sequence_length, hidden_size), optional) —
Sequence of hidden-states at the output of the last layer of the encoder of the model.   encoder_hidden_states (


tuple(tf.Tensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of tf.Tensor (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the encoder at the output of each layer plus the initial embedding outputs.   encoder_attentions (tuple(tf.Tensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of tf.Tensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the encoder, after the attention softmax, used to compute the weighted average in the
self-attention heads.    Base class for outputs of sequence-to-sequence sentence classification mode


TFMultipleChoiceModelOutput

class transformers.modeling_tf_outputs.TFMultipleChoiceModelOutput  < source > ( loss: tf.Tensor | None = None logits: tf.Tensor = None hidden_states: Tuple[tf.Tensor] | None = None attentions: Tuple[tf.Tensor] | None = None  )   Parameters   loss (tf.Tensor of shape (batch_size, ), optional, returned when labels is provided) —
Classification loss.   logits (tf.Tensor of shape (batch_size, num_choices)) —
num_choices is the second dimension of the input tensors. (see input_ids above).
Classification scores (before SoftMax).   hidden_states (tuple(tf.Tensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_s


TFMultipleChoiceModelOutput

tates=True) —
Tuple of tf.Tensor (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the initial embedding outputs.   attentions (tuple(tf.Tensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of tf.Tensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for outputs of multiple choice models.


TFTokenClassifierOutput

class transformers.modeling_tf_outputs.TFTokenClassifierOutput  < source > ( loss: tf.Tensor | None = None logits: tf.Tensor = None hidden_states: Tuple[tf.Tensor] | None = None attentions: Tuple[tf.Tensor] | None = None  )   Parameters   loss (tf.Tensor of shape (n,), optional, where n is the number of unmasked labels, returned when labels is provided)  —
Classification loss.   logits (tf.Tensor of shape (batch_size, sequence_length, config.num_labels)) —
Classification scores (before SoftMax).   hidden_states (tuple(tf.Tensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tup


TFTokenClassifierOutput

le of tf.Tensor (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the initial embedding outputs.   attentions (tuple(tf.Tensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of tf.Tensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for outputs of token classification models.


TFQuestionAnsweringModelOutput

class transformers.modeling_tf_outputs.TFQuestionAnsweringModelOutput  < source > ( loss: tf.Tensor | None = None start_logits: tf.Tensor = None end_logits: tf.Tensor = None hidden_states: Tuple[tf.Tensor] | None = None attentions: Tuple[tf.Tensor] | None = None  )   Parameters   loss (tf.Tensor of shape (batch_size, ), optional, returned when start_positions and end_positions are provided) —
Total span extraction loss is the sum of a Cross-Entropy for the start and end positions.   start_logits (tf.Tensor of shape (batch_size, sequence_length)) —
Span-start scores (before SoftMax).   end_logits (tf.Tensor of shape (batch_size, sequence_length)) —
Span-end scores (before SoftMax).   hidden_states (tuple(tf.Tensor), optional, r


TFQuestionAnsweringModelOutput

eturned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of tf.Tensor (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the initial embedding outputs.   attentions (tuple(tf.Tensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of tf.Tensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for outputs of question answering models


class transformers.modeling_tf_outputs.TFSeq2SeqQuestionAnsweringModelOutput  < source > ( loss: tf.Tensor | None = None start_logits: tf.Tensor = None end_logits: tf.Tensor = None past_key_values: List[tf.Tensor] | None = None decoder_hidden_states: Tuple[tf.Tensor] | None = None decoder_attentions: Tuple[tf.Tensor] | None = None encoder_last_hidden_state: tf.Tensor | None = None encoder_hidden_states: Tuple[tf.Tensor] | None = None encoder_attentions: Tuple[tf.Tensor] | None = None  )   Parameters   loss (tf.Tensor of shape (1,), optional, returned when labels is provided) —
Total span extraction loss is the sum of a Cross-Entropy for the start and end positions.   start_logits (tf.Tensor of shape (batch_size, sequence_length)) —
Span-start scores (before SoftM


ax).   end_logits (tf.Tensor of shape (batch_size, sequence_length)) —
Span-end scores (before SoftMax).   past_key_values (List[tf.Tensor], optional, returned when use_cache=True is passed or when config.use_cache=True) —
List of tf.Tensor of length config.n_layers, with each tensor of shape (2, batch_size, num_heads, sequence_length, embed_size_per_head)).
Contains pre-computed hidden-states (key and values in the attention blocks) of the decoder that can be
used (see past_key_values input) to speed up sequential decoding.   decoder_hidden_states (tuple(tf.Tensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of tf.Tensor (one for the output of the embeddings + one for the output of each layer) of


 shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the decoder at the output of each layer plus the initial embedding outputs.   decoder_attentions (tuple(tf.Tensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of tf.Tensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the decoder, after the attention softmax, used to compute the weighted average in the
self-attention heads.   encoder_last_hidden_state (tf.Tensor of shape (batch_size, sequence_length, hidden_size), optional) —
Sequence of hidden-states at the output of the last layer of the encoder of the model.   encoder_hidden_states (tuple(tf.Tensor), optional, returned


when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of tf.Tensor (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the encoder at the output of each layer plus the initial embedding outputs.   encoder_attentions (tuple(tf.Tensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of tf.Tensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the encoder, after the attention softmax, used to compute the weighted average in the
self-attention heads.    Base class for outputs of sequence-to-sequence question answering mode


FlaxBaseModelOutput

class transformers.modeling_flax_outputs.FlaxBaseModelOutput  < source > ( last_hidden_state: Array = None hidden_states: typing.Optional[typing.Tuple[jax.Array]] = None attentions: typing.Optional[typing.Tuple[jax.Array]] = None  )   Parameters   last_hidden_state (jnp.ndarray of shape (batch_size, sequence_length, hidden_size)) —
Sequence of hidden-states at the output of the last layer of the model.   hidden_states (tuple(jnp.ndarray), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of jnp.ndarray (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, s


FlaxBaseModelOutput

equence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the initial embedding outputs.   attentions (tuple(jnp.ndarray), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of jnp.ndarray (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for model’s outputs, with potential hidden states and attentions.  replace  < source > ( **updates  )    “Returns a new object replacing the specified fields with new values.


FlaxBaseModelOutputWithPast

class transformers.modeling_flax_outputs.FlaxBaseModelOutputWithPast  < source > ( last_hidden_state: Array = None past_key_values: typing.Union[typing.Dict[str, jax.Array], NoneType] = None hidden_states: typing.Optional[typing.Tuple[jax.Array]] = None attentions: typing.Optional[typing.Tuple[jax.Array]] = None  )   Parameters   last_hidden_state (jnp.ndarray of shape (batch_size, sequence_length, hidden_size)) —
Sequence of hidden-states at the output of the last layer of the model.   past_key_values (Dict[str, jnp.ndarray]) —
Dictionary of pre-computed hidden-states (key and values in the attention blocks) that can be used for fast
auto-regressive decoding. Pre-computed key and value hidden-states are of shape [batch_size, max_length].   hidden_states (tuple(jnp.ndarray), optional, returned when output_hidden_states=True


FlaxBaseModelOutputWithPast

is passed or when config.output_hidden_states=True) —
Tuple of jnp.ndarray (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the initial embedding outputs.   attentions (tuple(jnp.ndarray), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of jnp.ndarray (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for model’s outputs, with potential hidden states and attentions.  replace  < source > ( **updates  )    “Returns a new object replacing the specified fields with new values


FlaxBaseModelOutputWithPooling

class transformers.modeling_flax_outputs.FlaxBaseModelOutputWithPooling  < source > ( last_hidden_state: Array = None pooler_output: Array = None hidden_states: typing.Optional[typing.Tuple[jax.Array]] = None attentions: typing.Optional[typing.Tuple[jax.Array]] = None  )   Parameters   last_hidden_state (jnp.ndarray of shape (batch_size, sequence_length, hidden_size)) —
Sequence of hidden-states at the output of the last layer of the model.   pooler_output (jnp.ndarray of shape (batch_size, hidden_size)) —
Last layer hidden-state of the first token of the sequence (classification token) further processed by a
Linear layer and a Tanh activation function. The Linear layer weights are trained from the next sentence
prediction (classification) objective during pretraining.   hidden_states (tuple(jnp.ndarray), optional, returned when output_hidden_st


FlaxBaseModelOutputWithPooling

ates=True is passed or when config.output_hidden_states=True) —
Tuple of jnp.ndarray (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the initial embedding outputs.   attentions (tuple(jnp.ndarray), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of jnp.ndarray (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for model’s outputs that also contains a pooling of the last hidden states.  replace  < source > ( **updates  )    “Returns a new object replacing the specified fields with new values.


class transformers.modeling_flax_outputs.FlaxBaseModelOutputWithPastAndCrossAttentions  < source > ( last_hidden_state: Array = None past_key_values: typing.Optional[typing.Tuple[typing.Tuple[jax.Array]]] = None hidden_states: typing.Optional[typing.Tuple[jax.Array]] = None attentions: typing.Optional[typing.Tuple[jax.Array]] = None cross_attentions: typing.Optional[typing.Tuple[jax.Array]] = None  )   Parameters   last_hidden_state (jnp.ndarray of shape (batch_size, sequence_length, hidden_size)) —
Sequence of hidden-states at the output of the last layer of the model.
If past_key_values is used only the last hidden-state of the sequences of shape (batch_size, 1, hidden_size) is output.   past_key_values (tuple(tuple(jnp.ndarray)), optional, returned when use_cache=True is passed or when config.use_cache=True) —
Tuple of tuple(jnp.ndarray) of length config.n_layers, with each tuple having 2 tensors of shape


(batch_size, num_heads, sequence_length, embed_size_per_head)) and optionally if
config.is_encoder_decoder=True 2 additional tensors of shape (batch_size, num_heads, encoder_sequence_length, embed_size_per_head).
Contains pre-computed hidden-states (key and values in the self-attention blocks and optionally if
config.is_encoder_decoder=True in the cross-attention blocks) that can be used (see past_key_values
input) to speed up sequential decoding.   hidden_states (tuple(jnp.ndarray), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of jnp.ndarray (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the initial embedding outputs.   attentions (tuple(jnp.ndarray), optional, returned when output_attentions=True is passed


 or when config.output_attentions=True) —
Tuple of jnp.ndarray (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.   cross_attentions (tuple(jnp.ndarray), optional, returned when output_attentions=True and config.add_cross_attention=True is passed or when config.output_attentions=True) —
Tuple of jnp.ndarray (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the decoder’s cross-attention layer, after the attention softmax, used to compute the
weighted average in the cross-attention heads.    Base class for model’s outputs that may also contain a past key/values (to speed up sequential decoding).  replace  < source > ( **updates  )    “Returns a new object replacing the specified fields with new values.


class transformers.modeling_flax_outputs.FlaxSeq2SeqModelOutput  < source > ( last_hidden_state: Array = None past_key_values: typing.Optional[typing.Tuple[typing.Tuple[jax.Array]]] = None decoder_hidden_states: typing.Optional[typing.Tuple[jax.Array]] = None decoder_attentions: typing.Optional[typing.Tuple[jax.Array]] = None cross_attentions: typing.Optional[typing.Tuple[jax.Array]] = None encoder_last_hidden_state: typing.Optional[jax.Array] = None encoder_hidden_states: typing.Optional[typing.Tuple[jax.Array]] = None encoder_attentions: typing.Optional[typing.Tuple[jax.Array]] = None  )   Parameters   last_hidden_state (jnp.ndarray of shape (batch_size, sequence_length, hidden_size)) —
Sequence of hidden-states at the output of the last layer of the decoder of the model.
If past_key_values is used only the last hidden-state of the sequences of shape (batch_size, 1, hidden_size) is output.   past_key_values (tuple(tuple(jnp.ndarray)), optional, retur


ned when use_cache=True is passed or when config.use_cache=True) —
Tuple of tuple(jnp.ndarray) of length config.n_layers, with each tuple having 2 tensors of shape
(batch_size, num_heads, sequence_length, embed_size_per_head)) and 2 additional tensors of shape
(batch_size, num_heads, encoder_sequence_length, embed_size_per_head).
Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention
blocks) that can be used (see past_key_values input) to speed up sequential decoding.   decoder_hidden_states (tuple(jnp.ndarray), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of jnp.ndarray (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the decoder at the output of each layer plus the initial embedding outputs.   decoder_attentions (tuple(jnp.ndarray), optional


, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of jnp.ndarray (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the decoder, after the attention softmax, used to compute the weighted average in the
self-attention heads.   cross_attentions (tuple(jnp.ndarray), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of jnp.ndarray (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the decoder’s cross-attention layer, after the attention softmax, used to compute the
weighted average in the cross-attention heads.   encoder_last_hidden_state (jnp.ndarray of shape (batch_size, sequence_length, hidden_size), optional) —
Sequence of hidden-states at the output of the last layer of the encoder of the model.   encoder_hidden_states (tuple(jnp.ndarr


ay), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of jnp.ndarray (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the encoder at the output of each layer plus the initial embedding outputs.   encoder_attentions (tuple(jnp.ndarray), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of jnp.ndarray (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the encoder, after the attention softmax, used to compute the weighted average in the
self-attention heads.    Base class for model encoder’s outputs that also contains : pre-computed hidden states that can speed up sequential
decoding.  replace  < source > ( **updates  )    “Returns a new object replacing the specified fields with new values


FlaxCausalLMOutputWithCrossAttentions

class transformers.modeling_flax_outputs.FlaxCausalLMOutputWithCrossAttentions  < source > ( logits: Array = None past_key_values: typing.Optional[typing.Tuple[typing.Tuple[jax.Array]]] = None hidden_states: typing.Optional[typing.Tuple[jax.Array]] = None attentions: typing.Optional[typing.Tuple[jax.Array]] = None cross_attentions: typing.Optional[typing.Tuple[jax.Array]] = None  )   Parameters   logits (jnp.ndarray of shape (batch_size, sequence_length, config.vocab_size)) —
Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax).   hidden_states (tuple(jnp.ndarray), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of jnp.ndarray (one for the output of the embeddings + one for the outpu


FlaxCausalLMOutputWithCrossAttentions

t of each layer) of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the initial embedding outputs.   attentions (tuple(jnp.ndarray), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of jnp.ndarray (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.   cross_attentions (tuple(jnp.ndarray), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of jnp.ndarray (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Cross attentions weights after the


FlaxCausalLMOutputWithCrossAttentions

 attention softmax, used to compute the weighted average in the
cross-attention heads.   past_key_values (tuple(tuple(jnp.ndarray)), optional, returned when use_cache=True is passed or when config.use_cache=True) —
Tuple of jnp.ndarray tuples of length config.n_layers, with each tuple containing the cached key, value
states of the self-attention and the cross-attention layers if model is used in encoder-decoder setting.
Only relevant if config.is_decoder = True.
Contains pre-computed hidden-states (key and values in the attention blocks) that can be used (see
past_key_values input) to speed up sequential decoding.    Base class for causal language model (or autoregressive) outputs.  replace  < source > ( **updates  )    “Returns a new object replacing the specified fields with new values.


FlaxMaskedLMOutput

class transformers.modeling_flax_outputs.FlaxMaskedLMOutput  < source > ( logits: Array = None hidden_states: typing.Optional[typing.Tuple[jax.Array]] = None attentions: typing.Optional[typing.Tuple[jax.Array]] = None  )   Parameters   logits (jnp.ndarray of shape (batch_size, sequence_length, config.vocab_size)) —
Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax).   hidden_states (tuple(jnp.ndarray), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of jnp.ndarray (one for the output of the embeddings + one for the output of each layer) of


FlaxMaskedLMOutput

shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the initial embedding outputs.   attentions (tuple(jnp.ndarray), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of jnp.ndarray (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for masked language models outputs.  replace  < source > ( **updates  )    “Returns a new object replacing the specified fields with new values.


FlaxSeq2SeqLMOutput

class transformers.modeling_flax_outputs.FlaxSeq2SeqLMOutput  < source > ( logits: Array = None past_key_values: typing.Optional[typing.Tuple[typing.Tuple[jax.Array]]] = None decoder_hidden_states: typing.Optional[typing.Tuple[jax.Array]] = None decoder_attentions: typing.Optional[typing.Tuple[jax.Array]] = None cross_attentions: typing.Optional[typing.Tuple[jax.Array]] = None encoder_last_hidden_state: typing.Optional[jax.Array] = None encoder_hidden_states: typing.Optional[typing.Tuple[jax.Array]] = None encoder_attentions: typing.Optional[typing.Tuple[jax.Array]] = None  )   Parameters   logits (jnp.ndarray of shape (batch_size, sequence_length, config.vocab_size)) —
Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax).   past_key_values (tuple(tuple(jnp.ndarray)), optional, returned when use_cache=True is passed or when config.use_cache=True) —
Tuple of tup


FlaxSeq2SeqLMOutput

le(jnp.ndarray) of length config.n_layers, with each tuple having 2 tensors of shape
(batch_size, num_heads, sequence_length, embed_size_per_head)) and 2 additional tensors of shape
(batch_size, num_heads, encoder_sequence_length, embed_size_per_head).
Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention
blocks) that can be used (see past_key_values input) to speed up sequential decoding.   decoder_hidden_states (tuple(jnp.ndarray), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of jnp.ndarray (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the decoder at the output of each layer plus the initial embedding outputs.   decoder_attentions (tuple(jnp.ndarray), optional, returned when output_attentio


FlaxSeq2SeqLMOutput

ns=True is passed or when config.output_attentions=True) —
Tuple of jnp.ndarray (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the decoder, after the attention softmax, used to compute the weighted average in the
self-attention heads.   cross_attentions (tuple(jnp.ndarray), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of jnp.ndarray (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the decoder’s cross-attention layer, after the attention softmax, used to compute the
weighted average in the cross-attention heads.   encoder_last_hidden_state (jnp.ndarray of shape (batch_size, sequence_length, hidden_size), optional) —
Sequence of hidden-states at the output of the last layer of the encoder of the model.   encoder_hidden_states


FlaxSeq2SeqLMOutput

 (tuple(jnp.ndarray), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of jnp.ndarray (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the encoder at the output of each layer plus the initial embedding outputs.   encoder_attentions (tuple(jnp.ndarray), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of jnp.ndarray (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the encoder, after the attention softmax, used to compute the weighted average in the
self-attention heads.    Base class for sequence-to-sequence language models outputs.  replace  < source > ( **updates  )    “Returns a new object replacing the specified fields with new values


FlaxNextSentencePredictorOutput

class transformers.modeling_flax_outputs.FlaxNextSentencePredictorOutput  < source > ( logits: Array = None hidden_states: typing.Optional[typing.Tuple[jax.Array]] = None attentions: typing.Optional[typing.Tuple[jax.Array]] = None  )   Parameters   logits (jnp.ndarray of shape (batch_size, 2)) —
Prediction scores of the next sequence prediction (classification) head (scores of True/False continuation
before SoftMax).   hidden_states (tuple(jnp.ndarray), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of jnp.ndarray (one for the output of the embeddings + one for the output of each layer) of shape
(batch_siz


FlaxNextSentencePredictorOutput

e, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the initial embedding outputs.   attentions (tuple(jnp.ndarray), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of jnp.ndarray (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for outputs of models predicting if two sentences are consecutive or not.  replace  < source > ( **updates  )    “Returns a new object replacing the specified fields with new values


FlaxSequenceClassifierOutput

class transformers.modeling_flax_outputs.FlaxSequenceClassifierOutput  < source > ( logits: Array = None hidden_states: typing.Optional[typing.Tuple[jax.Array]] = None attentions: typing.Optional[typing.Tuple[jax.Array]] = None  )   Parameters   logits (jnp.ndarray of shape (batch_size, config.num_labels)) —
Classification (or regression if config.num_labels==1) scores (before SoftMax).   hidden_states (tuple(jnp.ndarray), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of jnp.ndarray (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size,


FlaxSequenceClassifierOutput

 sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the initial embedding outputs.   attentions (tuple(jnp.ndarray), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of jnp.ndarray (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for outputs of sentence classification models.  replace  < source > ( **updates  )    “Returns a new object replacing the specified fields with new values


class transformers.modeling_flax_outputs.FlaxSeq2SeqSequenceClassifierOutput  < source > ( logits: Array = None past_key_values: typing.Optional[typing.Tuple[typing.Tuple[jax.Array]]] = None decoder_hidden_states: typing.Optional[typing.Tuple[jax.Array]] = None decoder_attentions: typing.Optional[typing.Tuple[jax.Array]] = None cross_attentions: typing.Optional[typing.Tuple[jax.Array]] = None encoder_last_hidden_state: typing.Optional[jax.Array] = None encoder_hidden_states: typing.Optional[typing.Tuple[jax.Array]] = None encoder_attentions: typing.Optional[typing.Tuple[jax.Array]] = None  )   Parameters   logits (jnp.ndarray of shape (batch_size, config.num_labels)) —
Classification (or regression if config.num_labels==1) scores (before SoftMax).   past_key_values (tuple(tuple(jnp.ndarray)), optional, returned when use_cache=True is passed or when config.use_cache=True) —
Tuple of tuple(jnp.ndarray) of


length config.n_layers, with each tuple having 2 tensors of shape
(batch_size, num_heads, sequence_length, embed_size_per_head)) and 2 additional tensors of shape
(batch_size, num_heads, encoder_sequence_length, embed_size_per_head).
Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention
blocks) that can be used (see past_key_values input) to speed up sequential decoding.   decoder_hidden_states (tuple(jnp.ndarray), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of jnp.ndarray (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the decoder at the output of each layer plus the initial embedding outputs.   decoder_attentions (tuple(jnp.ndarray), optional, returned when output_attentions=True is passed


or when config.output_attentions=True) —
Tuple of jnp.ndarray (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the decoder, after the attention softmax, used to compute the weighted average in the
self-attention heads.   cross_attentions (tuple(jnp.ndarray), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of jnp.ndarray (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the decoder’s cross-attention layer, after the attention softmax, used to compute the
weighted average in the cross-attention heads.   encoder_last_hidden_state (jnp.ndarray of shape (batch_size, sequence_length, hidden_size), optional) —
Sequence of hidden-states at the output of the last layer of the encoder of the model.   encoder_hidden_states (tuple(jnp.ndarr


ay), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of jnp.ndarray (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the encoder at the output of each layer plus the initial embedding outputs.   encoder_attentions (tuple(jnp.ndarray), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of jnp.ndarray (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the encoder, after the attention softmax, used to compute the weighted average in the
self-attention heads.    Base class for outputs of sequence-to-sequence sentence classification models.  replace  < source > ( **updates  )    “Returns a new object replacing the specified fields with new valu


FlaxMultipleChoiceModelOutput

class transformers.modeling_flax_outputs.FlaxMultipleChoiceModelOutput  < source > ( logits: Array = None hidden_states: typing.Optional[typing.Tuple[jax.Array]] = None attentions: typing.Optional[typing.Tuple[jax.Array]] = None  )   Parameters   logits (jnp.ndarray of shape (batch_size, num_choices)) —
num_choices is the second dimension of the input tensors. (see input_ids above).
Classification scores (before SoftMax).   hidden_states (tuple(jnp.ndarray), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of jnp.ndarray (one for the output of the embeddings + one for the output of each layer)


FlaxMultipleChoiceModelOutput

 of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the initial embedding outputs.   attentions (tuple(jnp.ndarray), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of jnp.ndarray (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for outputs of multiple choice models.  replace  < source > ( **updates  )    “Returns a new object replacing the specified fields with new values


FlaxTokenClassifierOutput

class transformers.modeling_flax_outputs.FlaxTokenClassifierOutput  < source > ( logits: Array = None hidden_states: typing.Optional[typing.Tuple[jax.Array]] = None attentions: typing.Optional[typing.Tuple[jax.Array]] = None  )   Parameters   logits (jnp.ndarray of shape (batch_size, sequence_length, config.num_labels)) —
Classification scores (before SoftMax).   hidden_states (tuple(jnp.ndarray), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of jnp.ndarray (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, sequence_le


FlaxTokenClassifierOutput

ngth, hidden_size).
Hidden-states of the model at the output of each layer plus the initial embedding outputs.   attentions (tuple(jnp.ndarray), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of jnp.ndarray (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for outputs of token classification models.  replace  < source > ( **updates  )    “Returns a new object replacing the specified fields with new values.


FlaxQuestionAnsweringModelOutput

class transformers.modeling_flax_outputs.FlaxQuestionAnsweringModelOutput  < source > ( start_logits: Array = None end_logits: Array = None hidden_states: typing.Optional[typing.Tuple[jax.Array]] = None attentions: typing.Optional[typing.Tuple[jax.Array]] = None  )   Parameters   start_logits (jnp.ndarray of shape (batch_size, sequence_length)) —
Span-start scores (before SoftMax).   end_logits (jnp.ndarray of shape (batch_size, sequence_length)) —
Span-end scores (before SoftMax).   hidden_states (tuple(jnp.ndarray), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of jnp.ndarray (one for the output of the embeddings + one


FlaxQuestionAnsweringModelOutput

for the output of each layer) of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the initial embedding outputs.   attentions (tuple(jnp.ndarray), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of jnp.ndarray (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.    Base class for outputs of question answering models.  replace  < source > ( **updates  )    “Returns a new object replacing the specified fields with new values


class transformers.modeling_flax_outputs.FlaxSeq2SeqQuestionAnsweringModelOutput  < source > ( start_logits: Array = None end_logits: Array = None past_key_values: typing.Optional[typing.Tuple[typing.Tuple[jax.Array]]] = None decoder_hidden_states: typing.Optional[typing.Tuple[jax.Array]] = None decoder_attentions: typing.Optional[typing.Tuple[jax.Array]] = None cross_attentions: typing.Optional[typing.Tuple[jax.Array]] = None encoder_last_hidden_state: typing.Optional[jax.Array] = None encoder_hidden_states: typing.Optional[typing.Tuple[jax.Array]] = None encoder_attentions: typing.Optional[typing.Tuple[jax.Array]] = None  )   Parameters   start_logits (jnp.ndarray of shape (batch_size, sequence_length)) —
Span-start scores (before SoftMax).   end_logits (jnp.ndarray of shape (batch_size, sequence_length)) —
Span-end scores (before SoftMax).   past_key_values (tuple(tuple(jnp.ndarray)), optional, returned when use_cache=True


is passed or when config.use_cache=True) —
Tuple of tuple(jnp.ndarray) of length config.n_layers, with each tuple having 2 tensors of shape
(batch_size, num_heads, sequence_length, embed_size_per_head)) and 2 additional tensors of shape
(batch_size, num_heads, encoder_sequence_length, embed_size_per_head).
Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention
blocks) that can be used (see past_key_values input) to speed up sequential decoding.   decoder_hidden_states (tuple(jnp.ndarray), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of jnp.ndarray (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the decoder at the output of each layer plus the initial embedding outputs.   decoder_attentions (tuple(jnp.ndarray), option


al, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of jnp.ndarray (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the decoder, after the attention softmax, used to compute the weighted average in the
self-attention heads.   cross_attentions (tuple(jnp.ndarray), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of jnp.ndarray (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the decoder’s cross-attention layer, after the attention softmax, used to compute the
weighted average in the cross-attention heads.   encoder_last_hidden_state (jnp.ndarray of shape (batch_size, sequence_length, hidden_size), optional) —
Sequence of hidden-states at the output of the last layer of the encoder of the model.   encoder_hi


dden_states (tuple(jnp.ndarray), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of jnp.ndarray (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, sequence_length, hidden_size).
Hidden-states of the encoder at the output of each layer plus the initial embedding outputs.   encoder_attentions (tuple(jnp.ndarray), optional, returned when output_attentions=True is passed or when config.output_attentions=True) —
Tuple of jnp.ndarray (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights of the encoder, after the attention softmax, used to compute the weighted average in the
self-attention heads.    Base class for outputs of sequence-to-sequence question answering models.  replace  < source > ( **updates  )    “Returns a new object replacing the specified fields with new valu


AutoGPTQ Integration

🤗 Transformers has integrated optimum API to perform GPTQ quantization on language models. You can load and quantize your model in 8, 4, 3 or even 2 bits without a big drop of performance and faster inference speed! This is supported by most GPU hardwares.
To learn more about the the quantization model, check out:
the GPTQ paper the optimum guide on GPTQ quantization the AutoGPTQ library used as the backend


Requirements

You need to have the following requirements installed to run the code below:
Install latest AutoGPTQ library
pip install auto-gptq Install latest optimum from source
pip install git+https://github.com/huggingface/optimum.git Install latest transformers from source
pip install git+https://github.com/huggingface/transformers.git Install latest accelerate library
pip install --upgrade accelerate
Note that GPTQ integration supports for now only text models and you may encounter unexpected behaviour for vision, speech or multi-modal models.


Load and quantize a model

GPTQ is a quantization method that requires weights calibration before using the quantized models. If you want to quantize transformers model from scratch, it might take some time before producing the quantized model (~5 min on a Google colab for facebook/opt-350m model).
Hence, there are two different scenarios where you want to use GPTQ-quantized models. The first use case would be to load models that has been already quantized by other users that are available on the Hub, the second use case would be to quantize your model from scratch and save it or push it on the Hub so that other users can also use it.
 GPTQ Configuration
In order to load and quantize a model, you need to create a GPTQConfig. You need to pass the number of bits, a dataset in order to calibrate the quantization and the tokenizer of the model in order prepare the


Load and quantize a model

dataset.
  Copied model_id = "facebook/opt-125m"
tokenizer = AutoTokenizer.from_pretrained(model_id)
gptq_config = GPTQConfig(bits=4, dataset = "c4", tokenizer=tokenizer)
Note that you can pass your own dataset as a list of string. However, it is highly recommended to use the dataset from the GPTQ paper.
  Copied dataset = ["auto-gptq is an easy-to-use model quantization library with user-friendly apis, based on GPTQ algorithm."]
quantization = GPTQConfig(bits=4, dataset = dataset, tokenizer=tokenizer)
 Quantization
You can quantize a model by using from_pretrained and setting the quantization_config.
  Copied from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained(model_id, quantization_config=gptq_config)
Note that you will need a GPU to quantize a model. We will put the model in the cpu and move t


Load and quantize a model

he modules back and forth to the gpu in order to quantize them.
If you want to maximize your gpus usage while using cpu offload, you can set device_map = "auto".
  Copied from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto", quantization_config=gptq_config)
Note that disk offload is not supported. Furthermore, if you are out of memory because of the dataset, you may have to pass max_memory in from_pretained. Checkout this guide to learn more about device_map and max_memory.
GPTQ quantization only works for text model for now. Futhermore, the quantization process can a lot of time depending on one's hardware (175B model = 4 gpu hours using NVIDIA A100). Please check on the hub if there is not a GPTQ quantized version of the model. If not, you can submit a demand on githu


Push quantized model to 🤗 Hub

You can push the quantized model like any 🤗 model to Hub with push_to_hub. The quantization config will be saved and pushed along the model.
  Copied quantized_model.push_to_hub("opt-125m-gptq")
tokenizer.push_to_hub("opt-125m-gptq")
If you want to save your quantized model on your local machine, you can also do it with save_pretrained:
  Copied quantized_model.save_pretrained("opt-125m-gptq")
tokenizer.save_pretrained("opt-125m-gptq")
Note that if you have quantized your model with a device_map, make sure to move the entire model to one of your gpus or the cpu before saving it.
  Copied quantized_model.to("cpu")
quantized_model.save_pretrained("opt-125m-gptq")


Load a quantized model from the 🤗 Hub

You can load a quantized model from the Hub by using from_pretrained.
Make sure that the pushed weights are quantized, by checking that the attribute quantization_config is present in the model configuration object.
  Copied from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("{your_username}/opt-125m-gptq")
If you want to load a model faster and without allocating more memory than needed, the device_map argument also works with quantized model. Make sure that you have accelerate library installed.
  Copied from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("{your_username}/opt-125m-gptq", device_map="auto")


Exllama kernels for faster inference

For 4-bit model, you can use the exllama kernels in order to a faster inference speed. It is activated by default. You can change that behavior by passing disable_exllama in GPTQConfig. This will overwrite the quantization config stored in the config. Note that you will only be able to overwrite the attributes related to the kernels. Furthermore, you need to have the entire model on gpus if you want to use exllama kernels.
  Copied import torch
gptq_config = GPTQConfig(bits=4, disable_exllama=False)
mo