LlamaModel

 associated vectors than the
model’s internal embedding lookup matrix.   use_cache (bool, optional) —
If set to True, past_key_values key value states are returned and can be used to speed up decoding (see
past_key_values).   output_attentions (bool, optional) —
Whether or not to return the attentions tensors of all attention layers. See attentions under returned
tensors for more detail.   output_hidden_states (bool, optional) —
Whether or not to return the hidden states of all layers. See hidden_states under returned tensors for
more detail.   return_dict (bool, optional) —
Whether or not to return a ModelOutput instead of a plain tuple.    The LlamaModel forward method, overrides the __call__ special method. Although the recipe for forward pass needs to be defined within this function, one should call the Module
instance afterwards instead of this since the former takes care of running the pre and post processing steps while
the latter silently ignores them.


LlamaForCausalLM

class transformers.LlamaForCausalLM  < source > ( config  )     forward  < source > ( input_ids: LongTensor = None attention_mask: typing.Optional[torch.Tensor] = None position_ids: typing.Optional[torch.LongTensor] = None past_key_values: typing.Optional[typing.List[torch.FloatTensor]] = None inputs_embeds: typing.Optional[torch.FloatTensor] = None labels: typing.Optional[torch.LongTensor] = None use_cache: typing.Optional[bool] = None output_attentions: typing.Optional[bool] = None output_hidden_states: typing.Optional[bool] = None return_dict: typing.Optional[bool] = None  ) → transformers.modeling_outputs.CausalLMOutputWithPast or tuple(torch.FloatTensor)  Parameters   input_ids (torch.LongTensor of shape (batch_size, sequence_length)) —
Indices of input sequence tokens in the vocabulary. Padding will be ignored by default should you provide
it.
Indices can be


LlamaForCausalLM

 obtained using AutoTokenizer. See PreTrainedTokenizer.encode() and
PreTrainedTokenizer.call() for details.
What are input IDs?   attention_mask (torch.Tensor of shape (batch_size, sequence_length), optional) —
Mask to avoid performing attention on padding token indices. Mask values selected in [0, 1]:
1 for tokens that are not masked,
0 for tokens that are masked.
What are attention masks?
Indices can be obtained using AutoTokenizer. See PreTrainedTokenizer.encode() and
PreTrainedTokenizer.call() for details.
If past_key_values is used, optionally only the last input_ids have to be input (see
past_key_values).
If you want to change padding behavior, you should read modeling_opt._prepare_decoder_attention_mask
and modify to your needs. See diagram 1 in the paper for more
information on the default strategy.
1 indicates the head is not masked,
0 indicates the head


LlamaForCausalLM

is masked.
   position_ids (torch.LongTensor of shape (batch_size, sequence_length), optional) —
Indices of positions of each input sequence tokens in the position embeddings. Selected in the range [0, config.n_positions - 1].
What are position IDs?   past_key_values (tuple(tuple(torch.FloatTensor)), optional, returned when use_cache=True is passed or when config.use_cache=True) —
Tuple of tuple(torch.FloatTensor) of length config.n_layers, with each tuple having 2 tensors of shape
(batch_size, num_heads, sequence_length, embed_size_per_head)) and 2 additional tensors of shape
(batch_size, num_heads, encoder_sequence_length, embed_size_per_head).
Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention
blocks) that can be used (see past_key_values input) to speed up sequential decoding.
If past_key_values are used


LlamaForCausalLM

, the user can optionally input only the last input_ids (those that don’t
have their past key value states given to this model) of shape (batch_size, 1) instead of all input_ids
of shape (batch_size, sequence_length).   inputs_embeds (torch.FloatTensor of shape (batch_size, sequence_length, hidden_size), optional) —
Optionally, instead of passing input_ids you can choose to directly pass an embedded representation. This
is useful if you want more control over how to convert input_ids indices into associated vectors than the
model’s internal embedding lookup matrix.   use_cache (bool, optional) —
If set to True, past_key_values key value states are returned and can be used to speed up decoding (see
past_key_values).   output_attentions (bool, optional) —
Whether or not to return the attentions tensors of all attention layers. See attentions under returned
tensors f


LlamaForCausalLM

or more detail.   output_hidden_states (bool, optional) —
Whether or not to return the hidden states of all layers. See hidden_states under returned tensors for
more detail.   return_dict (bool, optional) —
Whether or not to return a ModelOutput instead of a plain tuple.
Args —
labels (torch.LongTensor of shape (batch_size, sequence_length), optional):
Labels for computing the masked language modeling loss. Indices should either be in [0, ..., config.vocab_size] or -100 (see input_ids docstring). Tokens with indices set to -100 are ignored
(masked), the loss is only computed for the tokens with labels in [0, ..., config.vocab_size].   Returns
transformers.modeling_outputs.CausalLMOutputWithPast or tuple(torch.FloatTensor)

A transformers.modeling_outputs.CausalLMOutputWithPast or a tuple of
torch.FloatTensor (if return_dict=False is passed or when config.return


LlamaForCausalLM

_dict=False) comprising various
elements depending on the configuration (LlamaConfig) and inputs.
loss (torch.FloatTensor of shape (1,), optional, returned when labels is provided) — Language modeling loss (for next-token prediction).
logits (torch.FloatTensor of shape (batch_size, sequence_length, config.vocab_size)) — Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax).
past_key_values (tuple(tuple(torch.FloatTensor)), optional, returned when use_cache=True is passed or when config.use_cache=True) — Tuple of tuple(torch.FloatTensor) of length config.n_layers, with each tuple having 2 tensors of shape
(batch_size, num_heads, sequence_length, embed_size_per_head))
Contains pre-computed hidden-states (key and values in the self-attention blocks) that can be used (see
past_key_values input) to speed up sequential decodin


LlamaForCausalLM

g.
hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) — Tuple of torch.FloatTensor (one for the output of the embeddings, if the model has an embedding layer, +
one for the output of each layer) of shape (batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.
attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) — Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention
heads.
  The LlamaForCausalLM forward method, overrides the __call__ special


LlamaForCausalLM

method. Although the recipe for forward pass needs to be defined within this function, one should call the Module
instance afterwards instead of this since the former takes care of running the pre and post processing steps while
the latter silently ignores them.  Example:   Copied >>> from transformers import AutoTokenizer, LlamaForCausalLM
>>> model = LlamaForCausalLM.from_pretrained(PATH_TO_CONVERTED_WEIGHTS)
>>> tokenizer = AutoTokenizer.from_pretrained(PATH_TO_CONVERTED_TOKENIZER)
>>> prompt = "Hey, are you conscious? Can you talk to me?"
>>> inputs = tokenizer(prompt, return_tensors="pt")
>>> # Generate
>>> generate_ids = model.generate(inputs.input_ids, max_length=30)
>>> tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]
"Hey, are you conscious? Can you talk to me?\nI'm not conscious, but I can talk to you.


LlamaForSequenceClassification

class transformers.LlamaForSequenceClassification  < source > ( config  )   Parameters   config (LlamaConfig) —
Model configuration class with all the parameters of the model. Initializing with a config file does not
load the weights associated with the model, only the configuration. Check out the
from_pretrained() method to load the model weights.    The LLaMa Model transformer with a sequence classification head on top (linear layer). LlamaForSequenceClassification uses the last token in order to do the classification, as other causal models
(e.g. GPT-2) do. Since it does classification on the last token, it requires to know the position of the last token. If a
pad_token_id is defined in the configuration, it finds the last token that is not a padding token in each row. If
no pad_token_id is defined, it simply takes the last value in each row of the batch. Since it cannot guess the
padding tokens when inputs_embeds are passed instead of input


LlamaForSequenceClassification

_ids, it does the same (take the last value in
each row of the batch). This model inherits from PreTrainedModel. Check the superclass documentation for the generic methods the
library implements for all its model (such as downloading or saving, resizing the input embeddings, pruning heads
etc.) This model is also a PyTorch torch.nn.Module subclass.
Use it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage
and behavior.  forward  < source > ( input_ids: LongTensor = None attention_mask: typing.Optional[torch.Tensor] = None position_ids: typing.Optional[torch.LongTensor] = None past_key_values: typing.Optional[typing.List[torch.FloatTensor]] = None inputs_embeds: typing.Optional[torch.FloatTensor] = None labels: typing.Optional[torch.LongTensor] = None use_cache: typing.Optional[bool] = None output_attentions: typing.Optional[bool] = None output_hidden_states: typing.Optional[bool] = None re


LlamaForSequenceClassification

turn_dict: typing.Optional[bool] = None  )   Parameters   input_ids (torch.LongTensor of shape (batch_size, sequence_length)) —
Indices of input sequence tokens in the vocabulary. Padding will be ignored by default should you provide
it.
Indices can be obtained using AutoTokenizer. See PreTrainedTokenizer.encode() and
PreTrainedTokenizer.call() for details.
What are input IDs?   attention_mask (torch.Tensor of shape (batch_size, sequence_length), optional) —
Mask to avoid performing attention on padding token indices. Mask values selected in [0, 1]:
1 for tokens that are not masked,
0 for tokens that are masked.
What are attention masks?
Indices can be obtained using AutoTokenizer. See PreTrainedTokenizer.encode() and
PreTrainedTokenizer.call() for details.
If past_key_values is used, optionally only the last input_ids have to be input (see
past_key_values).
If you want to change padding behavior, you should read modeling_opt._prepare_decoder_a


LlamaForSequenceClassification

ttention_mask
and modify to your needs. See diagram 1 in the paper for more
information on the default strategy.
1 indicates the head is not masked,
0 indicates the head is masked.
   position_ids (torch.LongTensor of shape (batch_size, sequence_length), optional) —
Indices of positions of each input sequence tokens in the position embeddings. Selected in the range [0, config.n_positions - 1].
What are position IDs?   past_key_values (tuple(tuple(torch.FloatTensor)), optional, returned when use_cache=True is passed or when config.use_cache=True) —
Tuple of tuple(torch.FloatTensor) of length config.n_layers, with each tuple having 2 tensors of shape
(batch_size, num_heads, sequence_length, embed_size_per_head)) and 2 additional tensors of shape
(batch_size, num_heads, encoder_sequence_length, embed_size_per_head).
Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention
blocks) that can be used


LlamaForSequenceClassification

(see past_key_values input) to speed up sequential decoding.
If past_key_values are used, the user can optionally input only the last input_ids (those that don’t
have their past key value states given to this model) of shape (batch_size, 1) instead of all input_ids
of shape (batch_size, sequence_length).   inputs_embeds (torch.FloatTensor of shape (batch_size, sequence_length, hidden_size), optional) —
Optionally, instead of passing input_ids you can choose to directly pass an embedded representation. This
is useful if you want more control over how to convert input_ids indices into associated vectors than the
model’s internal embedding lookup matrix.   use_cache (bool, optional) —
If set to True, past_key_values key value states are returned and can be used to speed up decoding (see
past_key_values).   output_attentions (bool, optional) —
Whether or not to return the attentions tensors of all attention layers. See attentions under returned
ten


LlamaForSequenceClassification

sors for more detail.   output_hidden_states (bool, optional) —
Whether or not to return the hidden states of all layers. See hidden_states under returned tensors for
more detail.   return_dict (bool, optional) —
Whether or not to return a ModelOutput instead of a plain tuple.   labels (torch.LongTensor of shape (batch_size,), optional) —
Labels for computing the sequence classification/regression loss. Indices should be in [0, ..., config.num_labels - 1]. If config.num_labels == 1 a regression loss is computed (Mean-Square loss), If
config.num_labels > 1 a classification loss is computed (Cross-Entropy).    The LlamaForSequenceClassification forward method, overrides the __call__ special method. Although the recipe for forward pass needs to be defined within this function, one should call the Module
instance afterwards instead of this since the former takes care of running the pre and post processing steps while
the latter silently ignores the


Overview

The FocalNet model was proposed in Focal Modulation Networks by Jianwei Yang, Chunyuan Li, Xiyang Dai, Lu Yuan, Jianfeng Gao.
FocalNets completely replace self-attention (used in models like ViT and Swin) by a focal modulation mechanism for modeling token interactions in vision.
The authors claim that FocalNets outperform self-attention based models with similar computational costs on the tasks of image classification, object detection, and segmentation.
The abstract from the paper is the following:
We propose focal modulation networks (FocalNets in short), where self-attention (SA) is completely replaced by a focal modulation mechanism for modeling token interactions in vision. Focal modulation comprises three components: (i) hierarchical contextualization, implemented using a stack of depth-wise convolutional l


Overview

ayers, to encode visual contexts from short to long ranges, (ii) gated aggregation to selectively gather contexts for each query token based on its
content, and (iii) element-wise modulation or affine transformation to inject the aggregated context into the query. Extensive experiments show FocalNets outperform the state-of-the-art SA counterparts (e.g., Swin and Focal Transformers) with similar computational costs on the tasks of image classification, object detection, and segmentation. Specifically, FocalNets with tiny and base size achieve 82.3% and 83.9% top-1 accuracy on ImageNet-1K. After pretrained on ImageNet-22K in 224 resolution, it attains 86.5% and 87.3% top-1 accuracy when finetuned with resolution 224 and 384, respectively. When transferred to downstream tasks, FocalNets exhibit clear superiority. F


Overview

or object detection with Mask R-CNN, FocalNet base trained with 1\times outperforms the Swin counterpart by 2.1 points and already surpasses Swin trained with 3\times schedule (49.0 v.s. 48.5). For semantic segmentation with UPerNet, FocalNet base at single-scale outperforms Swin by 2.4, and beats Swin at multi-scale (50.5 v.s. 49.7). Using large FocalNet and Mask2former, we achieve 58.5 mIoU for ADE20K semantic segmentation, and 57.9 PQ for COCO Panoptic Segmentation. Using huge FocalNet and DINO, we achieved 64.3 and 64.4 mAP on COCO minival and test-dev, respectively, establishing new SoTA on top of much larger attention-based models like Swinv2-G and BEIT-3.
Tips:
One can use the AutoImageProcessor class to prepare images for the model.
This model was contributed by nielsr.
The original code can be found here


FocalNetConfig

class transformers.FocalNetConfig  < source > ( image_size = 224 patch_size = 4 num_channels = 3 embed_dim = 96 use_conv_embed = False hidden_sizes = [192, 384, 768, 768] depths = [2, 2, 6, 2] focal_levels = [2, 2, 2, 2] focal_windows = [3, 3, 3, 3] hidden_act = 'gelu' mlp_ratio = 4.0 hidden_dropout_prob = 0.0 drop_path_rate = 0.1 use_layerscale = False layerscale_value = 0.0001 use_post_layernorm = False use_post_layernorm_in_modulation = False normalize_modulator = False initializer_range = 0.02 layer_norm_eps = 1e-05 encoder_stride = 32 out_features = None out_indices = None **kwargs  )   Parameters   image_size (int, optional, defaults to 224) —
The size (resolution) of each image.   patch_size (int, optional, defaults to 4) —
The size (resolution) of each patch in the embeddings layer.   num_channels (int, optional, defaults to 3) —
The number of input channels.   embed_dim (int, optional, defaults to 96) —
Dimensionality of


FocalNetConfig

 patch embedding.   use_conv_embed (bool, optional, defaults to False) —
Whether to use convolutional embedding. The authors noted that using convolutional embedding usually
improve the performance, but it’s not used by default.   hidden_sizes (List[int], optional, defaults to [192, 384, 768, 768]) —
Dimensionality (hidden size) at each stage.   depths (list(int), optional, defaults to [2, 2, 6, 2]) —
Depth (number of layers) of each stage in the encoder.   focal_levels (list(int), optional, defaults to [2, 2, 2, 2]) —
Number of focal levels in each layer of the respective stages in the encoder.   focal_windows (list(int), optional, defaults to [3, 3, 3, 3]) —
Focal window size in each layer of the respective stages in the encoder.   hidden_act (str or function, optional, defaults to "gelu") —
The non-linear activation function (function or string) in the encoder. If string, "gelu", "relu",
"selu" and "gelu_new" are supported.


FocalNetConfig

mlp_ratio (float, optional, defaults to 4.0) —
Ratio of MLP hidden dimensionality to embedding dimensionality.   hidden_dropout_prob (float, optional, defaults to 0.0) —
The dropout probability for all fully connected layers in the embeddings and encoder.   drop_path_rate (float, optional, defaults to 0.1) —
Stochastic depth rate.   use_layerscale (bool, optional, defaults to False) —
Whether to use layer scale in the encoder.   layerscale_value (float, optional, defaults to 0.0001) —
The initial value of the layer scale.   use_post_layernorm (bool, optional, defaults to False) —
Whether to use post layer normalization in the encoder.   use_post_layernorm_in_modulation (bool, optional, defaults to False) —
Whether to use post layer normalization in the modulation layer.   normalize_modulator (bool, optional, defaults to False) —
Whether to normalize the modulator.   initializer_range (float, optional, defaults to 0.02) —
The stan


FocalNetConfig

dard deviation of the truncated_normal_initializer for initializing all weight matrices.   layer_norm_eps (float, optional, defaults to 1e-05) —
The epsilon used by the layer normalization layers.   encoder_stride (int, optional, defaults to 32) —
Factor to increase the spatial resolution by in the decoder head for masked image modeling.   out_features (List[str], optional) —
If used as backbone, list of features to output. Can be any of "stem", "stage1", "stage2", etc.
(depending on how many stages the model has). If unset and out_indices is set, will default to the
corresponding stages. If unset and out_indices is unset, will default to the last stage.   out_indices (List[int], optional) —
If used as backbone, list of indices of features to output. Can be any of 0, 1, 2, etc. (depending on how
many stages the model has). If unset and out_features is set, will default to the corresponding stages.
If unset and out_features is uns


FocalNetConfig

et, will default to the last stage.    This is the configuration class to store the configuration of a FocalNetModel. It is used to instantiate a
FocalNet model according to the specified arguments, defining the model architecture. Instantiating a configuration
with the defaults will yield a similar configuration to that of the FocalNet
microsoft/focalnet-tiny architecture. Configuration objects inherit from PretrainedConfig and can be used to control the model outputs. Read the
documentation from PretrainedConfig for more information.  Example:   Copied >>> from transformers import FocalNetConfig, FocalNetModel
>>> # Initializing a FocalNet microsoft/focalnet-tiny style configuration
>>> configuration = FocalNetConfig()
>>> # Initializing a model (with random weights) from the microsoft/focalnet-tiny style configuration
>>> model = FocalNetModel(configuration)
>>> # Accessing the model configuration
>>> configuration = model.con


FocalNetModel

class transformers.FocalNetModel  < source > ( config add_pooling_layer = True use_mask_token = False  )   Parameters   config (FocalNetConfig) — Model configuration class with all the parameters of the model.
Initializing with a config file does not load the weights associated with the model, only the
configuration. Check out the from_pretrained() method to load the model weights.    The bare FocalNet Model outputting raw hidden-states without any specific head on top.
This model is a PyTorch torch.nn.Module sub-class. Use
it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage and
behavior.  forward  < source > ( pixel_values: typing.Optional[torch.FloatTensor] = None bool_masked_pos: typing.Optional[torch.BoolTensor] = None output_hidden_states: typ


FocalNetModel

ing.Optional[bool] = None return_dict: typing.Optional[bool] = None  ) → transformers.models.focalnet.modeling_focalnet.FocalNetModelOutput or tuple(torch.FloatTensor)  Parameters   pixel_values (torch.FloatTensor of shape (batch_size, num_channels, height, width)) —
Pixel values. Pixel values can be obtained using AutoImageProcessor. See
AutoImageProcessor.__call__() for details.   output_hidden_states (bool, optional) —
Whether or not to return the hidden states of all layers. See hidden_states under returned tensors for
more detail.   return_dict (bool, optional) —
Whether or not to return a ModelOutput instead of a plain tuple.   bool_masked_pos (torch.BoolTensor of shape (batch_size, num_patches)) —
Boolean masked positions. Indicates which patches are masked (1) and which aren’t (0).   Returns
tran


FocalNetModel

sformers.models.focalnet.modeling_focalnet.FocalNetModelOutput or tuple(torch.FloatTensor)

A transformers.models.focalnet.modeling_focalnet.FocalNetModelOutput or a tuple of
torch.FloatTensor (if return_dict=False is passed or when config.return_dict=False) comprising various
elements depending on the configuration (FocalNetConfig) and inputs.
last_hidden_state (torch.FloatTensor of shape (batch_size, sequence_length, hidden_size)) — Sequence of hidden-states at the output of the last layer of the model.
pooler_output (torch.FloatTensor of shape (batch_size, hidden_size), optional, returned when add_pooling_layer=True is passed) — Average pooling of the last layer hidden-state.
hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hid


FocalNetModel

den_states=True) — Tuple of torch.FloatTensor (one for the output of the embeddings + one for the output of each stage) of
shape (batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the initial embedding outputs.
reshaped_hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) — Tuple of torch.FloatTensor (one for the output of the embeddings + one for the output of each stage) of
shape (batch_size, hidden_size, height, width).
Hidden-states of the model at the output of each layer plus the initial embedding outputs reshaped to
include the spatial dimensions.
  The FocalNetModel forward method, overrides the __call__ special method. Although the recipe for forward pass


FocalNetModel

needs to be defined within this function, one should call the Module
instance afterwards instead of this since the former takes care of running the pre and post processing steps while
the latter silently ignores them.  Example:   Copied >>> from transformers import AutoImageProcessor, FocalNetModel
>>> import torch
>>> from datasets import load_dataset
>>> dataset = load_dataset("huggingface/cats-image")
>>> image = dataset["test"]["image"][0]
>>> image_processor = AutoImageProcessor.from_pretrained("microsoft/focalnet-tiny")
>>> model = FocalNetModel.from_pretrained("microsoft/focalnet-tiny")
>>> inputs = image_processor(image, return_tensors="pt")
>>> with torch.no_grad():
...     outputs = model(**inputs)
>>> last_hidden_states = outputs.last_hidden_state
>>> list(last_hidden_states.shape)
[1, 49, 768]


FocalNetForMaskedImageModeling

class transformers.FocalNetForMaskedImageModeling  < source > ( config  )   Parameters   config (FocalNetConfig) — Model configuration class with all the parameters of the model.
Initializing with a config file does not load the weights associated with the model, only the
configuration. Check out the from_pretrained() method to load the model weights.    FocalNet Model with a decoder on top for masked image modeling. This follows the same implementation as in SimMIM. Note that we provide a script to pre-train this model on custom data in our examples
directory. This model is a PyTorch torch.nn.Module sub-class. Use
it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage and
behavior.  forward  < source > ( pixel_values: typing.Optional[torch.FloatTensor] = None bool_masked_pos: typing.Optional[torch.BoolTensor] = None output_hidden_states:


FocalNetForMaskedImageModeling

 typing.Optional[bool] = None return_dict: typing.Optional[bool] = None  ) → transformers.models.focalnet.modeling_focalnet.FocalNetMaskedImageModelingOutput or tuple(torch.FloatTensor)  Parameters   pixel_values (torch.FloatTensor of shape (batch_size, num_channels, height, width)) —
Pixel values. Pixel values can be obtained using AutoImageProcessor. See
AutoImageProcessor.__call__() for details.   output_hidden_states (bool, optional) —
Whether or not to return the hidden states of all layers. See hidden_states under returned tensors for
more detail.   return_dict (bool, optional) —
Whether or not to return a ModelOutput instead of a plain tuple.   bool_masked_pos (torch.BoolTensor of shape (batch_size, num_patches)) —
Boolean masked positions. Indicates which patches are masked (1) and which aren’t (0).   Returns
transformers.models.focalnet.modeling_focalnet.FocalNetMaskedImageModelingO


FocalNetForMaskedImageModeling

utput or tuple(torch.FloatTensor)

A transformers.models.focalnet.modeling_focalnet.FocalNetMaskedImageModelingOutput or a tuple of
torch.FloatTensor (if return_dict=False is passed or when config.return_dict=False) comprising various
elements depending on the configuration (FocalNetConfig) and inputs.
loss (torch.FloatTensor of shape (1,), optional, returned when bool_masked_pos is provided) — Masked image modeling (MLM) loss.
reconstruction (torch.FloatTensor of shape (batch_size, num_channels, height, width)) — Reconstructed pixel values.
hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) — Tuple of torch.FloatTensor (one for the output of the embeddings + one for the output of each stage) of
shape (batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer p


FocalNetForMaskedImageModeling

lus the initial embedding outputs.
reshaped_hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) — Tuple of torch.FloatTensor (one for the output of the embeddings + one for the output of each stage) of
shape (batch_size, hidden_size, height, width).
Hidden-states of the model at the output of each layer plus the initial embedding outputs reshaped to
include the spatial dimensions.
  The FocalNetForMaskedImageModeling forward method, overrides the __call__ special method. Although the recipe for forward pass needs to be defined within this function, one should call the Module
instance afterwards instead of this since the former takes care of running the pre and post processing steps while
the latter silently ignores them.  Examples:   Copied >>> from transformers import AutoImageProcessor, FocalNetConfig


FocalNetForMaskedImageModeling

, FocalNetForMaskedImageModeling
>>> import torch
>>> from PIL import Image
>>> import requests
>>> url = "http://images.cocodataset.org/val2017/000000039769.jpg"
>>> image = Image.open(requests.get(url, stream=True).raw)
>>> image_processor = AutoImageProcessor.from_pretrained("microsoft/focalnet-base-simmim-window6-192")
>>> config = FocalNetConfig()
>>> model = FocalNetForMaskedImageModeling(config)
>>> num_patches = (model.config.image_size // model.config.patch_size) ** 2
>>> pixel_values = image_processor(images=image, return_tensors="pt").pixel_values
>>> # create random boolean mask of shape (batch_size, num_patches)
>>> bool_masked_pos = torch.randint(low=0, high=2, size=(1, num_patches)).bool()
>>> outputs = model(pixel_values, bool_masked_pos=bool_masked_pos)
>>> loss, reconstructed_pixel_values = outputs.loss, outputs.logits
>>> list(reconstructed_pixel_values.shape)
[1, 3, 192, 1


FocalNetForImageClassification

class transformers.FocalNetForImageClassification  < source > ( config  )   Parameters   config (FocalNetConfig) — Model configuration class with all the parameters of the model.
Initializing with a config file does not load the weights associated with the model, only the
configuration. Check out the from_pretrained() method to load the model weights.    FocalNet Model with an image classification head on top (a linear layer on top of the pooled output) e.g. for
ImageNet. This model is a PyTorch torch.nn.Module sub-class. Use
it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage and
behavior.  forward  < source > ( pixel_values: typing.Optional[torch.FloatTensor] = None labels: typing.Optional[torch.LongTensor] = None output_hidden_states: typing.Optional[bool] = None return_dict: typing.Optional[bool] =


FocalNetForImageClassification

 None  ) → transformers.models.focalnet.modeling_focalnet.FocalNetImageClassifierOutput or tuple(torch.FloatTensor)  Parameters   pixel_values (torch.FloatTensor of shape (batch_size, num_channels, height, width)) —
Pixel values. Pixel values can be obtained using AutoImageProcessor. See
AutoImageProcessor.__call__() for details.   output_hidden_states (bool, optional) —
Whether or not to return the hidden states of all layers. See hidden_states under returned tensors for
more detail.   return_dict (bool, optional) —
Whether or not to return a ModelOutput instead of a plain tuple.   labels (torch.LongTensor of shape (batch_size,), optional) —
Labels for computing the image classification/regression loss. Indices should be in [0, ..., config.num_labels - 1]. If config.num_labels == 1 a regression loss is computed (Mean-Square loss), If
config.num_labels > 1 a


FocalNetForImageClassification

classification loss is computed (Cross-Entropy).   Returns
transformers.models.focalnet.modeling_focalnet.FocalNetImageClassifierOutput or tuple(torch.FloatTensor)

A transformers.models.focalnet.modeling_focalnet.FocalNetImageClassifierOutput or a tuple of
torch.FloatTensor (if return_dict=False is passed or when config.return_dict=False) comprising various
elements depending on the configuration (FocalNetConfig) and inputs.
loss (torch.FloatTensor of shape (1,), optional, returned when labels is provided) — Classification (or regression if config.num_labels==1) loss.
logits (torch.FloatTensor of shape (batch_size, config.num_labels)) — Classification (or regression if config.num_labels==1) scores (before SoftMax).
hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True)


FocalNetForImageClassification

 — Tuple of torch.FloatTensor (one for the output of the embeddings + one for the output of each stage) of
shape (batch_size, sequence_length, hidden_size).
Hidden-states of the model at the output of each layer plus the initial embedding outputs.
reshaped_hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) — Tuple of torch.FloatTensor (one for the output of the embeddings + one for the output of each stage) of
shape (batch_size, hidden_size, height, width).
Hidden-states of the model at the output of each layer plus the initial embedding outputs reshaped to
include the spatial dimensions.
  The FocalNetForImageClassification forward method, overrides the __call__ special method. Although the recipe for forward pass needs to be defined within this function, one should c


FocalNetForImageClassification

all the Module
instance afterwards instead of this since the former takes care of running the pre and post processing steps while
the latter silently ignores them.  Example:   Copied >>> from transformers import AutoImageProcessor, FocalNetForImageClassification
>>> import torch
>>> from datasets import load_dataset
>>> dataset = load_dataset("huggingface/cats-image")
>>> image = dataset["test"]["image"][0]
>>> image_processor = AutoImageProcessor.from_pretrained("microsoft/focalnet-tiny")
>>> model = FocalNetForImageClassification.from_pretrained("microsoft/focalnet-tiny")
>>> inputs = image_processor(image, return_tensors="pt")
>>> with torch.no_grad():
...     logits = model(**inputs).logits
>>> # model predicts one of the 1000 ImageNet classes
>>> predicted_label = logits.argmax(-1).item()
>>> print(model.config.id2label[predicted_label])
tabby, tabby cat


Time Series Utilities

This page lists all the utility functions and classes that can be used for Time Series based models.
Most of those are only useful if you are studying the code of the time series models or you wish to add to the collection of distributional output classes.


Distributional Output

class transformers.time_series_utils.NormalOutput  < source > ( dim: int = 1  )    Normal distribution output class.
 class transformers.time_series_utils.StudentTOutput  < source > ( dim: int = 1  )    Student-T distribution output class.
 class transformers.time_series_utils.NegativeBinomialOutput  < source > ( dim: int = 1  )    Negative Binomial distribution output class.


Utilities for Generation

This page lists all the utility functions used by generate(),
greedy_search(),
contrastive_search(),
sample(),
beam_search(),
beam_sample(),
group_beam_search(), and
constrained_beam_search().
Most of those are only useful if you are studying the code of the generate methods in the library.


Generate Outputs

The output of generate() is an instance of a subclass of
ModelOutput. This output is a data structure containing all the information returned
by generate(), but that can also be used as tuple or dictionary.
Here’s an example:
  Copied from transformers import GPT2Tokenizer, GPT2LMHeadModel
tokenizer = GPT2Tokenizer.from_pretrained("gpt2")
model = GPT2LMHeadModel.from_pretrained("gpt2")
inputs = tokenizer("Hello, my dog is cute and ", return_tensors="pt")
generation_output = model.generate(**inputs, return_dict_in_generate=True, output_scores=True)
The generation_output object is a GreedySearchDecoderOnlyOutput, as we can
see in the documentation of that class bel


Generate Outputs

ow, it means it has the following attributes:
sequences: the generated sequences of tokens scores (optional): the prediction scores of the language modelling head, for each generation step hidden_states (optional): the hidden states of the model, for each generation step attentions (optional): the attention weights of the model, for each generation step
Here we have the scores since we passed along output_scores=True, but we don’t have hidden_states and
attentions because we didn’t pass output_hidden_states=True or output_attentions=True.
You can access each attribute as you would usually do, and if that attribute has not been returned by the model, you
will get


Generate Outputs

None. Here for instance generation_output.scores are all the generated prediction scores of the
language modeling head, and generation_output.attentions is None.
When using our generation_output object as a tuple, it only keeps the attributes that don’t have None values.
Here, for instance, it has two elements, loss then logits, so
  Copied generation_output[:2]
will return the tuple (generation_output.sequences, generation_output.scores) for instance.
When using our generation_output object as a dictionary, it only keeps the attributes that don’t have None
values. Here, for instance, it has two keys that are sequences and scores.
We document here all output type


PyTorch

class transformers.generation.GreedySearchEncoderDecoderOutput  < source > ( sequences: LongTensor = None scores: typing.Optional[typing.Tuple[torch.FloatTensor]] = None encoder_attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None encoder_hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None decoder_attentions: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None cross_attentions: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None decoder_hidden_states: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None  )   Parameters   sequences (torch.LongTensor of shape (batch_size, sequence_length)) —
The generated sequences. The second dimension (sequence_length) is either equal to max_length or shorter
if all batches finished early due to the eos_token_id.   scores (tuple(torch.FloatTensor) optional, returned when output_scores=True is passed or when config.output_scores=True) —
Processed predi


PyTorch

ction scores of the language modeling head (scores for each vocabulary token before SoftMax)
at each generation step. Tuple of torch.FloatTensor with up to max_new_tokens elements (one element for
each generated token), with each tensor of shape (batch_size, config.vocab_size).   encoder_attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer of the decoder) of shape (batch_size, num_heads, sequence_length, sequence_length).   encoder_hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of torch.FloatTensor (one for the output of the embeddings + one for the output of each layer) of
shape (batch_size, sequence_length, hidden_size).   decoder_attentions (tuple(tuple(torch.FloatTensor)), optional, returned when output_attentions=True is passed or config.ou


PyTorch

tput_attentions=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
torch.FloatTensor of shape (batch_size, num_heads, generated_length, sequence_length).   cross_attentions (tuple(tuple(torch.FloatTensor)), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
torch.FloatTensor of shape (batch_size, num_heads, generated_length, sequence_length).   decoder_hidden_states (tuple(tuple(torch.FloatTensor)), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
torch.FloatTensor of shape (batch_size, generated_length, hidden_size).    Base class for outputs of encoder-decoder generation models using greedy search. Hidden states an


PyTorch

d attention
weights of the decoder (respectively the encoder) can be accessed via the encoder_attentions and the
encoder_hidden_states attributes (respectively the decoder_attentions and the decoder_hidden_states attributes)
 class transformers.generation.GreedySearchDecoderOnlyOutput  < source > ( sequences: LongTensor = None scores: typing.Optional[typing.Tuple[torch.FloatTensor]] = None attentions: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None hidden_states: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None  )   Parameters   sequences (torch.LongTensor of shape (batch_size, sequence_length)) —
The generated sequences. The second dimension (sequence_length) is either equal to max_length or shorter
if all batches finished early due to the eos_token_id.   scores (tuple(torch.FloatTensor) optional, returned when output_scores=True is passed or when config.output_scores=True) —
Processed prediction scores of the language modelin


PyTorch

g head (scores for each vocabulary token before SoftMax)
at each generation step. Tuple of torch.FloatTensor with up to max_new_tokens elements (one element for
each generated token), with each tensor of shape (batch_size, config.vocab_size).   attentions (tuple(tuple(torch.FloatTensor)), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
torch.FloatTensor of shape (batch_size, num_heads, generated_length, sequence_length).   hidden_states (tuple(tuple(torch.FloatTensor)), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
torch.FloatTensor of shape (batch_size, generated_length, hidden_size).    Base class for outputs of decoder-only generation models using greedy search.
 cl


PyTorch

ass transformers.generation.SampleEncoderDecoderOutput  < source > ( sequences: LongTensor = None scores: typing.Optional[typing.Tuple[torch.FloatTensor]] = None encoder_attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None encoder_hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None decoder_attentions: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None cross_attentions: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None decoder_hidden_states: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None  )   Parameters   sequences (torch.LongTensor of shape (batch_size*num_return_sequences, sequence_length)) —
The generated sequences. The second dimension (sequence_length) is either equal to max_length or shorter
if all batches finished early due to the eos_token_id.   scores (tuple(torch.FloatTensor) optional, returned when output_scores=True is passed or when config.output_scores=True) —
Pr


PyTorch

ocessed prediction scores of the language modeling head (scores for each vocabulary token before SoftMax)
at each generation step. Tuple of torch.FloatTensor with up to max_new_tokens elements (one element for
each generated token), with each tensor of shape (batch_size*num_return_sequences, config.vocab_size).   encoder_attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer of the decoder) of shape
(batch_size*num_return_sequences, num_heads, sequence_length, sequence_length).   encoder_hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of torch.FloatTensor (one for the output of the embeddings + one for the output of each layer) of
shape (batch_size*num_return_sequences, sequence_length, hidden_size).   decoder_attentions (tuple(tuple(torch.FloatTen


PyTorch

sor)), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
torch.FloatTensor of shape (batch_size*num_return_sequences, num_heads, generated_length, sequence_length).   cross_attentions (tuple(tuple(torch.FloatTensor)), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
torch.FloatTensor of shape (batch_size, num_heads, generated_length, sequence_length).   decoder_hidden_states (tuple(tuple(torch.FloatTensor)), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
torch.FloatTensor of shape (batch_size*num_return_sequences, generated_leng


PyTorch

th, hidden_size).    Base class for outputs of encoder-decoder generation models using sampling. Hidden states and attention weights of
the decoder (respectively the encoder) can be accessed via the encoder_attentions and the encoder_hidden_states
attributes (respectively the decoder_attentions and the decoder_hidden_states attributes)
 class transformers.generation.SampleDecoderOnlyOutput  < source > ( sequences: LongTensor = None scores: typing.Optional[typing.Tuple[torch.FloatTensor]] = None attentions: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None hidden_states: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None  )   Parameters   sequences (torch.LongTensor of shape (batch_size*num_return_sequences, sequence_length)) —
The generated sequences. The second dimension (sequence_length) is either equal to max_length or shorter
if all batches finished early due to the eos_token_id.   scores (tuple(torch.FloatTensor) optional, ret


PyTorch

urned when output_scores=True is passed or when config.output_scores=True) —
Processed prediction scores of the language modeling head (scores for each vocabulary token before SoftMax)
at each generation step. Tuple of torch.FloatTensor with up to max_new_tokens elements (one element for
each generated token), with each tensor of shape (batch_size*num_return_sequences, config.vocab_size).   attentions (tuple(tuple(torch.FloatTensor)), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
torch.FloatTensor of shape (num_return_sequences*batch_size, num_heads, generated_length, sequence_length).   hidden_states (tuple(tuple(torch.FloatTensor)), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of th


PyTorch

e decoder) of
torch.FloatTensor of shape (num_return_sequences*batch_size, generated_length, hidden_size).    Base class for outputs of decoder-only generation models using sampling.
 class transformers.generation.BeamSearchEncoderDecoderOutput  < source > ( sequences: LongTensor = None sequences_scores: typing.Optional[torch.FloatTensor] = None scores: typing.Optional[typing.Tuple[torch.FloatTensor]] = None beam_indices: typing.Optional[torch.LongTensor] = None encoder_attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None encoder_hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None decoder_attentions: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None cross_attentions: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None decoder_hidden_states: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None  )   Parameters   sequences (torch.LongTensor of shape (batch_size*num_return_sequences, se


PyTorch

quence_length)) —
The generated sequences. The second dimension (sequence_length) is either equal to max_length or shorter
if all batches finished early due to the eos_token_id.   sequences_scores (torch.FloatTensor of shape (batch_size*num_return_sequences), optional, returned when output_scores=True is passed or when config.output_scores=True) —
Final beam scores of the generated sequences.   scores (tuple(torch.FloatTensor) optional, returned when output_scores=True is passed or when config.output_scores=True) —
Beam transition scores for each vocabulary token at each generation step. Beam transition scores consisting
of log probabilities of tokens conditioned on log softmax of previously generated tokens in this beam.
Tuple of torch.FloatTensor with up to max_new_tokens elements (one element for each generated token),
with each tensor of shape (batch_size*num_beams, config.vocab_size).   beam_indices (torch.LongTensor, optional, returned when output_scores=True is p


PyTorch

assed or when config.output_scores=True) —
Beam indices of generated token id at each generation step. torch.LongTensor of shape
(batch_size*num_return_sequences, sequence_length).   encoder_attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer of the decoder) of shape (batch_size, num_heads, sequence_length, sequence_length).   encoder_hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of torch.FloatTensor (one for the output of the embeddings + one for the output of each layer) of
shape (batch_size*num_beams*num_return_sequences, sequence_length, hidden_size).   decoder_attentions (tuple(tuple(torch.FloatTensor)), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple (one element for each generated token


PyTorch

) of tuples (one element for each layer of the decoder) of
torch.FloatTensor of shape (batch_size*num_beams*num_return_sequences, num_heads, generated_length, sequence_length).   cross_attentions (tuple(tuple(torch.FloatTensor)), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
torch.FloatTensor of shape (batch_size, num_heads, generated_length, sequence_length).   decoder_hidden_states (tuple(tuple(torch.FloatTensor)), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
torch.FloatTensor of shape (batch_size*num_beams*num_return_sequences, generated_length, hidden_size).    Base class for outputs of encoder-decoder generation models using beam search. Hidden states and atten


PyTorch

tion weights
of the decoder (respectively the encoder) can be accessed via the encoder_attentions and the encoder_hidden_states
attributes (respectively the decoder_attentions and the decoder_hidden_states attributes)
 class transformers.generation.BeamSearchDecoderOnlyOutput  < source > ( sequences: LongTensor = None sequences_scores: typing.Optional[torch.FloatTensor] = None scores: typing.Optional[typing.Tuple[torch.FloatTensor]] = None beam_indices: typing.Optional[torch.LongTensor] = None attentions: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None hidden_states: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None  )   Parameters   sequences (torch.LongTensor of shape (batch_size*num_return_sequences, sequence_length)) —
The generated sequences. The second dimension (sequence_length) is either equal to max_length or shorter
if all batches finished early due to the eos_token_id.   sequences_scores (torch.FloatTensor of shape (b


PyTorch

atch_size*num_return_sequences), optional, returned when output_scores=True is passed or when config.output_scores=True) —
Final beam scores of the generated sequences.   scores (tuple(torch.FloatTensor) optional, returned when output_scores=True is passed or when config.output_scores=True) —
Beam transition scores for each vocabulary token at each generation step. Beam transition scores consisting
of log probabilities of tokens conditioned on log softmax of previously generated tokens in this beam.
Tuple of torch.FloatTensor with up to max_new_tokens elements (one element for each generated token),
with each tensor of shape (batch_size*num_beams*num_return_sequences, config.vocab_size).   beam_indices (torch.LongTensor, optional, returned when output_scores=True is passed or when config.output_scores=True) —
Beam indices of generated token id at each generation step. torch.LongTensor of shape
(batch_size*num_return_sequences, sequence_length).   attentions (tuple(tuple


PyTorch

(torch.FloatTensor)), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
torch.FloatTensor of shape (batch_size*num_beams, num_heads, generated_length, sequence_length).   hidden_states (tuple(tuple(torch.FloatTensor)), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
torch.FloatTensor of shape (batch_size*num_beams*num_return_sequences, generated_length, hidden_size).    Base class for outputs of decoder-only generation models using beam search.
 class transformers.generation.BeamSampleEncoderDecoderOutput  < source > ( sequences: LongTensor = None sequences_scores: typing.Optional[torch.FloatTensor] = None scores: typing.Optional[typing.Tuple[torch.FloatTensor]] = None bea


PyTorch

m_indices: typing.Optional[torch.LongTensor] = None encoder_attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None encoder_hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None decoder_attentions: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None cross_attentions: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None decoder_hidden_states: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None  )   Parameters   sequences (torch.LongTensor of shape (batch_size*num_beams, sequence_length)) —
The generated sequences. The second dimension (sequence_length) is either equal to max_length or shorter
if all batches finished early due to the eos_token_id.   sequences_scores (torch.FloatTensor of shape (batch_size * num_return_sequence), optional, returned when output_scores=True is passed or when config.output_scores=True) —
Final beam scores of the generated sequences.   scores (tuple(torch.FloatTe


PyTorch

nsor) optional, returned when output_scores=True is passed or when config.output_scores=True) —
Beam transition scores for each vocabulary token at each generation step. Beam transition scores consisting
of log probabilities of tokens conditioned on log softmax of previously generated tokens in this beam.
Tuple of torch.FloatTensor with up to max_new_tokens elements (one element for each generated token),
with each tensor of shape (batch_size*num_beams, config.vocab_size)).   beam_indices (torch.LongTensor, optional, returned when output_scores=True is passed or when config.output_scores=True) —
Beam indices of generated token id at each generation step. torch.LongTensor of shape
(batch_size*num_return_sequences, sequence_length).   encoder_attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer of the decoder) of shape (batch_size, num_heads, sequen


PyTorch

ce_length, sequence_length).   encoder_hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of torch.FloatTensor (one for the output of the embeddings + one for the output of each layer) of
shape (batch_size*num_beams, sequence_length, hidden_size).   decoder_attentions (tuple(tuple(torch.FloatTensor)), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
torch.FloatTensor of shape (batch_size*num_beams, num_heads, generated_length, sequence_length).   cross_attentions (tuple(tuple(torch.FloatTensor)), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
torch.FloatTensor of shape (batch_size, n


PyTorch

um_heads, generated_length, sequence_length).   decoder_hidden_states (tuple(tuple(torch.FloatTensor)), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
torch.FloatTensor of shape (batch_size*num_beams, generated_length, hidden_size).    Base class for outputs of encoder-decoder generation models using beam sampling. Hidden states and attention
weights of the decoder (respectively the encoder) can be accessed via the encoder_attentions and the
encoder_hidden_states attributes (respectively the decoder_attentions and the decoder_hidden_states attributes)
 class transformers.generation.BeamSampleDecoderOnlyOutput  < source > ( sequences: LongTensor = None sequences_scores: typing.Optional[torch.FloatTensor] = None scores: typing.Optional[typing.Tuple[torch.FloatTensor]] = None beam_indices: typing.Optional[torch.LongT


PyTorch

ensor] = None attentions: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None hidden_states: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None  )   Parameters   sequences (torch.LongTensor of shape (batch_size*num_return_sequences, sequence_length)) —
The generated sequences. The second dimension (sequence_length) is either equal to max_length or shorter
if all batches finished early due to the eos_token_id.   sequences_scores (torch.FloatTensor of shape (batch_size * num_return_sequence), optional, returned when output_scores=True is passed or when config.output_scores=True) —
Final beam scores of the generated sequences.   scores (tuple(torch.FloatTensor) optional, returned when output_scores=True is passed or when config.output_scores=True) —
Beam transition scores for each vocabulary token at each generation step. Beam transition scores consisting
of log probabilities of tokens conditioned on log softmax of previously generated


PyTorch

tokens in this beam.
Tuple of torch.FloatTensor with up to max_new_tokens elements (one element for each generated token),
with each tensor of shape (batch_size*num_beams*num_return_sequences, config.vocab_size).   beam_indices (torch.LongTensor, optional, returned when output_scores=True is passed or when config.output_scores=True) —
Beam indices of generated token id at each generation step. torch.LongTensor of shape
(batch_size*num_return_sequences, sequence_length).   attentions (tuple(tuple(torch.FloatTensor)), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
torch.FloatTensor of shape (batch_size*num_beams, num_heads, generated_length, sequence_length).   hidden_states (tuple(tuple(torch.FloatTensor)), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple (one eleme


PyTorch

nt for each generated token) of tuples (one element for each layer of the decoder) of
torch.FloatTensor of shape (batch_size*num_beams, generated_length, hidden_size).    Base class for outputs of decoder-only generation models using beam sample.
 class transformers.generation.ContrastiveSearchEncoderDecoderOutput  < source > ( sequences: LongTensor = None scores: typing.Optional[typing.Tuple[torch.FloatTensor]] = None encoder_attentions: typing.Optional[typing.Tuple[torch.FloatTensor]] = None encoder_hidden_states: typing.Optional[typing.Tuple[torch.FloatTensor]] = None decoder_attentions: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None cross_attentions: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None decoder_hidden_states: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None  )   Parameters   sequences (torch.LongTensor of shape (batch_size, sequence_length)) —
The generated sequences. The second dimension (


PyTorch

sequence_length) is either equal to max_length or shorter
if all batches finished early due to the eos_token_id.   scores (tuple(torch.FloatTensor) optional, returned when output_scores=True is passed or when config.output_scores=True) —
Processed prediction scores of the language modeling head (scores for each vocabulary token before SoftMax)
at each generation step. Tuple of torch.FloatTensor with up to max_new_tokens elements (one element for
each generated token), with each tensor of shape (batch_size, config.vocab_size).   encoder_attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple of torch.FloatTensor (one for each layer of the decoder) of shape (batch_size, num_heads, sequence_length, sequence_length).   encoder_hidden_states (tuple(torch.FloatTensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of torch.FloatTenso


PyTorch

r (one for the output of the embeddings + one for the output of each layer) of
shape (batch_size, sequence_length, hidden_size).   decoder_attentions (tuple(tuple(torch.FloatTensor)), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
torch.FloatTensor of shape (batch_size, num_heads, generated_length, sequence_length).   cross_attentions (tuple(tuple(torch.FloatTensor)), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
torch.FloatTensor of shape (batch_size, num_heads, generated_length, sequence_length).   decoder_hidden_states (tuple(tuple(torch.FloatTensor)), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple (one element fo


PyTorch

r each generated token) of tuples (one element for each layer of the decoder) of
torch.FloatTensor of shape (batch_size, generated_length, hidden_size).    Base class for outputs of decoder-only generation models using contrastive search.
 class transformers.generation.ContrastiveSearchDecoderOnlyOutput  < source > ( sequences: LongTensor = None scores: typing.Optional[typing.Tuple[torch.FloatTensor]] = None attentions: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None hidden_states: typing.Optional[typing.Tuple[typing.Tuple[torch.FloatTensor]]] = None  )   Parameters   sequences (torch.LongTensor of shape (batch_size, sequence_length)) —
The generated sequences. The second dimension (sequence_length) is either equal to max_length or shorter
if all batches finished early due to the eos_token_id.   scores (tuple(torch.FloatTensor) optional, returned when output_scores=True is passed or when —
config.output_scores=True) —
Processed prediction scores of


PyTorch

 the language modeling head (scores for each vocabulary token before SoftMax)
at each generation step. Tuple of torch.FloatTensor with up to max_new_tokens elements (one element for
each generated token), with each tensor of shape (batch_size, config.vocab_size).   attentions (tuple(tuple(torch.FloatTensor)), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
torch.FloatTensor of shape (batch_size, num_heads, generated_length, sequence_length).   hidden_states (tuple(tuple(torch.FloatTensor)), optional, returned when output_hidden_states=True is —   passed or when config.output_hidden_states=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
torch.FloatTensor of shape (batch_size, generated_length, hidden_size).    Base class for outputs of decoder-only generation models


TensorFlow

class transformers.generation.TFGreedySearchEncoderDecoderOutput  < source > ( sequences: Tensor = None scores: typing.Optional[typing.Tuple[tensorflow.python.framework.ops.Tensor]] = None encoder_attentions: typing.Optional[typing.Tuple[tensorflow.python.framework.ops.Tensor]] = None encoder_hidden_states: typing.Optional[typing.Tuple[tensorflow.python.framework.ops.Tensor]] = None decoder_attentions: typing.Optional[typing.Tuple[typing.Tuple[tensorflow.python.framework.ops.Tensor]]] = None cross_attentions: typing.Optional[typing.Tuple[typing.Tuple[tensorflow.python.framework.ops.Tensor]]] = None decoder_hidden_states: typing.Optional[typing.Tuple[typing.Tuple[tensorflow.python.framework.ops.Tensor]]] = None  )   Parameters   sequences (tf.Tensor of shape (batch_size, sequence_length)) —
The generated sequences. The second dimension (sequence_length) is either equal to max_length or shorter
if all batches finished early due to the eos_token_id.   sco


TensorFlow

res (tuple(tf.Tensor) optional, returned when output_scores=True is passed or when config.output_scores=True) —
Processed prediction scores of the language modeling head (scores for each vocabulary token before SoftMax)
at each generation step. Tuple of tf.Tensor with up to max_new_tokens elements (one element for each
generated token), with each tensor of shape (batch_size, config.vocab_size).   encoder_attentions (tuple(tf.Tensor), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple of tf.Tensor (one for each layer of the decoder) of shape (batch_size, num_heads, sequence_length, sequence_length).   encoder_hidden_states (tuple(tf.Tensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of tf.Tensor (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, sequence_length, hidden_size).   decoder_attention


TensorFlow

s (tuple(tuple(tf.Tensor)), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
tf.Tensor of shape (batch_size, num_heads, generated_length, sequence_length).   cross_attentions (tuple(tuple(tf.Tensor)), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
tf.Tensor of shape (batch_size, num_heads, generated_length, sequence_length).   decoder_hidden_states (tuple(tuple(tf.Tensor)), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
tf.Tensor of shape (batch_size, generated_length, hidden_size).    Base class for outputs


TensorFlow

 of encoder-decoder generation models using greedy search. Hidden states and attention
weights of the decoder (respectively the encoder) can be accessed via the encoder_attentions and the
encoder_hidden_states attributes (respectively the decoder_attentions and the decoder_hidden_states attributes)
 class transformers.generation.TFGreedySearchDecoderOnlyOutput  < source > ( sequences: Tensor = None scores: typing.Optional[typing.Tuple[tensorflow.python.framework.ops.Tensor]] = None attentions: typing.Optional[typing.Tuple[typing.Tuple[tensorflow.python.framework.ops.Tensor]]] = None hidden_states: typing.Optional[typing.Tuple[typing.Tuple[tensorflow.python.framework.ops.Tensor]]] = None  )   Parameters   sequences (tf.Tensor of shape (batch_size, sequence_length)) —
The generated sequences. The second dimension (sequence_length) is either equal to max_length or shorter
if all batches finished early due to the eos_token_id.   scores (tuple(tf.Tensor) op


TensorFlow

tional, returned when output_scores=True is passed or when config.output_scores=True) —
Processed prediction scores of the language modeling head (scores for each vocabulary token before SoftMax)
at each generation step. Tuple of tf.Tensor with up to max_new_tokens elements (one element for each
generated token), with each tensor of shape (batch_size, config.vocab_size).   attentions (tuple(tuple(tf.Tensor)), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
tf.Tensor of shape (batch_size, num_heads, generated_length, sequence_length).   hidden_states (tuple(tuple(tf.Tensor)), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
tf.Tensor of shape (batch_size,


TensorFlow

 generated_length, hidden_size).    Base class for outputs of decoder-only generation models using greedy search.
 class transformers.generation.TFSampleEncoderDecoderOutput  < source > ( sequences: Tensor = None scores: typing.Optional[typing.Tuple[tensorflow.python.framework.ops.Tensor]] = None encoder_attentions: typing.Optional[typing.Tuple[tensorflow.python.framework.ops.Tensor]] = None encoder_hidden_states: typing.Optional[typing.Tuple[tensorflow.python.framework.ops.Tensor]] = None decoder_attentions: typing.Optional[typing.Tuple[typing.Tuple[tensorflow.python.framework.ops.Tensor]]] = None cross_attentions: typing.Optional[typing.Tuple[typing.Tuple[tensorflow.python.framework.ops.Tensor]]] = None decoder_hidden_states: typing.Optional[typing.Tuple[typing.Tuple[tensorflow.python.framework.ops.Tensor]]] = None  )   Parameters   sequences (tf.Tensor of shape (batch_size*num_return_sequences, sequence_length)) —
The generated sequences. The second


TensorFlow

 dimension (sequence_length) is either equal to max_length or shorter
if all batches finished early due to the eos_token_id.   scores (tuple(tf.Tensor) optional, returned when output_scores=True is passed or when config.output_scores=True) —
Processed prediction scores of the language modeling head (scores for each vocabulary token before SoftMax)
at each generation step. Tuple of tf.Tensor with up to max_new_tokens elements (one element for each
generated token), with each tensor of shape (batch_size*num_return_sequences, config.vocab_size).   encoder_attentions (tuple(tf.Tensor), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple of tf.Tensor (one for each layer of the decoder) of shape (batch_size*num_return_sequences, num_heads, sequence_length, sequence_length).   encoder_hidden_states (tuple(tf.Tensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=T


TensorFlow

rue) —
Tuple of tf.Tensor (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size*num_return_sequences, sequence_length, hidden_size).   decoder_attentions (tuple(tuple(tf.Tensor)), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
tf.Tensor of shape (batch_size*num_return_sequences, num_heads, generated_length, sequence_length).   cross_attentions (tuple(tuple(tf.Tensor)), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
tf.Tensor of shape (batch_size, num_heads, generated_length, sequence_length).   decoder_hidden_states (tuple(tuple(tf.Tensor)), optional, returned when output_hidden_states=True is passed or when config.output


TensorFlow

_hidden_states=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
tf.Tensor of shape (batch_size*num_return_sequences, generated_length, hidden_size).    Base class for outputs of encoder-decoder generation models using sampling. Hidden states and attention weights of
the decoder (respectively the encoder) can be accessed via the encoder_attentions and the encoder_hidden_states
attributes (respectively the decoder_attentions and the decoder_hidden_states attributes)
 class transformers.generation.TFSampleDecoderOnlyOutput  < source > ( sequences: Tensor = None scores: typing.Optional[typing.Tuple[tensorflow.python.framework.ops.Tensor]] = None attentions: typing.Optional[typing.Tuple[typing.Tuple[tensorflow.python.framework.ops.Tensor]]] = None hidden_states: typing.Optional[typing.Tuple[typing.Tuple[tensorflow.python.framework.ops.Tensor]]] = None  )   Parameters   sequences (tf.Tensor of shap


TensorFlow

e (batch_size*num_return_sequences, sequence_length)) —
The generated sequences. The second dimension (sequence_length) is either equal to max_length or shorter
if all batches finished early due to the eos_token_id.   scores (tuple(tf.Tensor) optional, returned when output_scores=True is passed or when config.output_scores=True) —
Processed prediction scores of the language modeling head (scores for each vocabulary token before SoftMax)
at each generation step. Tuple of tf.Tensor with up to max_new_tokens elements (one element for each
generated token), with each tensor of shape (batch_size*num_return_sequences, config.vocab_size).   attentions (tuple(tuple(tf.Tensor)), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
tf.Tensor of shape (num_return_sequences*batch_size, num_heads, generated_length, sequence_l


TensorFlow

ength).   hidden_states (tuple(tuple(tf.Tensor)), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
tf.Tensor of shape (num_return_sequences*batch_size, generated_length, hidden_size).    Base class for outputs of decoder-only generation models using sampling.
 class transformers.generation.TFBeamSearchEncoderDecoderOutput  < source > ( sequences: Tensor = None sequences_scores: typing.Optional[tensorflow.python.framework.ops.Tensor] = None scores: typing.Optional[typing.Tuple[tensorflow.python.framework.ops.Tensor]] = None beam_indices: typing.Optional[tensorflow.python.framework.ops.Tensor] = None encoder_attentions: typing.Optional[typing.Tuple[tensorflow.python.framework.ops.Tensor]] = None encoder_hidden_states: typing.Optional[typing.Tuple[tensorflow.python.framework.ops.Tensor]] = None decode


TensorFlow

r_attentions: typing.Optional[typing.Tuple[typing.Tuple[tensorflow.python.framework.ops.Tensor]]] = None cross_attentions: typing.Optional[typing.Tuple[typing.Tuple[tensorflow.python.framework.ops.Tensor]]] = None decoder_hidden_states: typing.Optional[typing.Tuple[typing.Tuple[tensorflow.python.framework.ops.Tensor]]] = None  )   Parameters   sequences (tf.Tensor of shape (batch_size*num_return_sequences, sequence_length)) —
The generated sequences. The second dimension (sequence_length) is either equal to max_length or shorter
if all batches finished early due to the eos_token_id.   sequences_scores (tf.Tensor of shape (batch_size*num_return_sequences), optional, returned when output_scores=True is passed or when config.output_scores=True) —
Final beam scores of the generated sequences.   scores (tuple(tf.Tensor) optional, returned when output_scores=True is passed or when config.output_scores=True) —
Processed beam scores for each vocabulary token a


TensorFlow

t each generation step. Beam scores consisting of log
softmax scores for each vocabulary token and sum of log softmax of previously generated tokens in this
beam. Tuple of tf.Tensorwith up tomax_new_tokenselements (one element for each generated token), with each tensor of shape(batch_size*num_beams, config.vocab_size)`.   beam_indices (tf.Tensor, optional, returned when output_scores=True is passed or when config.output_scores=True) —
Beam indices of generated token id at each generation step. tf.Tensor of shape
(batch_size*num_return_sequences, sequence_length).   encoder_attentions (tuple(tf.Tensor), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple of tf.Tensor (one for each layer of the decoder) of shape (batch_size, num_heads, sequence_length, sequence_length).   encoder_hidden_states (tuple(tf.Tensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=


TensorFlow

True) —
Tuple of tf.Tensor (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size*num_beams*num_return_sequences, sequence_length, hidden_size).   decoder_attentions (tuple(tuple(tf.Tensor)), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
tf.Tensor of shape (batch_size*num_beams*num_return_sequences, num_heads, generated_length, sequence_length).   cross_attentions (tuple(tuple(tf.Tensor)), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
tf.Tensor of shape (batch_size, num_heads, generated_length, sequence_length).   decoder_hidden_states (tuple(tuple(tf.Tensor)), optional, returned when output_hidden_states=True is passed


TensorFlow

or when config.output_hidden_states=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
tf.Tensor of shape (batch_size*num_beams*num_return_sequences, generated_length, hidden_size).    Base class for outputs of encoder-decoder generation models using beam search. Hidden states and attention weights
of the decoder (respectively the encoder) can be accessed via the encoder_attentions and the encoder_hidden_states
attributes (respectively the decoder_attentions and the decoder_hidden_states attributes)
 class transformers.generation.TFBeamSearchDecoderOnlyOutput  < source > ( sequences: Tensor = None sequences_scores: typing.Optional[tensorflow.python.framework.ops.Tensor] = None scores: typing.Optional[typing.Tuple[tensorflow.python.framework.ops.Tensor]] = None beam_indices: typing.Optional[tensorflow.python.framework.ops.Tensor] = None attentions: typing.Optional[typing.Tuple[typing.Tuple[tenso


TensorFlow

rflow.python.framework.ops.Tensor]]] = None hidden_states: typing.Optional[typing.Tuple[typing.Tuple[tensorflow.python.framework.ops.Tensor]]] = None  )   Parameters   sequences (tf.Tensor of shape (batch_size*num_return_sequences, sequence_length)) —
The generated sequences. The second dimension (sequence_length) is either equal to max_length or shorter
if all batches finished early due to the eos_token_id.   sequences_scores (tf.Tensor of shape (batch_size*num_return_sequences), optional, returned when output_scores=True is passed or when config.output_scores=True) —
Final beam scores of the generated sequences.   scores (tuple(tf.Tensor) optional, returned when output_scores=True is passed or when config.output_scores=True) —
Processed beam scores for each vocabulary token at each generation step. Beam scores consisting of log
softmax scores for each vocabulary token and sum of log softmax of previously generated tokens in this
beam. Tuple of tf.Ten


TensorFlow

sor with up to max_new_tokens elements (one element for each generated token),
with each tensor of shape (batch_size*num_beams*num_return_sequences, config.vocab_size).   beam_indices (tf.Tensor, optional, returned when output_scores=True is passed or when config.output_scores=True) —
Beam indices of generated token id at each generation step. tf.Tensor of shape
(batch_size*num_return_sequences, sequence_length).   attentions (tuple(tuple(tf.Tensor)), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
tf.Tensor of shape (batch_size*num_beams, num_heads, generated_length, sequence_length).   hidden_states (tuple(tuple(tf.Tensor)), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple (one element for each generated token) of tuples (one element for each lay


TensorFlow

er of the decoder) of
tf.Tensor of shape (batch_size*num_beams*num_return_sequences, generated_length, hidden_size).    Base class for outputs of decoder-only generation models using beam search.
 class transformers.generation.TFBeamSampleEncoderDecoderOutput  < source > ( sequences: Tensor = None sequences_scores: typing.Optional[tensorflow.python.framework.ops.Tensor] = None scores: typing.Optional[typing.Tuple[tensorflow.python.framework.ops.Tensor]] = None beam_indices: typing.Optional[tensorflow.python.framework.ops.Tensor] = None encoder_attentions: typing.Optional[typing.Tuple[tensorflow.python.framework.ops.Tensor]] = None encoder_hidden_states: typing.Optional[typing.Tuple[tensorflow.python.framework.ops.Tensor]] = None decoder_attentions: typing.Optional[typing.Tuple[typing.Tuple[tensorflow.python.framework.ops.Tensor]]] = None cross_attentions: typing.Optional[typing.Tuple[typing.Tuple[tensorflow.python.framework.ops.Tensor]]] = None decoder


TensorFlow

_hidden_states: typing.Optional[typing.Tuple[typing.Tuple[tensorflow.python.framework.ops.Tensor]]] = None  )   Parameters   sequences (tf.Tensor of shape (batch_size*num_beams, sequence_length)) —
The generated sequences. The second dimension (sequence_length) is either equal to max_length or shorter
if all batches finished early due to the eos_token_id.   sequences_scores (tf.Tensor of shape (batch_size * num_return_sequence), optional, returned when output_scores=True is passed or when config.output_scores=True) —
Final beam scores of the generated sequences.   scores (tuple(tf.Tensor) optional, returned when output_scores=True is passed or when config.output_scores=True) —
Processed beam scores for each vocabulary token at each generation step. Beam scores consisting of log
softmax scores for each vocabulary token and sum of log softmax of previously generated tokens in this
beam. Tuple of tf.Tensor with up to max_new_tokens elements (one element f


TensorFlow

or each generated token),
with each tensor of shape (batch_size*num_beams, config.vocab_size).   beam_indices (tf.Tensor, optional, returned when output_scores=True is passed or when config.output_scores=True) —
Beam indices of generated token id at each generation step. tf.Tensor of shape
(batch_size*num_return_sequences, sequence_length).   encoder_attentions (tuple(tf.Tensor), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple of tf.Tensor (one for each layer of the decoder) of shape (batch_size, num_heads, sequence_length, sequence_length).   encoder_hidden_states (tuple(tf.Tensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of tf.Tensor (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size*num_beams, sequence_length, hidden_size).   decoder_attentions (tuple(tuple(tf.Tensor)), optional, returne


TensorFlow

d when output_attentions=True is passed or config.output_attentions=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
tf.Tensor of shape (batch_size*num_beams, num_heads, generated_length, sequence_length).   cross_attentions (tuple(tuple(tf.Tensor)), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
tf.Tensor of shape (batch_size, num_heads, generated_length, sequence_length).   decoder_hidden_states (tuple(tuple(tf.Tensor)), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
tf.Tensor of shape (batch_size*num_beams, generated_length, hidden_size).    Base class for outputs of encoder-decoder gener


TensorFlow

ation models using beam sampling. Hidden states and attention
weights of the decoder (respectively the encoder) can be accessed via the encoder_attentions and the
encoder_hidden_states attributes (respectively the decoder_attentions and the decoder_hidden_states attributes)
 class transformers.generation.TFBeamSampleDecoderOnlyOutput  < source > ( sequences: Tensor = None sequences_scores: typing.Optional[tensorflow.python.framework.ops.Tensor] = None scores: typing.Optional[typing.Tuple[tensorflow.python.framework.ops.Tensor]] = None beam_indices: typing.Optional[tensorflow.python.framework.ops.Tensor] = None attentions: typing.Optional[typing.Tuple[typing.Tuple[tensorflow.python.framework.ops.Tensor]]] = None hidden_states: typing.Optional[typing.Tuple[typing.Tuple[tensorflow.python.framework.ops.Tensor]]] = None  )   Parameters   sequences (tf.Tensor of shape (batch_size*num_return_sequences, sequence_length)) —
The generated sequences. The second d


TensorFlow

imension (sequence_length) is either equal to max_length or shorter
if all batches finished early due to the eos_token_id.   sequences_scores (tf.Tensor of shape (batch_size * num_return_sequence), optional, returned when output_scores=True is passed or when config.output_scores=True) —
Final beam scores of the generated sequences.   scores (tuple(tf.Tensor) optional, returned when output_scores=True is passed or when config.output_scores=True) —
Processed beam scores for each vocabulary token at each generation step. Beam scores consisting of log
softmax scores for each vocabulary token and sum of log softmax of previously generated tokens in this
beam. Tuple of tf.Tensor with up to max_new_tokens elements (one element for each generated token),
with each tensor of shape (batch_size*num_beams*num_return_sequences, config.vocab_size).   beam_indices (tf.Tensor, optional, returned when output_scores=True is passed or when config.output_scores=True) —
Be


TensorFlow

am indices of generated token id at each generation step. tf.Tensor of shape
(batch_size*num_return_sequences, sequence_length).   attentions (tuple(tuple(tf.Tensor)), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
tf.Tensor of shape (batch_size*num_beams, num_heads, generated_length, sequence_length).   hidden_states (tuple(tuple(tf.Tensor)), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
tf.Tensor of shape (batch_size*num_beams, generated_length, hidden_size).    Base class for outputs of decoder-only generation models using beam sample.
 class transformers.generation.TFContrastiveSearchEncoderDecoderOutput  < source > ( sequences: Tensor = None sco


TensorFlow

res: typing.Optional[typing.Tuple[tensorflow.python.framework.ops.Tensor]] = None encoder_attentions: typing.Optional[typing.Tuple[tensorflow.python.framework.ops.Tensor]] = None encoder_hidden_states: typing.Optional[typing.Tuple[tensorflow.python.framework.ops.Tensor]] = None decoder_attentions: typing.Optional[typing.Tuple[typing.Tuple[tensorflow.python.framework.ops.Tensor]]] = None cross_attentions: typing.Optional[typing.Tuple[typing.Tuple[tensorflow.python.framework.ops.Tensor]]] = None decoder_hidden_states: typing.Optional[typing.Tuple[typing.Tuple[tensorflow.python.framework.ops.Tensor]]] = None  )   Parameters   sequences (tf.Tensor of shape (batch_size, sequence_length)) —
The generated sequences. The second dimension (sequence_length) is either equal to max_length or shorter
if all batches finished early due to the eos_token_id.   scores (tuple(tf.Tensor) optional, returned when output_scores=True is passed or when config.output_scores=Tru


TensorFlow

e) —
Processed prediction scores of the language modeling head (scores for each vocabulary token before SoftMax)
at each generation step. Tuple of tf.Tensor with up to max_new_tokens elements (one element for each
generated token), with each tensor of shape (batch_size, config.vocab_size).   encoder_attentions (tuple(tf.Tensor), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple of tf.Tensor (one for each layer of the decoder) of shape (batch_size, num_heads, sequence_length, sequence_length).   encoder_hidden_states (tuple(tf.Tensor), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple of tf.Tensor (one for the output of the embeddings + one for the output of each layer) of shape
(batch_size, sequence_length, hidden_size).   decoder_attentions (tuple(tuple(tf.Tensor)), optional, returned when output_attentions=True is passed or config.output_atten


TensorFlow

tions=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
tf.Tensor of shape (batch_size, num_heads, generated_length, sequence_length).   cross_attentions (tuple(tuple(tf.Tensor)), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
tf.Tensor of shape (batch_size, num_heads, generated_length, sequence_length).   decoder_hidden_states (tuple(tuple(tf.Tensor)), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
tf.Tensor of shape (batch_size, generated_length, hidden_size).    Base class for outputs of encoder-decoder generation models using contrastive search. Hidden states and attention
weights of the


TensorFlow

decoder (respectively the encoder) can be accessed via the encoder_attentions and the
encoder_hidden_states attributes (respectively the decoder_attentions and the decoder_hidden_states attributes)
 class transformers.generation.TFContrastiveSearchDecoderOnlyOutput  < source > ( sequences: Tensor = None scores: typing.Optional[typing.Tuple[tensorflow.python.framework.ops.Tensor]] = None attentions: typing.Optional[typing.Tuple[typing.Tuple[tensorflow.python.framework.ops.Tensor]]] = None hidden_states: typing.Optional[typing.Tuple[typing.Tuple[tensorflow.python.framework.ops.Tensor]]] = None  )   Parameters   sequences (tf.Tensor of shape (batch_size, sequence_length)) —
The generated sequences. The second dimension (sequence_length) is either equal to max_length or shorter
if all batches finished early due to the eos_token_id.   scores (tuple(tf.Tensor) optional, returned when output_scores=True is passed or when config.output_scores=True) —
Processed


TensorFlow

 prediction scores of the language modeling head (scores for each vocabulary token before SoftMax)
at each generation step. Tuple of tf.Tensor with up to max_new_tokens elements (one element for each
generated token), with each tensor of shape (batch_size, config.vocab_size).   attentions (tuple(tuple(tf.Tensor)), optional, returned when output_attentions=True is passed or config.output_attentions=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
tf.Tensor of shape (batch_size, num_heads, generated_length, sequence_length).   hidden_states (tuple(tuple(tf.Tensor)), optional, returned when output_hidden_states=True is passed or when config.output_hidden_states=True) —
Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of
tf.Tensor of shape (batch_size, generated_length, hidden_size).    Base class for outputs of decoder-only generation models usin


FLAX

class transformers.generation.FlaxSampleOutput  < source > ( sequences: Array = None  )   Parameters   sequences (jnp.ndarray of shape (batch_size, max_length)) —
The generated sequences.    Flax Base class for outputs of decoder-only generation models using sampling.  replace  < source > ( **updates  )    “Returns a new object replacing the specified fields with new values.
 class transformers.generation.FlaxGreedySearchOutput  < source > ( sequences: Array = None  )   Parameters   sequences (jnp.ndarray of shape (batch_size, max_length)) —
The generated sequences.    Flax Base class for outputs of decoder-only generation models usi


FLAX

ng greedy search.  replace  < source > ( **updates  )    “Returns a new object replacing the specified fields with new values.
 class transformers.generation.FlaxBeamSearchOutput  < source > ( sequences: Array = None scores: Array = None  )   Parameters   sequences (jnp.ndarray of shape (batch_size, max_length)) —
The generated sequences.   scores (jnp.ndarray of shape (batch_size,)) —
The scores (log probabilities) of the generated sequences.    Flax Base class for outputs of decoder-only generation models using greedy search.  replace  < source > ( **updates  )    “Returns a new object replacing the specified fields with new values


LogitsProcessor

A LogitsProcessor can be used to modify the prediction scores of a language model head for
generation.


PyTorch

class transformers.AlternatingCodebooksLogitsProcessor  < source > ( input_start_len: int semantic_vocab_size: int codebook_size: int  )   Parameters   input_start_len (int) —
The length of the initial input sequence.   semantic_vocab_size (int) —
Vocabulary size of the semantic part, i.e number of tokens associated to the semantic vocabulary.   codebook_size (int) —
Number of tokens associated to the codebook.    LogitsProcessor enforcing alternated generation between the two codebooks of Bark’s fine submodel.  __call__  < source > ( input_ids: LongTensor scores: FloatTensor  )
 class transformers.ClassifierFreeGuidanceLogitsProcessor  < source > ( guidance_scale  )   Parameters   guidance_scale (float) —
The guidance scale for classifier free guidance (CFG). CFG is enabled by setting guidance_scale > 1.
Higher guidance scale encourages the model to generate samples that are more closely linked to the input
prompt, usually at the expense of poorer quality.    Logits processor for


PyTorch

 classifier free guidance (CFG). The scores are split over the batch dimension,
where the first half correspond to the conditional logits (predicted from the input prompt) and the second half
correspond to the unconditional logits (predicted from an empty or ‘null’ prompt). The processor computes a
weighted average across the conditional and unconditional logits, parameterised by the guidance_scale. See the paper for more information.  __call__  < source > ( input_ids: LongTensor scores: FloatTensor  ) → torch.FloatTensor of shape (batch_size, config.vocab_size)  Parameters   input_ids (torch.LongTensor of shape (batch_size, sequence_length)) —
Indices of input sequence tokens in the vocabulary. What are input IDs?   scores (torch.FloatTensor of shape (batch_size, config.vocab_size)) —
Prediction scores of a language modeling head. These can be logits for each vocabulary when not using beam
search or log softmax for each vocabulary token when using beam search   Returns
torch.FloatT


PyTorch

ensor of shape (batch_size, config.vocab_size)

The processed prediction scores.

 class transformers.EncoderNoRepeatNGramLogitsProcessor  < source > ( encoder_ngram_size: int encoder_input_ids: LongTensor  )   Parameters   encoder_ngram_size (int) —
All ngrams of size ngram_size can only occur within the encoder input ids.   encoder_input_ids (int) —
The encoder_input_ids that should not be repeated within the decoder ids.    LogitsProcessor that enforces no repetition of encoder input ids n-grams for the decoder ids. See
ParlAI.  __call__  < source > ( input_ids: LongTensor scores: FloatTensor  ) → torch.FloatTensor of shape (batch_size, config.vocab_size)  Parameters   input_ids (torch.LongTensor of shape (batch_size, sequence_length)) —
Indices of input sequence tokens in the vocabulary. What are input IDs?   scores (torch.FloatTensor of shape (batch_size, config.vocab_size)) —
Prediction scores of a language modeling head. These can be logits for each vocabulary when not usin


PyTorch

g beam
search or log softmax for each vocabulary token when using beam search   Returns
torch.FloatTensor of shape (batch_size, config.vocab_size)

The processed prediction scores.

 class transformers.EncoderRepetitionPenaltyLogitsProcessor  < source > ( penalty: float encoder_input_ids: LongTensor  )   Parameters   hallucination_penalty (float) —
The parameter for hallucination penalty. 1.0 means no penalty.   encoder_input_ids (torch.LongTensor) —
The encoder_input_ids that should be repeated within the decoder ids.    LogitsProcessor enforcing an exponential penalty on tokens that are not in the original input.  __call__  < source > ( input_ids: LongTensor scores: FloatTensor  ) → torch.FloatTensor of shape (batch_size, config.vocab_size)  Parameters   input_ids (torch.LongTensor of shape (batch_size, sequence_length)) —
Indices of input sequence tokens in the vocabulary. What are input IDs?   scores (torch.FloatTensor of shape (batch_size, config.vocab_size)) —
Prediction sc