FlaxAutoModelForMaskedLM

m huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = FlaxAutoModelForMaskedLM.from_config(config)  from_pretrained  < source > ( *model_args **kwargs  )   Parameters   pretrained_model_name_or_path (str or os.PathLike) —
Can be either:
A string, the model id of a pretrained model hosted inside a model repo on huggingface.co.
Valid model ids can be located at the root-level, like bert-base-uncased, or namespaced under a
user or organization name, like dbmdz/bert-base-german-cased.
A path to a directory containing model weights saved using
save_pretrained(), e.g., ./my_model_directory/.
A path or url to a PyTorch state_dict save file (e.g, ./pt_model/pytorch_model.bin). In this
case, from_pt should be set to True and a configuration object should be provided as config
argument. This loading path is slower than converting the PyTorch model in a TensorFlow mo


FlaxAutoModelForMaskedLM

del
using the provided conversion scripts and loading the TensorFlow model afterwards.
   model_args (additional positional arguments, optional) —
Will be passed along to the underlying model __init__() method.   config (PretrainedConfig, optional) —
Configuration for the model to use instead of an automatically loaded configuration. Configuration can
be automatically loaded when:
The model is a model provided by the library (loaded with the model id string of a pretrained
model).
The model was saved using save_pretrained() and is reloaded by supplying the
save directory.
The model is loaded by supplying a local directory as pretrained_model_name_or_path and a
configuration JSON file named config.json is found in the directory.
   cache_dir (str or os.PathLike, optional) —
Path to a directory in which a downloaded pretrained model configuration should be cached if the
standard cache should not be use


FlaxAutoModelForMaskedLM

d.   from_pt (bool, optional, defaults to False) —
Load the model weights from a PyTorch checkpoint save file (see docstring of
pretrained_model_name_or_path argument).   force_download (bool, optional, defaults to False) —
Whether or not to force the (re-)download of the model weights and configuration files, overriding the
cached versions if they exist.   resume_download (bool, optional, defaults to False) —
Whether or not to delete incompletely received files. Will attempt to resume the download if such a
file exists.   proxies (Dict[str, str], optional) —
A dictionary of proxy servers to use by protocol or endpoint, e.g., {'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}. The proxies are used on each request.   output_loading_info(bool, optional, defaults to False) —
Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.   local_files_onl


FlaxAutoModelForMaskedLM

y(bool, optional, defaults to False) —
Whether or not to only look at local files (e.g., not try downloading the model).   revision (str, optional, defaults to "main") —
The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
git-based system for storing models and other artifacts on huggingface.co, so revision can be any
identifier allowed by git.   trust_remote_code (bool, optional, defaults to False) —
Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
should only be set to True for repositories you trust and in which you have read the code, as it will
execute code present on the Hub on your local machine.   code_revision (str, optional, defaults to "main") —
The specific revision to use for the code on the Hub, if the code leaves in a different repository than
the rest of the model. It can be a branc


FlaxAutoModelForMaskedLM

h name, a tag name, or a commit id, since we use a git-based
system for storing models and other artifacts on huggingface.co, so revision can be any identifier
allowed by git.   kwargs (additional keyword arguments, optional) —
Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
output_attentions=True). Behaves differently depending on whether a config is provided or
automatically loaded:
If a configuration is provided with config, **kwargs will be directly passed to the
underlying model’s __init__ method (we assume all relevant updates to the configuration have
already been done)
If a configuration is not provided, kwargs will be first passed to the configuration class
initialization function (from_pretrained()). Each key of kwargs that
corresponds to a configuration attribute will be used to override said attribute with the
supplied kwargs value. Rem


FlaxAutoModelForMaskedLM

aining keys that do not correspond to any configuration attribute
will be passed to the underlying model’s __init__ function.
    Instantiate one of the model classes of the library (with a masked language modeling head) from a pretrained model. The model class to instantiate is selected based on the model_type property of the config object (either
passed as an argument or loaded from pretrained_model_name_or_path if possible), or when it’s missing, by
falling back to using pattern matching on pretrained_model_name_or_path: albert — FlaxAlbertForMaskedLM (ALBERT model) bart — FlaxBartForConditionalGeneration (BART model) bert — FlaxBertForMaskedLM (BERT model) big_bird — FlaxBigBirdForMaskedLM (BigBird model) distilbert — FlaxDistilBertForMaskedLM (DistilBERT model) electra — FlaxElectraForMaskedLM (ELECTRA model) mbart — FlaxMBartForConditionalGeneration (mBART model) roberta — FlaxRobertaForMaskedL


FlaxAutoModelForMaskedLM

M (RoBERTa model) roberta-prelayernorm — FlaxRobertaPreLayerNormForMaskedLM (RoBERTa-PreLayerNorm model) roformer — FlaxRoFormerForMaskedLM (RoFormer model) xlm-roberta — FlaxXLMRobertaForMaskedLM (XLM-RoBERTa model)  Examples:   Copied >>> from transformers import AutoConfig, FlaxAutoModelForMaskedLM
>>> # Download model and configuration from huggingface.co and cache.
>>> model = FlaxAutoModelForMaskedLM.from_pretrained("bert-base-cased")
>>> # Update configuration during loading
>>> model = FlaxAutoModelForMaskedLM.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True
>>> # Loading from a PyTorch checkpoint file instead of a TensorFlow model (slower)
>>> config = AutoConfig.from_pretrained("./pt_model/bert_pt_model_config.json")
>>> model = FlaxAutoModelForMaskedLM.from_pretrained(
...     "./pt_model/bert_pytorch_model.bin", from_pt=True, config=config


class transformers.AutoModelForSeq2SeqLM  < source > ( *args **kwargs  )    This is a generic model class that will be instantiated as one of the model classes of the library (with a sequence-to-sequence language modeling head) when created
with the from_pretrained() class method or the from_config() class
method. This class cannot be instantiated directly using __init__() (throws an error).  from_config  < source > ( **kwargs  )   Parameters   config (PretrainedConfig) —
The model class to instantiate is selected based on the configuration class:
BartConfig configuration class: BartForConditionalGeneration (BART model)
BigBirdPegasusConfig configuration class: BigBirdPegasusForConditionalGeneration (BigBird-Pegasus model)
BlenderbotConfig configuration class: BlenderbotForConditionalGeneration (Blenderbot model)
BlenderbotSmallConfig configuration class: BlenderbotSmallForConditionalGeneration (BlenderbotSmall model)
EncoderDecoderConfig configuration clas


s: EncoderDecoderModel (Encoder decoder model)
FSMTConfig configuration class: FSMTForConditionalGeneration (FairSeq Machine-Translation model)
GPTSanJapaneseConfig configuration class: GPTSanJapaneseForConditionalGeneration (GPTSAN-japanese model)
LEDConfig configuration class: LEDForConditionalGeneration (LED model)
LongT5Config configuration class: LongT5ForConditionalGeneration (LongT5 model)
M2M100Config configuration class: M2M100ForConditionalGeneration (M2M100 model)
MBartConfig configuration class: MBartForConditionalGeneration (mBART model)
MT5Config configuration class: MT5ForConditionalGeneration (MT5 model)
MarianConfig configuration class: MarianMTModel (Marian model)
MvpConfig configuration class: MvpForConditionalGeneration (MVP model)
NllbMoeConfig configuration class: NllbMoeForConditionalGeneration (NLLB-MOE model)
PLBartConfig configuration class: PLBartForConditionalGeneration (PLBart model)
PegasusConfig configuration class: PegasusFor


ConditionalGeneration (Pegasus model)
PegasusXConfig configuration class: PegasusXForConditionalGeneration (PEGASUS-X model)
ProphetNetConfig configuration class: ProphetNetForConditionalGeneration (ProphetNet model)
SwitchTransformersConfig configuration class: SwitchTransformersForConditionalGeneration (SwitchTransformers model)
T5Config configuration class: T5ForConditionalGeneration (T5 model)
UMT5Config configuration class: UMT5ForConditionalGeneration (UMT5 model)
XLMProphetNetConfig configuration class: XLMProphetNetForConditionalGeneration (XLM-ProphetNet model)
    Instantiates one of the model classes of the library (with a sequence-to-sequence language modeling head) from a configuration. Note:
Loading a model from its configuration file does not load the model weights. It only affects the
model’s configuration. Use from_pretrained() to load the model weights.  Examples:   Copied >>> from transformers import AutoConfig, AutoModelForSeq2SeqLM
>>>


# Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("t5-base")
>>> model = AutoModelForSeq2SeqLM.from_config(config)  from_pretrained  < source > ( *model_args **kwargs  )   Parameters   pretrained_model_name_or_path (str or os.PathLike) —
Can be either:
A string, the model id of a pretrained model hosted inside a model repo on huggingface.co.
Valid model ids can be located at the root-level, like bert-base-uncased, or namespaced under a
user or organization name, like dbmdz/bert-base-german-cased.
A path to a directory containing model weights saved using
save_pretrained(), e.g., ./my_model_directory/.
A path or url to a tensorflow index checkpoint file (e.g, ./tf_model/model.ckpt.index). In
this case, from_tf should be set to True and a configuration object should be provided as
config argument. This loading path is slower than converting the TensorFlow checkpoint in a
PyTorch model using the provided conversion


 scripts and loading the PyTorch model afterwards.
   model_args (additional positional arguments, optional) —
Will be passed along to the underlying model __init__() method.   config (PretrainedConfig, optional) —
Configuration for the model to use instead of an automatically loaded configuration. Configuration can
be automatically loaded when:
The model is a model provided by the library (loaded with the model id string of a pretrained
model).
The model was saved using save_pretrained() and is reloaded by supplying the
save directory.
The model is loaded by supplying a local directory as pretrained_model_name_or_path and a
configuration JSON file named config.json is found in the directory.
   state_dict (Dict[str, torch.Tensor], optional) —
A state dictionary to use instead of a state dictionary loaded from saved weights file.
This option can be used if you want to create a model from a pretrained configuration but load your own
weights. In this case tho


ugh, you should check if using save_pretrained() and
from_pretrained() is not a simpler option.   cache_dir (str or os.PathLike, optional) —
Path to a directory in which a downloaded pretrained model configuration should be cached if the
standard cache should not be used.   from_tf (bool, optional, defaults to False) —
Load the model weights from a TensorFlow checkpoint save file (see docstring of
pretrained_model_name_or_path argument).   force_download (bool, optional, defaults to False) —
Whether or not to force the (re-)download of the model weights and configuration files, overriding the
cached versions if they exist.   resume_download (bool, optional, defaults to False) —
Whether or not to delete incompletely received files. Will attempt to resume the download if such a
file exists.   proxies (Dict[str, str], optional) —
A dictionary of proxy servers to use by protocol or endpoint, e.g., {'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}. The


 proxies are used on each request.   output_loading_info(bool, optional, defaults to False) —
Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.   local_files_only(bool, optional, defaults to False) —
Whether or not to only look at local files (e.g., not try downloading the model).   revision (str, optional, defaults to "main") —
The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
git-based system for storing models and other artifacts on huggingface.co, so revision can be any
identifier allowed by git.   trust_remote_code (bool, optional, defaults to False) —
Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
should only be set to True for repositories you trust and in which you have read the code, as it will
execute code present on the Hub on your local machine.   code_revision (str, optional, defaults


to "main") —
The specific revision to use for the code on the Hub, if the code leaves in a different repository than
the rest of the model. It can be a branch name, a tag name, or a commit id, since we use a git-based
system for storing models and other artifacts on huggingface.co, so revision can be any identifier
allowed by git.   kwargs (additional keyword arguments, optional) —
Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
output_attentions=True). Behaves differently depending on whether a config is provided or
automatically loaded:
If a configuration is provided with config, **kwargs will be directly passed to the
underlying model’s __init__ method (we assume all relevant updates to the configuration have
already been done)
If a configuration is not provided, kwargs will be first passed to the configuration class
initialization function (from_pretrained()). Each key of kwargs that
corresponds to a


configuration attribute will be used to override said attribute with the
supplied kwargs value. Remaining keys that do not correspond to any configuration attribute
will be passed to the underlying model’s __init__ function.
    Instantiate one of the model classes of the library (with a sequence-to-sequence language modeling head) from a pretrained model. The model class to instantiate is selected based on the model_type property of the config object (either
passed as an argument or loaded from pretrained_model_name_or_path if possible), or when it’s missing, by
falling back to using pattern matching on pretrained_model_name_or_path: bart — BartForConditionalGeneration (BART model) bigbird_pegasus — BigBirdPegasusForConditionalGeneration (BigBird-Pegasus model) blenderbot — BlenderbotForConditionalGeneration (Blenderbot model) blenderbot-small — BlenderbotSmallForConditionalGeneration (BlenderbotSmall model) encoder-decoder — EncoderDecoderModel (Encoder d


ecoder model) fsmt — FSMTForConditionalGeneration (FairSeq Machine-Translation model) gptsan-japanese — GPTSanJapaneseForConditionalGeneration (GPTSAN-japanese model) led — LEDForConditionalGeneration (LED model) longt5 — LongT5ForConditionalGeneration (LongT5 model) m2m_100 — M2M100ForConditionalGeneration (M2M100 model) marian — MarianMTModel (Marian model) mbart — MBartForConditionalGeneration (mBART model) mt5 — MT5ForConditionalGeneration (MT5 model) mvp — MvpForConditionalGeneration (MVP model) nllb-moe — NllbMoeForConditionalGeneration (NLLB-MOE model) pegasus — PegasusForConditionalGeneration (Pegasus model) pegasus_x — PegasusXForConditionalGeneration (PEGASUS-X model) plbart — PLBartForConditionalGeneration (PLBart model) prophetnet — ProphetNetForConditionalGeneration (ProphetNet model) switch_transformers — SwitchTransformersForConditionalGeneration (SwitchTransformers model) t5 — T5ForConditionalGeneration (T5 model) umt5 — UMT5ForConditionalGe


neration (UMT5 model) xlm-prophetnet — XLMProphetNetForConditionalGeneration (XLM-ProphetNet model) The model is set in evaluation mode by default using model.eval() (so for instance, dropout modules are
deactivated). To train the model, you should first set it back in training mode with model.train()  Examples:   Copied >>> from transformers import AutoConfig, AutoModelForSeq2SeqLM
>>> # Download model and configuration from huggingface.co and cache.
>>> model = AutoModelForSeq2SeqLM.from_pretrained("t5-base")
>>> # Update configuration during loading
>>> model = AutoModelForSeq2SeqLM.from_pretrained("t5-base", output_attentions=True)
>>> model.config.output_attentions
True
>>> # Loading from a TF checkpoint file instead of a PyTorch model (slower)
>>> config = AutoConfig.from_pretrained("./tf_model/t5_tf_model_config.json")
>>> model = AutoModelForSeq2SeqLM.from_pretrained(
...     "./tf_model/t5_tf_checkpoint.ckpt.index", from_tf=True, config=config
...


class transformers.TFAutoModelForSeq2SeqLM  < source > ( *args **kwargs  )    This is a generic model class that will be instantiated as one of the model classes of the library (with a sequence-to-sequence language modeling head) when created
with the from_pretrained() class method or the from_config() class
method. This class cannot be instantiated directly using __init__() (throws an error).  from_config  < source > ( **kwargs  )   Parameters   config (PretrainedConfig) —
The model class to instantiate is selected based on the configuration class:
BartConfig configuration class: TFBartForConditionalGeneration (BART model)
BlenderbotConfig configuration class: TFBlenderbotForConditionalGeneration (Blenderbot model)
BlenderbotSmallConfig configuration class: TFBlenderbotSmallForConditionalGeneration (BlenderbotSmall model)
EncoderDecoderConfig configuration class: TFEncoderDecoderModel (Enc


oder decoder model)
LEDConfig configuration class: TFLEDForConditionalGeneration (LED model)
MBartConfig configuration class: TFMBartForConditionalGeneration (mBART model)
MT5Config configuration class: TFMT5ForConditionalGeneration (MT5 model)
MarianConfig configuration class: TFMarianMTModel (Marian model)
PegasusConfig configuration class: TFPegasusForConditionalGeneration (Pegasus model)
T5Config configuration class: TFT5ForConditionalGeneration (T5 model)
    Instantiates one of the model classes of the library (with a sequence-to-sequence language modeling head) from a configuration. Note:
Loading a model from its configuration file does not load the model weights. It only affects the
model’s configuration. Use from_pretrained() to load the model weights.  Examples:   Copied >>> from transformers import AutoConfig, TFAutoModelForSeq2SeqLM
>>> # Download configuration from huggingface.


co and cache.
>>> config = AutoConfig.from_pretrained("t5-base")
>>> model = TFAutoModelForSeq2SeqLM.from_config(config)  from_pretrained  < source > ( *model_args **kwargs  )   Parameters   pretrained_model_name_or_path (str or os.PathLike) —
Can be either:
A string, the model id of a pretrained model hosted inside a model repo on huggingface.co.
Valid model ids can be located at the root-level, like bert-base-uncased, or namespaced under a
user or organization name, like dbmdz/bert-base-german-cased.
A path to a directory containing model weights saved using
save_pretrained(), e.g., ./my_model_directory/.
A path or url to a PyTorch state_dict save file (e.g, ./pt_model/pytorch_model.bin). In this
case, from_pt should be set to True and a configuration object should be provided as config
argument. This loading path is slower than converting the PyTorch model in a TensorFlow model
using the


 provided conversion scripts and loading the TensorFlow model afterwards.
   model_args (additional positional arguments, optional) —
Will be passed along to the underlying model __init__() method.   config (PretrainedConfig, optional) —
Configuration for the model to use instead of an automatically loaded configuration. Configuration can
be automatically loaded when:
The model is a model provided by the library (loaded with the model id string of a pretrained
model).
The model was saved using save_pretrained() and is reloaded by supplying the
save directory.
The model is loaded by supplying a local directory as pretrained_model_name_or_path and a
configuration JSON file named config.json is found in the directory.
   cache_dir (str or os.PathLike, optional) —
Path to a directory in which a downloaded pretrained model configuration should be cached if the
standard cache should not be used.


  from_pt (bool, optional, defaults to False) —
Load the model weights from a PyTorch checkpoint save file (see docstring of
pretrained_model_name_or_path argument).   force_download (bool, optional, defaults to False) —
Whether or not to force the (re-)download of the model weights and configuration files, overriding the
cached versions if they exist.   resume_download (bool, optional, defaults to False) —
Whether or not to delete incompletely received files. Will attempt to resume the download if such a
file exists.   proxies (Dict[str, str], optional) —
A dictionary of proxy servers to use by protocol or endpoint, e.g., {'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}. The proxies are used on each request.   output_loading_info(bool, optional, defaults to False) —
Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.   local_fi


les_only(bool, optional, defaults to False) —
Whether or not to only look at local files (e.g., not try downloading the model).   revision (str, optional, defaults to "main") —
The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
git-based system for storing models and other artifacts on huggingface.co, so revision can be any
identifier allowed by git.   trust_remote_code (bool, optional, defaults to False) —
Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
should only be set to True for repositories you trust and in which you have read the code, as it will
execute code present on the Hub on your local machine.   code_revision (str, optional, defaults to "main") —
The specific revision to use for the code on the Hub, if the code leaves in a different repository than
the rest of the model.


It can be a branch name, a tag name, or a commit id, since we use a git-based
system for storing models and other artifacts on huggingface.co, so revision can be any identifier
allowed by git.   kwargs (additional keyword arguments, optional) —
Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
output_attentions=True). Behaves differently depending on whether a config is provided or
automatically loaded:
If a configuration is provided with config, **kwargs will be directly passed to the
underlying model’s __init__ method (we assume all relevant updates to the configuration have
already been done)
If a configuration is not provided, kwargs will be first passed to the configuration class
initialization function (from_pretrained()). Each key of kwargs that
corresponds to a configuration attribute will be used to override said attribute with the


supplied kwargs value. Remaining keys that do not correspond to any configuration attribute
will be passed to the underlying model’s __init__ function.
    Instantiate one of the model classes of the library (with a sequence-to-sequence language modeling head) from a pretrained model. The model class to instantiate is selected based on the model_type property of the config object (either
passed as an argument or loaded from pretrained_model_name_or_path if possible), or when it’s missing, by
falling back to using pattern matching on pretrained_model_name_or_path: bart — TFBartForConditionalGeneration (BART model) blenderbot — TFBlenderbotForConditionalGeneration (Blenderbot model) blenderbot-small — TFBlenderbotSmallForConditionalGeneration (BlenderbotSmall model) encoder-decoder — TFEncoderDecoderModel (Encoder decoder model) led — TFLEDForConditionalGeneration (LED model) marian — TFMari


anMTModel (Marian model) mbart — TFMBartForConditionalGeneration (mBART model) mt5 — TFMT5ForConditionalGeneration (MT5 model) pegasus — TFPegasusForConditionalGeneration (Pegasus model) t5 — TFT5ForConditionalGeneration (T5 model)  Examples:   Copied >>> from transformers import AutoConfig, TFAutoModelForSeq2SeqLM
>>> # Download model and configuration from huggingface.co and cache.
>>> model = TFAutoModelForSeq2SeqLM.from_pretrained("t5-base")
>>> # Update configuration during loading
>>> model = TFAutoModelForSeq2SeqLM.from_pretrained("t5-base", output_attentions=True)
>>> model.config.output_attentions
True
>>> # Loading from a PyTorch checkpoint file instead of a TensorFlow model (slower)
>>> config = AutoConfig.from_pretrained("./pt_model/t5_pt_model_config.json")
>>> model = TFAutoModelForSeq2SeqLM.from_pretrained(
...     "./pt_model/t5_pytorch_model.bin", from_pt=True, config=confi


class transformers.FlaxAutoModelForSeq2SeqLM  < source > ( *args **kwargs  )    This is a generic model class that will be instantiated as one of the model classes of the library (with a sequence-to-sequence language modeling head) when created
with the from_pretrained() class method or the from_config() class
method. This class cannot be instantiated directly using __init__() (throws an error).  from_config  < source > ( **kwargs  )   Parameters   config (PretrainedConfig) —
The model class to instantiate is selected based on the configuration class:
BartConfig configuration class: FlaxBartForConditionalGeneration (BART model)
BlenderbotConfig configuration class: FlaxBlenderbotForConditionalGeneration (Blenderbot model)
BlenderbotSmallConfig configuration class: FlaxBlenderbotSmallForConditionalGeneration (BlenderbotSmall model)
EncoderDecoderConfig configuration class: FlaxEncoderDecoderModel (E


ncoder decoder model)
LongT5Config configuration class: FlaxLongT5ForConditionalGeneration (LongT5 model)
MBartConfig configuration class: FlaxMBartForConditionalGeneration (mBART model)
MT5Config configuration class: FlaxMT5ForConditionalGeneration (MT5 model)
MarianConfig configuration class: FlaxMarianMTModel (Marian model)
PegasusConfig configuration class: FlaxPegasusForConditionalGeneration (Pegasus model)
T5Config configuration class: FlaxT5ForConditionalGeneration (T5 model)
    Instantiates one of the model classes of the library (with a sequence-to-sequence language modeling head) from a configuration. Note:
Loading a model from its configuration file does not load the model weights. It only affects the
model’s configuration. Use from_pretrained() to load the model weights.  Examples:   Copied >>> from transformers import AutoConfig, FlaxAutoModelForSeq2SeqLM
>>> # Download configuration


from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("t5-base")
>>> model = FlaxAutoModelForSeq2SeqLM.from_config(config)  from_pretrained  < source > ( *model_args **kwargs  )   Parameters   pretrained_model_name_or_path (str or os.PathLike) —
Can be either:
A string, the model id of a pretrained model hosted inside a model repo on huggingface.co.
Valid model ids can be located at the root-level, like bert-base-uncased, or namespaced under a
user or organization name, like dbmdz/bert-base-german-cased.
A path to a directory containing model weights saved using
save_pretrained(), e.g., ./my_model_directory/.
A path or url to a PyTorch state_dict save file (e.g, ./pt_model/pytorch_model.bin). In this
case, from_pt should be set to True and a configuration object should be provided as config
argument. This loading path is slower than converting the PyTorch model in a TensorFlow mode


l
using the provided conversion scripts and loading the TensorFlow model afterwards.
   model_args (additional positional arguments, optional) —
Will be passed along to the underlying model __init__() method.   config (PretrainedConfig, optional) —
Configuration for the model to use instead of an automatically loaded configuration. Configuration can
be automatically loaded when:
The model is a model provided by the library (loaded with the model id string of a pretrained
model).
The model was saved using save_pretrained() and is reloaded by supplying the
save directory.
The model is loaded by supplying a local directory as pretrained_model_name_or_path and a
configuration JSON file named config.json is found in the directory.
   cache_dir (str or os.PathLike, optional) —
Path to a directory in which a downloaded pretrained model configuration should be cached if the
standard cache should not be use


d.   from_pt (bool, optional, defaults to False) —
Load the model weights from a PyTorch checkpoint save file (see docstring of
pretrained_model_name_or_path argument).   force_download (bool, optional, defaults to False) —
Whether or not to force the (re-)download of the model weights and configuration files, overriding the
cached versions if they exist.   resume_download (bool, optional, defaults to False) —
Whether or not to delete incompletely received files. Will attempt to resume the download if such a
file exists.   proxies (Dict[str, str], optional) —
A dictionary of proxy servers to use by protocol or endpoint, e.g., {'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}. The proxies are used on each request.   output_loading_info(bool, optional, defaults to False) —
Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.   local_files_o


nly(bool, optional, defaults to False) —
Whether or not to only look at local files (e.g., not try downloading the model).   revision (str, optional, defaults to "main") —
The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
git-based system for storing models and other artifacts on huggingface.co, so revision can be any
identifier allowed by git.   trust_remote_code (bool, optional, defaults to False) —
Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
should only be set to True for repositories you trust and in which you have read the code, as it will
execute code present on the Hub on your local machine.   code_revision (str, optional, defaults to "main") —
The specific revision to use for the code on the Hub, if the code leaves in a different repository than
the rest of the model. It can be a b


ranch name, a tag name, or a commit id, since we use a git-based
system for storing models and other artifacts on huggingface.co, so revision can be any identifier
allowed by git.   kwargs (additional keyword arguments, optional) —
Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
output_attentions=True). Behaves differently depending on whether a config is provided or
automatically loaded:
If a configuration is provided with config, **kwargs will be directly passed to the
underlying model’s __init__ method (we assume all relevant updates to the configuration have
already been done)
If a configuration is not provided, kwargs will be first passed to the configuration class
initialization function (from_pretrained()). Each key of kwargs that
corresponds to a configuration attribute will be used to override said attribute with the
supplied kwargs valu


e. Remaining keys that do not correspond to any configuration attribute
will be passed to the underlying model’s __init__ function.
    Instantiate one of the model classes of the library (with a sequence-to-sequence language modeling head) from a pretrained model. The model class to instantiate is selected based on the model_type property of the config object (either
passed as an argument or loaded from pretrained_model_name_or_path if possible), or when it’s missing, by
falling back to using pattern matching on pretrained_model_name_or_path: bart — FlaxBartForConditionalGeneration (BART model) blenderbot — FlaxBlenderbotForConditionalGeneration (Blenderbot model) blenderbot-small — FlaxBlenderbotSmallForConditionalGeneration (BlenderbotSmall model) encoder-decoder — FlaxEncoderDecoderModel (Encoder decoder model) longt5 — FlaxLongT5ForConditionalGeneration (LongT5 model) marian — FlaxMarianMTMode


l (Marian model) mbart — FlaxMBartForConditionalGeneration (mBART model) mt5 — FlaxMT5ForConditionalGeneration (MT5 model) pegasus — FlaxPegasusForConditionalGeneration (Pegasus model) t5 — FlaxT5ForConditionalGeneration (T5 model)  Examples:   Copied >>> from transformers import AutoConfig, FlaxAutoModelForSeq2SeqLM
>>> # Download model and configuration from huggingface.co and cache.
>>> model = FlaxAutoModelForSeq2SeqLM.from_pretrained("t5-base")
>>> # Update configuration during loading
>>> model = FlaxAutoModelForSeq2SeqLM.from_pretrained("t5-base", output_attentions=True)
>>> model.config.output_attentions
True
>>> # Loading from a PyTorch checkpoint file instead of a TensorFlow model (slower)
>>> config = AutoConfig.from_pretrained("./pt_model/t5_pt_model_config.json")
>>> model = FlaxAutoModelForSeq2SeqLM.from_pretrained(
...     "./pt_model/t5_pytorch_model.bin", from_pt=True, config=confi


AutoModelForSequenceClassification

class transformers.AutoModelForSequenceClassification  < source > ( *args **kwargs  )    This is a generic model class that will be instantiated as one of the model classes of the library (with a sequence classification head) when created
with the from_pretrained() class method or the from_config() class
method. This class cannot be instantiated directly using __init__() (throws an error).  from_config  < source > ( **kwargs  )   Parameters   config (PretrainedConfig) —
The model class to instantiate is selected based on the configuration class:
AlbertConfig configuration class: AlbertForSequenceClassification (ALBERT model)
BartConfig configuration class: BartForSequenceClassification (BART model)
BertConfig configuration class: BertForSequenceClassification (BERT model)
BigBirdConfig configuration class: BigBirdForSequenceClassification (BigBird model)
BigBirdPegasusConfig configuration class: BigBirdPegasusForSequenceClassification (Bi


AutoModelForSequenceClassification

gBird-Pegasus model)
BioGptConfig configuration class: BioGptForSequenceClassification (BioGpt model)
BloomConfig configuration class: BloomForSequenceClassification (BLOOM model)
CTRLConfig configuration class: CTRLForSequenceClassification (CTRL model)
CamembertConfig configuration class: CamembertForSequenceClassification (CamemBERT model)
CanineConfig configuration class: CanineForSequenceClassification (CANINE model)
ConvBertConfig configuration class: ConvBertForSequenceClassification (ConvBERT model)
Data2VecTextConfig configuration class: Data2VecTextForSequenceClassification (Data2VecText model)
DebertaConfig configuration class: DebertaForSequenceClassification (DeBERTa model)
DebertaV2Config configuration class: DebertaV2ForSequenceClassification (DeBERTa-v2 model)
DistilBertConfig configuration class: DistilBertForSequenceClassification (DistilBERT model)
ElectraConfig configuration class: ElectraForSequenceClassification (ELE


AutoModelForSequenceClassification

CTRA model)
ErnieConfig configuration class: ErnieForSequenceClassification (ERNIE model)
ErnieMConfig configuration class: ErnieMForSequenceClassification (ErnieM model)
EsmConfig configuration class: EsmForSequenceClassification (ESM model)
FNetConfig configuration class: FNetForSequenceClassification (FNet model)
FalconConfig configuration class: FalconForSequenceClassification (Falcon model)
FlaubertConfig configuration class: FlaubertForSequenceClassification (FlauBERT model)
FunnelConfig configuration class: FunnelForSequenceClassification (Funnel Transformer model)
GPT2Config configuration class: GPT2ForSequenceClassification (OpenAI GPT-2 model)
GPTBigCodeConfig configuration class: GPTBigCodeForSequenceClassification (GPTBigCode model)
GPTJConfig configuration class: GPTJForSequenceClassification (GPT-J model)
GPTNeoConfig configuration class: GPTNeoForSequenceClassification (GPT Neo model)
GPTNeoXConfig configuration class: GPTN


AutoModelForSequenceClassification

eoXForSequenceClassification (GPT NeoX model)
IBertConfig configuration class: IBertForSequenceClassification (I-BERT model)
LEDConfig configuration class: LEDForSequenceClassification (LED model)
LayoutLMConfig configuration class: LayoutLMForSequenceClassification (LayoutLM model)
LayoutLMv2Config configuration class: LayoutLMv2ForSequenceClassification (LayoutLMv2 model)
LayoutLMv3Config configuration class: LayoutLMv3ForSequenceClassification (LayoutLMv3 model)
LiltConfig configuration class: LiltForSequenceClassification (LiLT model)
LlamaConfig configuration class: LlamaForSequenceClassification (LLaMA model)
LongformerConfig configuration class: LongformerForSequenceClassification (Longformer model)
LukeConfig configuration class: LukeForSequenceClassification (LUKE model)
MBartConfig configuration class: MBartForSequenceClassification (mBART model)
MPNetConfig configuration class: MPNetForSequenceClassification (MPNet model)
MT5Co


AutoModelForSequenceClassification

nfig configuration class: MT5ForSequenceClassification (MT5 model)
MarkupLMConfig configuration class: MarkupLMForSequenceClassification (MarkupLM model)
MegaConfig configuration class: MegaForSequenceClassification (MEGA model)
MegatronBertConfig configuration class: MegatronBertForSequenceClassification (Megatron-BERT model)
MistralConfig configuration class: MistralForSequenceClassification (Mistral model)
MobileBertConfig configuration class: MobileBertForSequenceClassification (MobileBERT model)
MptConfig configuration class: MptForSequenceClassification (MPT model)
MraConfig configuration class: MraForSequenceClassification (MRA model)
MvpConfig configuration class: MvpForSequenceClassification (MVP model)
NezhaConfig configuration class: NezhaForSequenceClassification (Nezha model)
NystromformerConfig configuration class: NystromformerForSequenceClassification (Nyströmformer model)
OPTConfig configuration class: OPTForSequenceClass


AutoModelForSequenceClassification

ification (OPT model)
OpenAIGPTConfig configuration class: OpenAIGPTForSequenceClassification (OpenAI GPT model)
OpenLlamaConfig configuration class: OpenLlamaForSequenceClassification (OpenLlama model)
PLBartConfig configuration class: PLBartForSequenceClassification (PLBart model)
PerceiverConfig configuration class: PerceiverForSequenceClassification (Perceiver model)
PersimmonConfig configuration class: PersimmonForSequenceClassification (Persimmon model)
QDQBertConfig configuration class: QDQBertForSequenceClassification (QDQBert model)
ReformerConfig configuration class: ReformerForSequenceClassification (Reformer model)
RemBertConfig configuration class: RemBertForSequenceClassification (RemBERT model)
RoCBertConfig configuration class: RoCBertForSequenceClassification (RoCBert model)
RoFormerConfig configuration class: RoFormerForSequenceClassification (RoFormer model)
RobertaConfig configuration class: RobertaForSequenceClassific


AutoModelForSequenceClassification

ation (RoBERTa model)
RobertaPreLayerNormConfig configuration class: RobertaPreLayerNormForSequenceClassification (RoBERTa-PreLayerNorm model)
SqueezeBertConfig configuration class: SqueezeBertForSequenceClassification (SqueezeBERT model)
T5Config configuration class: T5ForSequenceClassification (T5 model)
TapasConfig configuration class: TapasForSequenceClassification (TAPAS model)
TransfoXLConfig configuration class: TransfoXLForSequenceClassification (Transformer-XL model)
UMT5Config configuration class: UMT5ForSequenceClassification (UMT5 model)
XLMConfig configuration class: XLMForSequenceClassification (XLM model)
XLMRobertaConfig configuration class: XLMRobertaForSequenceClassification (XLM-RoBERTa model)
XLMRobertaXLConfig configuration class: XLMRobertaXLForSequenceClassification (XLM-RoBERTa-XL model)
XLNetConfig configuration class: XLNetForSequenceClassification (XLNet model)
XmodConfig configuration class: XmodForSequenceClas


AutoModelForSequenceClassification

sification (X-MOD model)
YosoConfig configuration class: YosoForSequenceClassification (YOSO model)
    Instantiates one of the model classes of the library (with a sequence classification head) from a configuration. Note:
Loading a model from its configuration file does not load the model weights. It only affects the
model’s configuration. Use from_pretrained() to load the model weights.  Examples:   Copied >>> from transformers import AutoConfig, AutoModelForSequenceClassification
>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = AutoModelForSequenceClassification.from_config(config)  from_pretrained  < source > ( *model_args **kwargs  )   Parameters   pretrained_model_name_or_path (str or os.PathLike) —
Can be either:
A string, the model id of a pretrained model hosted inside a model repo on huggingface.co.
Valid model ids can be located at the root-level,


AutoModelForSequenceClassification

 like bert-base-uncased, or namespaced under a
user or organization name, like dbmdz/bert-base-german-cased.
A path to a directory containing model weights saved using
save_pretrained(), e.g., ./my_model_directory/.
A path or url to a tensorflow index checkpoint file (e.g, ./tf_model/model.ckpt.index). In
this case, from_tf should be set to True and a configuration object should be provided as
config argument. This loading path is slower than converting the TensorFlow checkpoint in a
PyTorch model using the provided conversion scripts and loading the PyTorch model afterwards.
   model_args (additional positional arguments, optional) —
Will be passed along to the underlying model __init__() method.   config (PretrainedConfig, optional) —
Configuration for the model to use instead of an automatically loaded configuration. Configuration can
be automatically loaded when:
The model is a model provided by the library (loaded with the model id s


AutoModelForSequenceClassification

tring of a pretrained
model).
The model was saved using save_pretrained() and is reloaded by supplying the
save directory.
The model is loaded by supplying a local directory as pretrained_model_name_or_path and a
configuration JSON file named config.json is found in the directory.
   state_dict (Dict[str, torch.Tensor], optional) —
A state dictionary to use instead of a state dictionary loaded from saved weights file.
This option can be used if you want to create a model from a pretrained configuration but load your own
weights. In this case though, you should check if using save_pretrained() and
from_pretrained() is not a simpler option.   cache_dir (str or os.PathLike, optional) —
Path to a directory in which a downloaded pretrained model configuration should be cached if the
standard cache should not be used.   from_tf (bool, optional, defaults to False) —
Load the model weights from a TensorFlow checkpoint save file (see docstring of


AutoModelForSequenceClassification

pretrained_model_name_or_path argument).   force_download (bool, optional, defaults to False) —
Whether or not to force the (re-)download of the model weights and configuration files, overriding the
cached versions if they exist.   resume_download (bool, optional, defaults to False) —
Whether or not to delete incompletely received files. Will attempt to resume the download if such a
file exists.   proxies (Dict[str, str], optional) —
A dictionary of proxy servers to use by protocol or endpoint, e.g., {'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}. The proxies are used on each request.   output_loading_info(bool, optional, defaults to False) —
Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.   local_files_only(bool, optional, defaults to False) —
Whether or not to only look at local files (e.g., not try downloading the model).   revision (str, optional, defaults to "main")


AutoModelForSequenceClassification

 —
The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
git-based system for storing models and other artifacts on huggingface.co, so revision can be any
identifier allowed by git.   trust_remote_code (bool, optional, defaults to False) —
Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
should only be set to True for repositories you trust and in which you have read the code, as it will
execute code present on the Hub on your local machine.   code_revision (str, optional, defaults to "main") —
The specific revision to use for the code on the Hub, if the code leaves in a different repository than
the rest of the model. It can be a branch name, a tag name, or a commit id, since we use a git-based
system for storing models and other artifacts on huggingface.co, so revision can be any identifier
allowed by git.   kwargs (additional keyword a


AutoModelForSequenceClassification

rguments, optional) —
Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
output_attentions=True). Behaves differently depending on whether a config is provided or
automatically loaded:
If a configuration is provided with config, **kwargs will be directly passed to the
underlying model’s __init__ method (we assume all relevant updates to the configuration have
already been done)
If a configuration is not provided, kwargs will be first passed to the configuration class
initialization function (from_pretrained()). Each key of kwargs that
corresponds to a configuration attribute will be used to override said attribute with the
supplied kwargs value. Remaining keys that do not correspond to any configuration attribute
will be passed to the underlying model’s __init__ function.
    Instantiate one of the model classes of the library (with a sequence classification head) from a pretrained model.


AutoModelForSequenceClassification

The model class to instantiate is selected based on the model_type property of the config object (either
passed as an argument or loaded from pretrained_model_name_or_path if possible), or when it’s missing, by
falling back to using pattern matching on pretrained_model_name_or_path: albert — AlbertForSequenceClassification (ALBERT model) bart — BartForSequenceClassification (BART model) bert — BertForSequenceClassification (BERT model) big_bird — BigBirdForSequenceClassification (BigBird model) bigbird_pegasus — BigBirdPegasusForSequenceClassification (BigBird-Pegasus model) biogpt — BioGptForSequenceClassification (BioGpt model) bloom — BloomForSequenceClassification (BLOOM model) camembert — CamembertForSequenceClassification (CamemBERT model) canine — CanineForSequenceClassification (CANINE model) code_llama — LlamaForSequenceClassification (CodeLlama model) convbert — ConvBertForSequenceClassification (ConvBERT model) ctrl — CTRLForSe


AutoModelForSequenceClassification

quenceClassification (CTRL model) data2vec-text — Data2VecTextForSequenceClassification (Data2VecText model) deberta — DebertaForSequenceClassification (DeBERTa model) deberta-v2 — DebertaV2ForSequenceClassification (DeBERTa-v2 model) distilbert — DistilBertForSequenceClassification (DistilBERT model) electra — ElectraForSequenceClassification (ELECTRA model) ernie — ErnieForSequenceClassification (ERNIE model) ernie_m — ErnieMForSequenceClassification (ErnieM model) esm — EsmForSequenceClassification (ESM model) falcon — FalconForSequenceClassification (Falcon model) flaubert — FlaubertForSequenceClassification (FlauBERT model) fnet — FNetForSequenceClassification (FNet model) funnel — FunnelForSequenceClassification (Funnel Transformer model) gpt-sw3 — GPT2ForSequenceClassification (GPT-Sw3 model) gpt2 — GPT2ForSequenceClassification (OpenAI GPT-2 model) gpt_bigcode — GPTBigCodeForSequenceClassification (GPTBigCode model) gpt_neo — GPTN


AutoModelForSequenceClassification

eoForSequenceClassification (GPT Neo model) gpt_neox — GPTNeoXForSequenceClassification (GPT NeoX model) gptj — GPTJForSequenceClassification (GPT-J model) ibert — IBertForSequenceClassification (I-BERT model) layoutlm — LayoutLMForSequenceClassification (LayoutLM model) layoutlmv2 — LayoutLMv2ForSequenceClassification (LayoutLMv2 model) layoutlmv3 — LayoutLMv3ForSequenceClassification (LayoutLMv3 model) led — LEDForSequenceClassification (LED model) lilt — LiltForSequenceClassification (LiLT model) llama — LlamaForSequenceClassification (LLaMA model) longformer — LongformerForSequenceClassification (Longformer model) luke — LukeForSequenceClassification (LUKE model) markuplm — MarkupLMForSequenceClassification (MarkupLM model) mbart — MBartForSequenceClassification (mBART model) mega — MegaForSequenceClassification (MEGA model) megatron-bert — MegatronBertForSequenceClassification (Megatron-BERT model) mistral — MistralForSequenceClassif


AutoModelForSequenceClassification

ication (Mistral model) mobilebert — MobileBertForSequenceClassification (MobileBERT model) mpnet — MPNetForSequenceClassification (MPNet model) mpt — MptForSequenceClassification (MPT model) mra — MraForSequenceClassification (MRA model) mt5 — MT5ForSequenceClassification (MT5 model) mvp — MvpForSequenceClassification (MVP model) nezha — NezhaForSequenceClassification (Nezha model) nystromformer — NystromformerForSequenceClassification (Nyströmformer model) open-llama — OpenLlamaForSequenceClassification (OpenLlama model) openai-gpt — OpenAIGPTForSequenceClassification (OpenAI GPT model) opt — OPTForSequenceClassification (OPT model) perceiver — PerceiverForSequenceClassification (Perceiver model) persimmon — PersimmonForSequenceClassification (Persimmon model) plbart — PLBartForSequenceClassification (PLBart model) qdqbert — QDQBertForSequenceClassification (QDQBert model) reformer — ReformerForSequenceClassification (Reformer model) re


AutoModelForSequenceClassification

mbert — RemBertForSequenceClassification (RemBERT model) roberta — RobertaForSequenceClassification (RoBERTa model) roberta-prelayernorm — RobertaPreLayerNormForSequenceClassification (RoBERTa-PreLayerNorm model) roc_bert — RoCBertForSequenceClassification (RoCBert model) roformer — RoFormerForSequenceClassification (RoFormer model) squeezebert — SqueezeBertForSequenceClassification (SqueezeBERT model) t5 — T5ForSequenceClassification (T5 model) tapas — TapasForSequenceClassification (TAPAS model) transfo-xl — TransfoXLForSequenceClassification (Transformer-XL model) umt5 — UMT5ForSequenceClassification (UMT5 model) xlm — XLMForSequenceClassification (XLM model) xlm-roberta — XLMRobertaForSequenceClassification (XLM-RoBERTa model) xlm-roberta-xl — XLMRobertaXLForSequenceClassification (XLM-RoBERTa-XL model) xlnet — XLNetForSequenceClassification (XLNet model) xmod — XmodForSequenceClassification (X-MOD model) yoso — YosoForSequenceClassif


AutoModelForSequenceClassification

ication (YOSO model) The model is set in evaluation mode by default using model.eval() (so for instance, dropout modules are
deactivated). To train the model, you should first set it back in training mode with model.train()  Examples:   Copied >>> from transformers import AutoConfig, AutoModelForSequenceClassification
>>> # Download model and configuration from huggingface.co and cache.
>>> model = AutoModelForSequenceClassification.from_pretrained("bert-base-cased")
>>> # Update configuration during loading
>>> model = AutoModelForSequenceClassification.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True
>>> # Loading from a TF checkpoint file instead of a PyTorch model (slower)
>>> config = AutoConfig.from_pretrained("./tf_model/bert_tf_model_config.json")
>>> model = AutoModelForSequenceClassification.from_pretrained(
...     "./tf_model/bert_tf_checkpoint.ckpt.index", from_tf=True, config


TFAutoModelForSequenceClassification

class transformers.TFAutoModelForSequenceClassification  < source > ( *args **kwargs  )    This is a generic model class that will be instantiated as one of the model classes of the library (with a sequence classification head) when created
with the from_pretrained() class method or the from_config() class
method. This class cannot be instantiated directly using __init__() (throws an error).  from_config  < source > ( **kwargs  )   Parameters   config (PretrainedConfig) —
The model class to instantiate is selected based on the configuration class:
AlbertConfig configuration class: TFAlbertForSequenceClassification (ALBERT model)
BartConfig configuration class: TFBartForSequenceClassification (BART model)
BertConfig configuration class: TFBertForSequenceClassification (BERT model)
CTRLConfig configuration class: TFCTRLForSequenceClassification (CTRL model)
CamembertConfig configuration class: TFCamembertForSequenceClassification (CamemBER


TFAutoModelForSequenceClassification

T model)
ConvBertConfig configuration class: TFConvBertForSequenceClassification (ConvBERT model)
DebertaConfig configuration class: TFDebertaForSequenceClassification (DeBERTa model)
DebertaV2Config configuration class: TFDebertaV2ForSequenceClassification (DeBERTa-v2 model)
DistilBertConfig configuration class: TFDistilBertForSequenceClassification (DistilBERT model)
ElectraConfig configuration class: TFElectraForSequenceClassification (ELECTRA model)
EsmConfig configuration class: TFEsmForSequenceClassification (ESM model)
FlaubertConfig configuration class: TFFlaubertForSequenceClassification (FlauBERT model)
FunnelConfig configuration class: TFFunnelForSequenceClassification (Funnel Transformer model)
GPT2Config configuration class: TFGPT2ForSequenceClassification (OpenAI GPT-2 model)
GPTJConfig configuration class: TFGPTJForSequenceClassification (GPT-J model)
LayoutLMConfig configuration class: TFLayoutLMForSequenceClassification


TFAutoModelForSequenceClassification

(LayoutLM model)
LayoutLMv3Config configuration class: TFLayoutLMv3ForSequenceClassification (LayoutLMv3 model)
LongformerConfig configuration class: TFLongformerForSequenceClassification (Longformer model)
MPNetConfig configuration class: TFMPNetForSequenceClassification (MPNet model)
MobileBertConfig configuration class: TFMobileBertForSequenceClassification (MobileBERT model)
OpenAIGPTConfig configuration class: TFOpenAIGPTForSequenceClassification (OpenAI GPT model)
RemBertConfig configuration class: TFRemBertForSequenceClassification (RemBERT model)
RoFormerConfig configuration class: TFRoFormerForSequenceClassification (RoFormer model)
RobertaConfig configuration class: TFRobertaForSequenceClassification (RoBERTa model)
RobertaPreLayerNormConfig configuration class: TFRobertaPreLayerNormForSequenceClassification (RoBERTa-PreLayerNorm model)
TapasConfig configuration class: TFTapasForSequenceClassification (TAPAS model)
TransfoXLCon


TFAutoModelForSequenceClassification

fig configuration class: TFTransfoXLForSequenceClassification (Transformer-XL model)
XLMConfig configuration class: TFXLMForSequenceClassification (XLM model)
XLMRobertaConfig configuration class: TFXLMRobertaForSequenceClassification (XLM-RoBERTa model)
XLNetConfig configuration class: TFXLNetForSequenceClassification (XLNet model)
    Instantiates one of the model classes of the library (with a sequence classification head) from a configuration. Note:
Loading a model from its configuration file does not load the model weights. It only affects the
model’s configuration. Use from_pretrained() to load the model weights.  Examples:   Copied >>> from transformers import AutoConfig, TFAutoModelForSequenceClassification
>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = TFAutoModelForSequenceClassification.from_config(config)  from_pretrained  < source > ( *model_


TFAutoModelForSequenceClassification

args **kwargs  )   Parameters   pretrained_model_name_or_path (str or os.PathLike) —
Can be either:
A string, the model id of a pretrained model hosted inside a model repo on huggingface.co.
Valid model ids can be located at the root-level, like bert-base-uncased, or namespaced under a
user or organization name, like dbmdz/bert-base-german-cased.
A path to a directory containing model weights saved using
save_pretrained(), e.g., ./my_model_directory/.
A path or url to a PyTorch state_dict save file (e.g, ./pt_model/pytorch_model.bin). In this
case, from_pt should be set to True and a configuration object should be provided as config
argument. This loading path is slower than converting the PyTorch model in a TensorFlow model
using the provided conversion scripts and loading the TensorFlow model afterwards.
   model_args (additional positional arguments, optional) —
Will be passed along to the underlying model __init__() method.   config


TFAutoModelForSequenceClassification

(PretrainedConfig, optional) —
Configuration for the model to use instead of an automatically loaded configuration. Configuration can
be automatically loaded when:
The model is a model provided by the library (loaded with the model id string of a pretrained
model).
The model was saved using save_pretrained() and is reloaded by supplying the
save directory.
The model is loaded by supplying a local directory as pretrained_model_name_or_path and a
configuration JSON file named config.json is found in the directory.
   cache_dir (str or os.PathLike, optional) —
Path to a directory in which a downloaded pretrained model configuration should be cached if the
standard cache should not be used.   from_pt (bool, optional, defaults to False) —
Load the model weights from a PyTorch checkpoint save file (see docstring of
pretrained_model_name_or_path argument).   force_download (bool, optional, defaults to False) —
Whether or not to force the (re-)d


TFAutoModelForSequenceClassification

ownload of the model weights and configuration files, overriding the
cached versions if they exist.   resume_download (bool, optional, defaults to False) —
Whether or not to delete incompletely received files. Will attempt to resume the download if such a
file exists.   proxies (Dict[str, str], optional) —
A dictionary of proxy servers to use by protocol or endpoint, e.g., {'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}. The proxies are used on each request.   output_loading_info(bool, optional, defaults to False) —
Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.   local_files_only(bool, optional, defaults to False) —
Whether or not to only look at local files (e.g., not try downloading the model).   revision (str, optional, defaults to "main") —
The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
git-based system for st


TFAutoModelForSequenceClassification

oring models and other artifacts on huggingface.co, so revision can be any
identifier allowed by git.   trust_remote_code (bool, optional, defaults to False) —
Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
should only be set to True for repositories you trust and in which you have read the code, as it will
execute code present on the Hub on your local machine.   code_revision (str, optional, defaults to "main") —
The specific revision to use for the code on the Hub, if the code leaves in a different repository than
the rest of the model. It can be a branch name, a tag name, or a commit id, since we use a git-based
system for storing models and other artifacts on huggingface.co, so revision can be any identifier
allowed by git.   kwargs (additional keyword arguments, optional) —
Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
outpu


TFAutoModelForSequenceClassification

t_attentions=True). Behaves differently depending on whether a config is provided or
automatically loaded:
If a configuration is provided with config, **kwargs will be directly passed to the
underlying model’s __init__ method (we assume all relevant updates to the configuration have
already been done)
If a configuration is not provided, kwargs will be first passed to the configuration class
initialization function (from_pretrained()). Each key of kwargs that
corresponds to a configuration attribute will be used to override said attribute with the
supplied kwargs value. Remaining keys that do not correspond to any configuration attribute
will be passed to the underlying model’s __init__ function.
    Instantiate one of the model classes of the library (with a sequence classification head) from a pretrained model. The model class to instantiate is selected based on the model_type property of the config object (either
passed as an argument


TFAutoModelForSequenceClassification

or loaded from pretrained_model_name_or_path if possible), or when it’s missing, by
falling back to using pattern matching on pretrained_model_name_or_path: albert — TFAlbertForSequenceClassification (ALBERT model) bart — TFBartForSequenceClassification (BART model) bert — TFBertForSequenceClassification (BERT model) camembert — TFCamembertForSequenceClassification (CamemBERT model) convbert — TFConvBertForSequenceClassification (ConvBERT model) ctrl — TFCTRLForSequenceClassification (CTRL model) deberta — TFDebertaForSequenceClassification (DeBERTa model) deberta-v2 — TFDebertaV2ForSequenceClassification (DeBERTa-v2 model) distilbert — TFDistilBertForSequenceClassification (DistilBERT model) electra — TFElectraForSequenceClassification (ELECTRA model) esm — TFEsmForSequenceClassification (ESM model) flaubert — TFFlaubertForSequenceClassification (FlauBERT model) funnel — TFFunnelForSequenceClassification (Funnel Transformer model) gpt-s


TFAutoModelForSequenceClassification

w3 — TFGPT2ForSequenceClassification (GPT-Sw3 model) gpt2 — TFGPT2ForSequenceClassification (OpenAI GPT-2 model) gptj — TFGPTJForSequenceClassification (GPT-J model) layoutlm — TFLayoutLMForSequenceClassification (LayoutLM model) layoutlmv3 — TFLayoutLMv3ForSequenceClassification (LayoutLMv3 model) longformer — TFLongformerForSequenceClassification (Longformer model) mobilebert — TFMobileBertForSequenceClassification (MobileBERT model) mpnet — TFMPNetForSequenceClassification (MPNet model) openai-gpt — TFOpenAIGPTForSequenceClassification (OpenAI GPT model) rembert — TFRemBertForSequenceClassification (RemBERT model) roberta — TFRobertaForSequenceClassification (RoBERTa model) roberta-prelayernorm — TFRobertaPreLayerNormForSequenceClassification (RoBERTa-PreLayerNorm model) roformer — TFRoFormerForSequenceClassification (RoFormer model) tapas — TFTapasForSequenceClassification (TAPAS model) transfo-xl — TFTransfoXLForSequenceClassificati


TFAutoModelForSequenceClassification

on (Transformer-XL model) xlm — TFXLMForSequenceClassification (XLM model) xlm-roberta — TFXLMRobertaForSequenceClassification (XLM-RoBERTa model) xlnet — TFXLNetForSequenceClassification (XLNet model)  Examples:   Copied >>> from transformers import AutoConfig, TFAutoModelForSequenceClassification
>>> # Download model and configuration from huggingface.co and cache.
>>> model = TFAutoModelForSequenceClassification.from_pretrained("bert-base-cased")
>>> # Update configuration during loading
>>> model = TFAutoModelForSequenceClassification.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True
>>> # Loading from a PyTorch checkpoint file instead of a TensorFlow model (slower)
>>> config = AutoConfig.from_pretrained("./pt_model/bert_pt_model_config.json")
>>> model = TFAutoModelForSequenceClassification.from_pretrained(
...     "./pt_model/bert_pytorch_model.bin", from_pt=True, config=config
...


FlaxAutoModelForSequenceClassification

class transformers.FlaxAutoModelForSequenceClassification  < source > ( *args **kwargs  )    This is a generic model class that will be instantiated as one of the model classes of the library (with a sequence classification head) when created
with the from_pretrained() class method or the from_config() class
method. This class cannot be instantiated directly using __init__() (throws an error).  from_config  < source > ( **kwargs  )   Parameters   config (PretrainedConfig) —
The model class to instantiate is selected based on the configuration class:
AlbertConfig configuration class: FlaxAlbertForSequenceClassification (ALBERT model)
BartConfig configuration class: FlaxBartForSequenceClassification (BART model)
BertConfig configuration class: FlaxBertForSequenceClassification (BERT model)
BigBirdConfig configuration class: FlaxBigBirdForSequenceClassification (BigBird model)
DistilBertConfig configuration class: FlaxDistilBertForSequenceCl


FlaxAutoModelForSequenceClassification

assification (DistilBERT model)
ElectraConfig configuration class: FlaxElectraForSequenceClassification (ELECTRA model)
MBartConfig configuration class: FlaxMBartForSequenceClassification (mBART model)
RoFormerConfig configuration class: FlaxRoFormerForSequenceClassification (RoFormer model)
RobertaConfig configuration class: FlaxRobertaForSequenceClassification (RoBERTa model)
RobertaPreLayerNormConfig configuration class: FlaxRobertaPreLayerNormForSequenceClassification (RoBERTa-PreLayerNorm model)
XLMRobertaConfig configuration class: FlaxXLMRobertaForSequenceClassification (XLM-RoBERTa model)
    Instantiates one of the model classes of the library (with a sequence classification head) from a configuration. Note:
Loading a model from its configuration file does not load the model weights. It only affects the
model’s configuration. Use from_pretrained() to load the model weights.  Examples:   Copied >>> from transformers import AutoCon


FlaxAutoModelForSequenceClassification

fig, FlaxAutoModelForSequenceClassification
>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = FlaxAutoModelForSequenceClassification.from_config(config)  from_pretrained  < source > ( *model_args **kwargs  )   Parameters   pretrained_model_name_or_path (str or os.PathLike) —
Can be either:
A string, the model id of a pretrained model hosted inside a model repo on huggingface.co.
Valid model ids can be located at the root-level, like bert-base-uncased, or namespaced under a
user or organization name, like dbmdz/bert-base-german-cased.
A path to a directory containing model weights saved using
save_pretrained(), e.g., ./my_model_directory/.
A path or url to a PyTorch state_dict save file (e.g, ./pt_model/pytorch_model.bin). In this
case, from_pt should be set to True and a configuration object should be provided as config
argument. This loading path is slower t


FlaxAutoModelForSequenceClassification

han converting the PyTorch model in a TensorFlow model
using the provided conversion scripts and loading the TensorFlow model afterwards.
   model_args (additional positional arguments, optional) —
Will be passed along to the underlying model __init__() method.   config (PretrainedConfig, optional) —
Configuration for the model to use instead of an automatically loaded configuration. Configuration can
be automatically loaded when:
The model is a model provided by the library (loaded with the model id string of a pretrained
model).
The model was saved using save_pretrained() and is reloaded by supplying the
save directory.
The model is loaded by supplying a local directory as pretrained_model_name_or_path and a
configuration JSON file named config.json is found in the directory.
   cache_dir (str or os.PathLike, optional) —
Path to a directory in which a downloaded pretrained model configuration should be cached if the
standard cache shoul


FlaxAutoModelForSequenceClassification

d not be used.   from_pt (bool, optional, defaults to False) —
Load the model weights from a PyTorch checkpoint save file (see docstring of
pretrained_model_name_or_path argument).   force_download (bool, optional, defaults to False) —
Whether or not to force the (re-)download of the model weights and configuration files, overriding the
cached versions if they exist.   resume_download (bool, optional, defaults to False) —
Whether or not to delete incompletely received files. Will attempt to resume the download if such a
file exists.   proxies (Dict[str, str], optional) —
A dictionary of proxy servers to use by protocol or endpoint, e.g., {'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}. The proxies are used on each request.   output_loading_info(bool, optional, defaults to False) —
Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.   local_files_only(bool, optional, defaults


FlaxAutoModelForSequenceClassification

to False) —
Whether or not to only look at local files (e.g., not try downloading the model).   revision (str, optional, defaults to "main") —
The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
git-based system for storing models and other artifacts on huggingface.co, so revision can be any
identifier allowed by git.   trust_remote_code (bool, optional, defaults to False) —
Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
should only be set to True for repositories you trust and in which you have read the code, as it will
execute code present on the Hub on your local machine.   code_revision (str, optional, defaults to "main") —
The specific revision to use for the code on the Hub, if the code leaves in a different repository than
the rest of the model. It can be a branch name, a tag name, or a commit id, since we use a git-based
syste


FlaxAutoModelForSequenceClassification

m for storing models and other artifacts on huggingface.co, so revision can be any identifier
allowed by git.   kwargs (additional keyword arguments, optional) —
Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
output_attentions=True). Behaves differently depending on whether a config is provided or
automatically loaded:
If a configuration is provided with config, **kwargs will be directly passed to the
underlying model’s __init__ method (we assume all relevant updates to the configuration have
already been done)
If a configuration is not provided, kwargs will be first passed to the configuration class
initialization function (from_pretrained()). Each key of kwargs that
corresponds to a configuration attribute will be used to override said attribute with the
supplied kwargs value. Remaining keys that do not correspond to any configuration attribute
will be passed to the underlying model’


FlaxAutoModelForSequenceClassification

s __init__ function.
    Instantiate one of the model classes of the library (with a sequence classification head) from a pretrained model. The model class to instantiate is selected based on the model_type property of the config object (either
passed as an argument or loaded from pretrained_model_name_or_path if possible), or when it’s missing, by
falling back to using pattern matching on pretrained_model_name_or_path: albert — FlaxAlbertForSequenceClassification (ALBERT model) bart — FlaxBartForSequenceClassification (BART model) bert — FlaxBertForSequenceClassification (BERT model) big_bird — FlaxBigBirdForSequenceClassification (BigBird model) distilbert — FlaxDistilBertForSequenceClassification (DistilBERT model) electra — FlaxElectraForSequenceClassification (ELECTRA model) mbart — FlaxMBartForSequenceClassification (mBART model) roberta — FlaxRobertaForSequenceClassification (RoBERTa model) roberta-prelayernorm — FlaxRobertaPreLaye


FlaxAutoModelForSequenceClassification

rNormForSequenceClassification (RoBERTa-PreLayerNorm model) roformer — FlaxRoFormerForSequenceClassification (RoFormer model) xlm-roberta — FlaxXLMRobertaForSequenceClassification (XLM-RoBERTa model)  Examples:   Copied >>> from transformers import AutoConfig, FlaxAutoModelForSequenceClassification
>>> # Download model and configuration from huggingface.co and cache.
>>> model = FlaxAutoModelForSequenceClassification.from_pretrained("bert-base-cased")
>>> # Update configuration during loading
>>> model = FlaxAutoModelForSequenceClassification.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True
>>> # Loading from a PyTorch checkpoint file instead of a TensorFlow model (slower)
>>> config = AutoConfig.from_pretrained("./pt_model/bert_pt_model_config.json")
>>> model = FlaxAutoModelForSequenceClassification.from_pretrained(
...     "./pt_model/bert_pytorch_model.bin", from_pt=True, config=config


AutoModelForMultipleChoice

class transformers.AutoModelForMultipleChoice  < source > ( *args **kwargs  )    This is a generic model class that will be instantiated as one of the model classes of the library (with a multiple choice head) when created
with the from_pretrained() class method or the from_config() class
method. This class cannot be instantiated directly using __init__() (throws an error).  from_config  < source > ( **kwargs  )   Parameters   config (PretrainedConfig) —
The model class to instantiate is selected based on the configuration class:
AlbertConfig configuration class: AlbertForMultipleChoice (ALBERT model)
BertConfig configuration class: BertForMultipleChoice (BERT model)
BigBirdConfig configuration class: BigBirdForMultipleChoice (BigBird model)
CamembertConfig configuration class: CamembertForMultipleChoice (CamemBERT model)
CanineConfig configuration class: CanineForMultipleChoice (CANINE model)
ConvBertConfig configuration class: C


AutoModelForMultipleChoice

onvBertForMultipleChoice (ConvBERT model)
Data2VecTextConfig configuration class: Data2VecTextForMultipleChoice (Data2VecText model)
DebertaV2Config configuration class: DebertaV2ForMultipleChoice (DeBERTa-v2 model)
DistilBertConfig configuration class: DistilBertForMultipleChoice (DistilBERT model)
ElectraConfig configuration class: ElectraForMultipleChoice (ELECTRA model)
ErnieConfig configuration class: ErnieForMultipleChoice (ERNIE model)
ErnieMConfig configuration class: ErnieMForMultipleChoice (ErnieM model)
FNetConfig configuration class: FNetForMultipleChoice (FNet model)
FlaubertConfig configuration class: FlaubertForMultipleChoice (FlauBERT model)
FunnelConfig configuration class: FunnelForMultipleChoice (Funnel Transformer model)
IBertConfig configuration class: IBertForMultipleChoice (I-BERT model)
LongformerConfig configuration class: LongformerForMultipleChoice (Longformer model)
LukeConfig configuration class: LukeF


AutoModelForMultipleChoice

orMultipleChoice (LUKE model)
MPNetConfig configuration class: MPNetForMultipleChoice (MPNet model)
MegaConfig configuration class: MegaForMultipleChoice (MEGA model)
MegatronBertConfig configuration class: MegatronBertForMultipleChoice (Megatron-BERT model)
MobileBertConfig configuration class: MobileBertForMultipleChoice (MobileBERT model)
MraConfig configuration class: MraForMultipleChoice (MRA model)
NezhaConfig configuration class: NezhaForMultipleChoice (Nezha model)
NystromformerConfig configuration class: NystromformerForMultipleChoice (Nyströmformer model)
QDQBertConfig configuration class: QDQBertForMultipleChoice (QDQBert model)
RemBertConfig configuration class: RemBertForMultipleChoice (RemBERT model)
RoCBertConfig configuration class: RoCBertForMultipleChoice (RoCBert model)
RoFormerConfig configuration class: RoFormerForMultipleChoice (RoFormer model)
RobertaConfig configuration class: RobertaForMultipleChoice (RoBE


AutoModelForMultipleChoice

RTa model)
RobertaPreLayerNormConfig configuration class: RobertaPreLayerNormForMultipleChoice (RoBERTa-PreLayerNorm model)
SqueezeBertConfig configuration class: SqueezeBertForMultipleChoice (SqueezeBERT model)
XLMConfig configuration class: XLMForMultipleChoice (XLM model)
XLMRobertaConfig configuration class: XLMRobertaForMultipleChoice (XLM-RoBERTa model)
XLMRobertaXLConfig configuration class: XLMRobertaXLForMultipleChoice (XLM-RoBERTa-XL model)
XLNetConfig configuration class: XLNetForMultipleChoice (XLNet model)
XmodConfig configuration class: XmodForMultipleChoice (X-MOD model)
YosoConfig configuration class: YosoForMultipleChoice (YOSO model)
    Instantiates one of the model classes of the library (with a multiple choice head) from a configuration. Note:
Loading a model from its configuration file does not load the model weights. It only affects the
model’s configuration. Use from_pretrained() to load the model weights.


AutoModelForMultipleChoice

 Examples:   Copied >>> from transformers import AutoConfig, AutoModelForMultipleChoice
>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = AutoModelForMultipleChoice.from_config(config)  from_pretrained  < source > ( *model_args **kwargs  )   Parameters   pretrained_model_name_or_path (str or os.PathLike) —
Can be either:
A string, the model id of a pretrained model hosted inside a model repo on huggingface.co.
Valid model ids can be located at the root-level, like bert-base-uncased, or namespaced under a
user or organization name, like dbmdz/bert-base-german-cased.
A path to a directory containing model weights saved using
save_pretrained(), e.g., ./my_model_directory/.
A path or url to a tensorflow index checkpoint file (e.g, ./tf_model/model.ckpt.index). In
this case, from_tf should be set to True and a configuration object should be provided as
con


AutoModelForMultipleChoice

fig argument. This loading path is slower than converting the TensorFlow checkpoint in a
PyTorch model using the provided conversion scripts and loading the PyTorch model afterwards.
   model_args (additional positional arguments, optional) —
Will be passed along to the underlying model __init__() method.   config (PretrainedConfig, optional) —
Configuration for the model to use instead of an automatically loaded configuration. Configuration can
be automatically loaded when:
The model is a model provided by the library (loaded with the model id string of a pretrained
model).
The model was saved using save_pretrained() and is reloaded by supplying the
save directory.
The model is loaded by supplying a local directory as pretrained_model_name_or_path and a
configuration JSON file named config.json is found in the directory.
   state_dict (Dict[str, torch.Tensor], optional) —
A state dictionary to use instead of a state dictionary lo


AutoModelForMultipleChoice

aded from saved weights file.
This option can be used if you want to create a model from a pretrained configuration but load your own
weights. In this case though, you should check if using save_pretrained() and
from_pretrained() is not a simpler option.   cache_dir (str or os.PathLike, optional) —
Path to a directory in which a downloaded pretrained model configuration should be cached if the
standard cache should not be used.   from_tf (bool, optional, defaults to False) —
Load the model weights from a TensorFlow checkpoint save file (see docstring of
pretrained_model_name_or_path argument).   force_download (bool, optional, defaults to False) —
Whether or not to force the (re-)download of the model weights and configuration files, overriding the
cached versions if they exist.   resume_download (bool, optional, defaults to False) —
Whether or not to delete incompletely received files. Will attempt to resume the download if such


AutoModelForMultipleChoice

a
file exists.   proxies (Dict[str, str], optional) —
A dictionary of proxy servers to use by protocol or endpoint, e.g., {'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}. The proxies are used on each request.   output_loading_info(bool, optional, defaults to False) —
Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.   local_files_only(bool, optional, defaults to False) —
Whether or not to only look at local files (e.g., not try downloading the model).   revision (str, optional, defaults to "main") —
The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
git-based system for storing models and other artifacts on huggingface.co, so revision can be any
identifier allowed by git.   trust_remote_code (bool, optional, defaults to False) —
Whether or not to allow for custom models defined on the Hub in their own modeling file


AutoModelForMultipleChoice

s. This option
should only be set to True for repositories you trust and in which you have read the code, as it will
execute code present on the Hub on your local machine.   code_revision (str, optional, defaults to "main") —
The specific revision to use for the code on the Hub, if the code leaves in a different repository than
the rest of the model. It can be a branch name, a tag name, or a commit id, since we use a git-based
system for storing models and other artifacts on huggingface.co, so revision can be any identifier
allowed by git.   kwargs (additional keyword arguments, optional) —
Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
output_attentions=True). Behaves differently depending on whether a config is provided or
automatically loaded:
If a configuration is provided with config, **kwargs will be directly passed to the
underlying model’s __init__ method (we assume all


AutoModelForMultipleChoice

 relevant updates to the configuration have
already been done)
If a configuration is not provided, kwargs will be first passed to the configuration class
initialization function (from_pretrained()). Each key of kwargs that
corresponds to a configuration attribute will be used to override said attribute with the
supplied kwargs value. Remaining keys that do not correspond to any configuration attribute
will be passed to the underlying model’s __init__ function.
    Instantiate one of the model classes of the library (with a multiple choice head) from a pretrained model. The model class to instantiate is selected based on the model_type property of the config object (either
passed as an argument or loaded from pretrained_model_name_or_path if possible), or when it’s missing, by
falling back to using pattern matching on pretrained_model_name_or_path: albert — AlbertForMultipleChoice (ALBERT model) bert — BertForMultipleChoice (BERT m


AutoModelForMultipleChoice

odel) big_bird — BigBirdForMultipleChoice (BigBird model) camembert — CamembertForMultipleChoice (CamemBERT model) canine — CanineForMultipleChoice (CANINE model) convbert — ConvBertForMultipleChoice (ConvBERT model) data2vec-text — Data2VecTextForMultipleChoice (Data2VecText model) deberta-v2 — DebertaV2ForMultipleChoice (DeBERTa-v2 model) distilbert — DistilBertForMultipleChoice (DistilBERT model) electra — ElectraForMultipleChoice (ELECTRA model) ernie — ErnieForMultipleChoice (ERNIE model) ernie_m — ErnieMForMultipleChoice (ErnieM model) flaubert — FlaubertForMultipleChoice (FlauBERT model) fnet — FNetForMultipleChoice (FNet model) funnel — FunnelForMultipleChoice (Funnel Transformer model) ibert — IBertForMultipleChoice (I-BERT model) longformer — LongformerForMultipleChoice (Longformer model) luke — LukeForMultipleChoice (LUKE model) mega — MegaForMultipleChoice (MEGA model) megatron-bert — MegatronBertForMultipleChoice (Meg


AutoModelForMultipleChoice

atron-BERT model) mobilebert — MobileBertForMultipleChoice (MobileBERT model) mpnet — MPNetForMultipleChoice (MPNet model) mra — MraForMultipleChoice (MRA model) nezha — NezhaForMultipleChoice (Nezha model) nystromformer — NystromformerForMultipleChoice (Nyströmformer model) qdqbert — QDQBertForMultipleChoice (QDQBert model) rembert — RemBertForMultipleChoice (RemBERT model) roberta — RobertaForMultipleChoice (RoBERTa model) roberta-prelayernorm — RobertaPreLayerNormForMultipleChoice (RoBERTa-PreLayerNorm model) roc_bert — RoCBertForMultipleChoice (RoCBert model) roformer — RoFormerForMultipleChoice (RoFormer model) squeezebert — SqueezeBertForMultipleChoice (SqueezeBERT model) xlm — XLMForMultipleChoice (XLM model) xlm-roberta — XLMRobertaForMultipleChoice (XLM-RoBERTa model) xlm-roberta-xl — XLMRobertaXLForMultipleChoice (XLM-RoBERTa-XL model) xlnet — XLNetForMultipleChoice (XLNet model) xmod — XmodForMultipleChoice (X-MOD model


AutoModelForMultipleChoice

) yoso — YosoForMultipleChoice (YOSO model) The model is set in evaluation mode by default using model.eval() (so for instance, dropout modules are
deactivated). To train the model, you should first set it back in training mode with model.train()  Examples:   Copied >>> from transformers import AutoConfig, AutoModelForMultipleChoice
>>> # Download model and configuration from huggingface.co and cache.
>>> model = AutoModelForMultipleChoice.from_pretrained("bert-base-cased")
>>> # Update configuration during loading
>>> model = AutoModelForMultipleChoice.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True
>>> # Loading from a TF checkpoint file instead of a PyTorch model (slower)
>>> config = AutoConfig.from_pretrained("./tf_model/bert_tf_model_config.json")
>>> model = AutoModelForMultipleChoice.from_pretrained(
...     "./tf_model/bert_tf_checkpoint.ckpt.index", from_tf=True, config=


TFAutoModelForMultipleChoice

class transformers.TFAutoModelForMultipleChoice  < source > ( *args **kwargs  )    This is a generic model class that will be instantiated as one of the model classes of the library (with a multiple choice head) when created
with the from_pretrained() class method or the from_config() class
method. This class cannot be instantiated directly using __init__() (throws an error).  from_config  < source > ( **kwargs  )   Parameters   config (PretrainedConfig) —
The model class to instantiate is selected based on the configuration class:
AlbertConfig configuration class: TFAlbertForMultipleChoice (ALBERT model)
BertConfig configuration class: TFBertForMultipleChoice (BERT model)
CamembertConfig configuration class: TFCamembertForMultipleChoice (CamemBERT model)
ConvBertConfig configuration class: TFConvBertForMultipleChoice (ConvBERT model)
DebertaV2Config configuration class: TFDebertaV2ForMultipleChoice (DeBERTa-v2 model)
Distil


TFAutoModelForMultipleChoice

BertConfig configuration class: TFDistilBertForMultipleChoice (DistilBERT model)
ElectraConfig configuration class: TFElectraForMultipleChoice (ELECTRA model)
FlaubertConfig configuration class: TFFlaubertForMultipleChoice (FlauBERT model)
FunnelConfig configuration class: TFFunnelForMultipleChoice (Funnel Transformer model)
LongformerConfig configuration class: TFLongformerForMultipleChoice (Longformer model)
MPNetConfig configuration class: TFMPNetForMultipleChoice (MPNet model)
MobileBertConfig configuration class: TFMobileBertForMultipleChoice (MobileBERT model)
RemBertConfig configuration class: TFRemBertForMultipleChoice (RemBERT model)
RoFormerConfig configuration class: TFRoFormerForMultipleChoice (RoFormer model)
RobertaConfig configuration class: TFRobertaForMultipleChoice (RoBERTa model)
RobertaPreLayerNormConfig configuration class: TFRobertaPreLayerNormForMultipleChoice (RoBERTa-PreLayerNorm model)
XLMConfig con


TFAutoModelForMultipleChoice

figuration class: TFXLMForMultipleChoice (XLM model)
XLMRobertaConfig configuration class: TFXLMRobertaForMultipleChoice (XLM-RoBERTa model)
XLNetConfig configuration class: TFXLNetForMultipleChoice (XLNet model)
    Instantiates one of the model classes of the library (with a multiple choice head) from a configuration. Note:
Loading a model from its configuration file does not load the model weights. It only affects the
model’s configuration. Use from_pretrained() to load the model weights.  Examples:   Copied >>> from transformers import AutoConfig, TFAutoModelForMultipleChoice
>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = TFAutoModelForMultipleChoice.from_config(config)  from_pretrained  < source > ( *model_args **kwargs  )   Parameters   pretrained_model_name_or_path (str or os.PathLike) —
Can be either:
A string, the model id of a pretr


TFAutoModelForMultipleChoice

ained model hosted inside a model repo on huggingface.co.
Valid model ids can be located at the root-level, like bert-base-uncased, or namespaced under a
user or organization name, like dbmdz/bert-base-german-cased.
A path to a directory containing model weights saved using
save_pretrained(), e.g., ./my_model_directory/.
A path or url to a PyTorch state_dict save file (e.g, ./pt_model/pytorch_model.bin). In this
case, from_pt should be set to True and a configuration object should be provided as config
argument. This loading path is slower than converting the PyTorch model in a TensorFlow model
using the provided conversion scripts and loading the TensorFlow model afterwards.
   model_args (additional positional arguments, optional) —
Will be passed along to the underlying model __init__() method.   config (PretrainedConfig, optional) —
Configuration for the model to use instead of an automatically loaded configuration. Conf


TFAutoModelForMultipleChoice

iguration can
be automatically loaded when:
The model is a model provided by the library (loaded with the model id string of a pretrained
model).
The model was saved using save_pretrained() and is reloaded by supplying the
save directory.
The model is loaded by supplying a local directory as pretrained_model_name_or_path and a
configuration JSON file named config.json is found in the directory.
   cache_dir (str or os.PathLike, optional) —
Path to a directory in which a downloaded pretrained model configuration should be cached if the
standard cache should not be used.   from_pt (bool, optional, defaults to False) —
Load the model weights from a PyTorch checkpoint save file (see docstring of
pretrained_model_name_or_path argument).   force_download (bool, optional, defaults to False) —
Whether or not to force the (re-)download of the model weights and configuration files, overriding the
cached versions if they exist.   resum


TFAutoModelForMultipleChoice

e_download (bool, optional, defaults to False) —
Whether or not to delete incompletely received files. Will attempt to resume the download if such a
file exists.   proxies (Dict[str, str], optional) —
A dictionary of proxy servers to use by protocol or endpoint, e.g., {'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}. The proxies are used on each request.   output_loading_info(bool, optional, defaults to False) —
Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.   local_files_only(bool, optional, defaults to False) —
Whether or not to only look at local files (e.g., not try downloading the model).   revision (str, optional, defaults to "main") —
The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
git-based system for storing models and other artifacts on huggingface.co, so revision can be any
identifier allowed


TFAutoModelForMultipleChoice

by git.   trust_remote_code (bool, optional, defaults to False) —
Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
should only be set to True for repositories you trust and in which you have read the code, as it will
execute code present on the Hub on your local machine.   code_revision (str, optional, defaults to "main") —
The specific revision to use for the code on the Hub, if the code leaves in a different repository than
the rest of the model. It can be a branch name, a tag name, or a commit id, since we use a git-based
system for storing models and other artifacts on huggingface.co, so revision can be any identifier
allowed by git.   kwargs (additional keyword arguments, optional) —
Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
output_attentions=True). Behaves differently depending on whether a config is provided


TFAutoModelForMultipleChoice

 or
automatically loaded:
If a configuration is provided with config, **kwargs will be directly passed to the
underlying model’s __init__ method (we assume all relevant updates to the configuration have
already been done)
If a configuration is not provided, kwargs will be first passed to the configuration class
initialization function (from_pretrained()). Each key of kwargs that
corresponds to a configuration attribute will be used to override said attribute with the
supplied kwargs value. Remaining keys that do not correspond to any configuration attribute
will be passed to the underlying model’s __init__ function.
    Instantiate one of the model classes of the library (with a multiple choice head) from a pretrained model. The model class to instantiate is selected based on the model_type property of the config object (either
passed as an argument or loaded from pretrained_model_name_or_path if possible), or when it’s miss


TFAutoModelForMultipleChoice

ing, by
falling back to using pattern matching on pretrained_model_name_or_path: albert — TFAlbertForMultipleChoice (ALBERT model) bert — TFBertForMultipleChoice (BERT model) camembert — TFCamembertForMultipleChoice (CamemBERT model) convbert — TFConvBertForMultipleChoice (ConvBERT model) deberta-v2 — TFDebertaV2ForMultipleChoice (DeBERTa-v2 model) distilbert — TFDistilBertForMultipleChoice (DistilBERT model) electra — TFElectraForMultipleChoice (ELECTRA model) flaubert — TFFlaubertForMultipleChoice (FlauBERT model) funnel — TFFunnelForMultipleChoice (Funnel Transformer model) longformer — TFLongformerForMultipleChoice (Longformer model) mobilebert — TFMobileBertForMultipleChoice (MobileBERT model) mpnet — TFMPNetForMultipleChoice (MPNet model) rembert — TFRemBertForMultipleChoice (RemBERT model) roberta — TFRobertaForMultipleChoice (RoBERTa model) roberta-prelayernorm — TFRobertaPreLayerNormForMultipleChoice (RoBERTa-PreLay


TFAutoModelForMultipleChoice

erNorm model) roformer — TFRoFormerForMultipleChoice (RoFormer model) xlm — TFXLMForMultipleChoice (XLM model) xlm-roberta — TFXLMRobertaForMultipleChoice (XLM-RoBERTa model) xlnet — TFXLNetForMultipleChoice (XLNet model)  Examples:   Copied >>> from transformers import AutoConfig, TFAutoModelForMultipleChoice
>>> # Download model and configuration from huggingface.co and cache.
>>> model = TFAutoModelForMultipleChoice.from_pretrained("bert-base-cased")
>>> # Update configuration during loading
>>> model = TFAutoModelForMultipleChoice.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True
>>> # Loading from a PyTorch checkpoint file instead of a TensorFlow model (slower)
>>> config = AutoConfig.from_pretrained("./pt_model/bert_pt_model_config.json")
>>> model = TFAutoModelForMultipleChoice.from_pretrained(
...     "./pt_model/bert_pytorch_model.bin", from_pt=True, config=config
...


FlaxAutoModelForMultipleChoice

class transformers.FlaxAutoModelForMultipleChoice  < source > ( *args **kwargs  )    This is a generic model class that will be instantiated as one of the model classes of the library (with a multiple choice head) when created
with the from_pretrained() class method or the from_config() class
method. This class cannot be instantiated directly using __init__() (throws an error).  from_config  < source > ( **kwargs  )   Parameters   config (PretrainedConfig) —
The model class to instantiate is selected based on the configuration class:
AlbertConfig configuration class: FlaxAlbertForMultipleChoice (ALBERT model)
BertConfig configuration class: FlaxBertForMultipleChoice (BERT model)
BigBirdConfig configuration class: FlaxBigBirdForMultipleChoice (BigBird model)
DistilBertConfig configuration class: FlaxDistilBertForMultipleChoice (DistilBERT model)
ElectraConfig configuration class: FlaxE


FlaxAutoModelForMultipleChoice

lectraForMultipleChoice (ELECTRA model)
RoFormerConfig configuration class: FlaxRoFormerForMultipleChoice (RoFormer model)
RobertaConfig configuration class: FlaxRobertaForMultipleChoice (RoBERTa model)
RobertaPreLayerNormConfig configuration class: FlaxRobertaPreLayerNormForMultipleChoice (RoBERTa-PreLayerNorm model)
XLMRobertaConfig configuration class: FlaxXLMRobertaForMultipleChoice (XLM-RoBERTa model)
    Instantiates one of the model classes of the library (with a multiple choice head) from a configuration. Note:
Loading a model from its configuration file does not load the model weights. It only affects the
model’s configuration. Use from_pretrained() to load the model weights.  Examples:   Copied >>> from transformers import AutoConfig, FlaxAutoModelForMultipleChoice
>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-


FlaxAutoModelForMultipleChoice

cased")
>>> model = FlaxAutoModelForMultipleChoice.from_config(config)  from_pretrained  < source > ( *model_args **kwargs  )   Parameters   pretrained_model_name_or_path (str or os.PathLike) —
Can be either:
A string, the model id of a pretrained model hosted inside a model repo on huggingface.co.
Valid model ids can be located at the root-level, like bert-base-uncased, or namespaced under a
user or organization name, like dbmdz/bert-base-german-cased.
A path to a directory containing model weights saved using
save_pretrained(), e.g., ./my_model_directory/.
A path or url to a PyTorch state_dict save file (e.g, ./pt_model/pytorch_model.bin). In this
case, from_pt should be set to True and a configuration object should be provided as config
argument. This loading path is slower than converting the PyTorch model in a TensorFlow model
using the provided conversion scripts and loading the


FlaxAutoModelForMultipleChoice

 TensorFlow model afterwards.
   model_args (additional positional arguments, optional) —
Will be passed along to the underlying model __init__() method.   config (PretrainedConfig, optional) —
Configuration for the model to use instead of an automatically loaded configuration. Configuration can
be automatically loaded when:
The model is a model provided by the library (loaded with the model id string of a pretrained
model).
The model was saved using save_pretrained() and is reloaded by supplying the
save directory.
The model is loaded by supplying a local directory as pretrained_model_name_or_path and a
configuration JSON file named config.json is found in the directory.
   cache_dir (str or os.PathLike, optional) —
Path to a directory in which a downloaded pretrained model configuration should be cached if the
standard cache should not be used.   from_pt (bool, optional, defaults to