pere
/

roberta-base-exp-32B

Model card Files Files and versions

roberta-base-exp-32B / copy_tokenizer_config.py

pere's picture

test

a340e1f almost 4 years ago

Raw History Blame Contribute Delete

312 Bytes

	# Code used for copying tokenizer and config

	from transformers import XLMRobertaTokenizerFast, XLMRobertaConfig

	tokenizer = XLMRobertaTokenizerFast.from_pretrained("xlm-roberta-base")
	config = XLMRobertaConfig.from_pretrained("xlm-roberta-base")

	tokenizer.save_pretrained("./")
	config.save_pretrained("./")