`tokenizer.model` is missing
As the title said, it seems that all Phi4 models don't have this tokenizer file, despite the architecture being the same as Phi3. But it is needed by some third party deployment framework.
Having a similar problem for a fine-tuned model that uses tokenizer.json.
@elTigre914 @happyme531
If you have inference errors, please double check the environ and dependencies, this is what we suggest
https://huggingface.co/microsoft/Phi-4-multimodal-instruct#requirements
Alternatively you can look at this dockerfile
https://github.com/anastasiosyal/phi4-multimodal-instruct-server/blob/main/dockerfile
For finetuning you could take a look at Korean finetuning writeup contributed by the community
https://huggingface.co/microsoft/Phi-4-multimodal-instruct#appendix-b-fine-tuning-korean-speech
@elTigre914 @happyme531
If you have inference errors, please double check the environ and dependencies, this is what we suggest
https://huggingface.co/microsoft/Phi-4-multimodal-instruct#requirementsAlternatively you can look at this dockerfile
https://github.com/anastasiosyal/phi4-multimodal-instruct-server/blob/main/dockerfileFor finetuning you could take a look at Korean finetuning writeup contributed by the community
https://huggingface.co/microsoft/Phi-4-multimodal-instruct#appendix-b-fine-tuning-korean-speech
But how can we get a tokenizer.model anyway?
Hi @happyme531 ,
We are using gpt-4o tokenizer converted from tiktoken.
So, we have tokenizer.json file, but we don't support tokenizer.model.
Can you try to use tokenizers library?
Hi @happyme531 ,
We are using gpt-4o tokenizer converted from tiktoken.
So, we havetokenizer.jsonfile, but we don't supporttokenizer.model.Can you try to use tokenizers library?
tokenizers library works, but there are third party frameworks that rely on tokenizer.model. Is there anyway to generate one using the existing tokenizer?
tokenizer.json is a recommended way to maintain all the information.
根据上面图片给出pyautogui操作步骤，比如: pyautogui.click(x, y)，pyautogui.mouseUp()，pyautogui.moveTo(x, y)，pyautogui.moveTo(x, y)，pyautogui.mouseDown()。其中(x, y)需要准确的数字，和pyautogui操作无关的内容不要输出。图片左上角坐标(0,0)，回复坐标用实际值，不要用假设值。