tool calling support
hi, firstly, great work, thanks a lot!
Also, what about tool calling, did you figure this out? Anything that could be done to make it work? Thanks!
I was never successful in getting official vllm tool calling to work with these internvl 3 models. Since these models have a base layer of qwen 3, I tried the hermes option (--tool-call-parser hermes) with no success. I'm guessing it didn't work with just that because the chat template that comes stock with internvl models doesn't look to have function calling injection in the template. Could be as simple as adding that to the chat template from a base qwen 3 model chat template.
Deep wiki suggested trying the internlm specific option with chat template--tool-call-parser internlm --chat-template examples/tool_chat_template_internlm2_tool.jinjaand that one references using the chat template too, which is my best guess on what's missing.
All in all, I haven't had the chance to follow up in this, but best guess would be a combination of tool-call-parser (hermes or internlm) and adding in proper function templating into the chat template for the model
https://github.com/vllm-project/vllm/blob/36429096/docs%2Ffeatures%2Ftool_calling.md