Model only outputs "!!!!!!!!!!"
#1
by mrhendrey - opened
Launched the model using vLLM (v0.6.4.post1), but the model only generates a string of "!!!!!" until the context window fills up. Any thoughts on what might be causing this behavior? Thanks in advance.
Hi @mrhendrey - could you try updating the version of vllm you're using? You'll also need to set the dtype to float16 when running the model.
Thanks