QuyenAnhDE/Diseases_Symptoms
Viewer • Updated • 400 • 300 • 66
How to use shivvamm/MedLLM with Transformers:
# Use a pipeline as a high-level helper
from transformers import pipeline
pipe = pipeline("text-generation", model="shivvamm/MedLLM") # Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("shivvamm/MedLLM")
model = AutoModelForCausalLM.from_pretrained("shivvamm/MedLLM")How to use shivvamm/MedLLM with vLLM:
# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "shivvamm/MedLLM"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "shivvamm/MedLLM",
"prompt": "Once upon a time,",
"max_tokens": 512,
"temperature": 0.5
}'docker model run hf.co/shivvamm/MedLLM
How to use shivvamm/MedLLM with SGLang:
# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
--model-path "shivvamm/MedLLM" \
--host 0.0.0.0 \
--port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "shivvamm/MedLLM",
"prompt": "Once upon a time,",
"max_tokens": 512,
"temperature": 0.5
}'docker run --gpus all \
--shm-size 32g \
-p 30000:30000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HF_TOKEN=<secret>" \
--ipc=host \
lmsysorg/sglang:latest \
python3 -m sglang.launch_server \
--model-path "shivvamm/MedLLM" \
--host 0.0.0.0 \
--port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "shivvamm/MedLLM",
"prompt": "Once upon a time,",
"max_tokens": 512,
"temperature": 0.5
}'How to use shivvamm/MedLLM with Docker Model Runner:
docker model run hf.co/shivvamm/MedLLM
MedLLM is a language model designed to assist healthcare professionals and patients by providing detailed information about the symptoms of various diseases. This model can help in preliminary assessments and educational purposes by offering accurate and concise symptom descriptions.
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "shivvamm/MedLLM"
model = AutoModelForCausalLM.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)
inputs = tokenizer("What are the symptoms of diabetes?", return_tensors="pt")
outputs = model.generate(**inputs)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)