Instructions to use ibrahimmlk/trialsieve-qwen2.5-1.5b-lora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use ibrahimmlk/trialsieve-qwen2.5-1.5b-lora with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct") model = PeftModel.from_pretrained(base_model, "ibrahimmlk/trialsieve-qwen2.5-1.5b-lora") - Transformers
How to use ibrahimmlk/trialsieve-qwen2.5-1.5b-lora with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="ibrahimmlk/trialsieve-qwen2.5-1.5b-lora") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("ibrahimmlk/trialsieve-qwen2.5-1.5b-lora", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use ibrahimmlk/trialsieve-qwen2.5-1.5b-lora with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "ibrahimmlk/trialsieve-qwen2.5-1.5b-lora" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ibrahimmlk/trialsieve-qwen2.5-1.5b-lora", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/ibrahimmlk/trialsieve-qwen2.5-1.5b-lora
- SGLang
How to use ibrahimmlk/trialsieve-qwen2.5-1.5b-lora with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "ibrahimmlk/trialsieve-qwen2.5-1.5b-lora" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ibrahimmlk/trialsieve-qwen2.5-1.5b-lora", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "ibrahimmlk/trialsieve-qwen2.5-1.5b-lora" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ibrahimmlk/trialsieve-qwen2.5-1.5b-lora", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use ibrahimmlk/trialsieve-qwen2.5-1.5b-lora with Docker Model Runner:
docker model run hf.co/ibrahimmlk/trialsieve-qwen2.5-1.5b-lora
Qwen2.5-1.5B-Instruct LoRA for criterion-level clinical-trial eligibility screening
A LoRA adapter for Qwen/Qwen2.5-1.5B-Instruct. It takes a patient note (numbered sentences), a criterion type (inclusion or exclusion) and the criterion text. It returns the expert-style eligibility label and the IDs of the note sentences that support it, as JSON:
{"label": "not enough information", "evidence_sentence_ids": [2]}
Educational research prototype (university assignment). It is not a clinical enrollment decision tool.
- Author: Muhammad Ibrahim Malik (23i-0694), FAST-NUCES Islamabad. Generative AI, Assignment 2.
- Code, report and application: https://github.com/ibrahimmlk/trialsieve
- Demo video: https://youtu.be/n4dS_2brjwQ
- Draft model for speculative decoding: ibrahimmlk/trialsieve-qwen2.5-0.5b-draft-lora
Labels
| Criterion type | Labels |
|---|---|
| inclusion | included, not included, not enough information, not applicable |
| exclusion | excluded, not excluded, not enough information, not applicable |
not enough information and not applicable are kept as separate classes and are never collapsed into yes/no.
Results
Evaluated on the patient-grouped test split (8 unseen patients, 206 criteria). The test split was evaluated once,
with greedy decoding (repetition_penalty=1.0).
| Model | Accuracy | Macro-F1 | Evidence micro-F1 | Evidence F1 (gold non-empty) |
|---|---|---|---|---|
| Qwen2.5-1.5B-Instruct (base) | 0.325 | 0.256 | 0.123 | 0.111 |
| + this adapter | 0.743 | 0.545 | 0.508 | 0.533 |
Validation (8 patients, 206 criteria): accuracy 0.733, macro-F1 0.431, evidence micro-F1 0.522.
Per class (test): included F1 0.86, not excluded 0.83, not enough information 0.63, not applicable 0.40,
not included 0.00 (only 13 training examples).
Training
- Data: ncbi/TrialGPT-Criterion-Annotations
(expert labels and expert evidence sentences only; the GPT-4 columns are never used). Patient-grouped split: sorted
patient IDs shuffled with
numpy.random.default_rng(42), 37/8/8 patients (603/206/206 criteria). The manifest is in split_manifest.json. - Objective: causal-LM cross-entropy on the JSON answer tokens only (prompt tokens masked).
- LoRA: r=16, alpha=32, dropout 0.05, on q/k/v/o/gate/up/down projections (18.5M trainable parameters, 1.18%).
- Optimisation: AdamW, lr 2e-4, cosine schedule, 5% warm-up, 3 epochs, micro-batch 2 x gradient accumulation 8 (effective batch 16, 114 steps), fp16 with gradient checkpointing, seed 42.
- Checkpoint: step 80, selected by validation loss (0.113).
- Compute: 1x NVIDIA T4 (Kaggle), 15.8 minutes. No input was truncated (maximum 650 tokens; limit 1024).
How to use
The prompt format must match training. The easiest way is the repository's helper:
# pip install torch transformers peft ; git clone https://github.com/ibrahimmlk/trialsieve
import sys, torch
sys.path.insert(0, "trialsieve/src")
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
from trialsieve.data import messages, parse_output, number_note, note_sentences
base = "Qwen/Qwen2.5-1.5B-Instruct"
tok = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, dtype=torch.float32)
model = PeftModel.from_pretrained(model, "ibrahimmlk/trialsieve-qwen2.5-1.5b-lora").merge_and_unload().eval()
note = number_note("A 62-year-old man with type 2 diabetes. He had a myocardial infarction two years ago.")
rec = {"note": note, "criterion_type": "exclusion", "criterion_text": "Myocardial infarction within the past 6 months"}
prompt = tok.apply_chat_template(messages(rec), tokenize=False, add_generation_prompt=True)
ids = tok(prompt, return_tensors="pt", add_special_tokens=False).input_ids
out = model.generate(ids, do_sample=False, max_new_tokens=64, repetition_penalty=1.0)
text = tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True)
print(parse_output(text, "exclusion", len(note_sentences(note)))) # (label, evidence_ids, parsed_ok)
For a complete screening workflow (all criteria of a trial, evidence, rule-based outcome, report export), run the
browser application in the repository (run_app.bat).
Limitations
- Small training set (37 patients). The minority classes
not included(13 examples) andexcluded(8) are not learned, and the test split has noexcludedcase. - The most frequent error is reading missing information as a negative finding (
not enough informationpredicted asnot excluded, 28 of 206 test criteria). It also struggles with numeric range checks. - Low evidence recall on exclusion criteria (0.09): the model often answers
not excludedwithout citing a sentence. - The patient notes are short synthetic case descriptions, not real EHR text.
- Not validated for clinical use. Every output must be reviewed by qualified clinical research staff.
Citation
The dataset comes from: Jin et al., "Matching patients to clinical trials with large language models", Nature Communications 15, 9074 (2024). https://www.nature.com/articles/s41467-024-53081-z
- Downloads last month
- 88