Instructions to use AkifJK/trialmatch-qwen2.5-1.5b-lora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use AkifJK/trialmatch-qwen2.5-1.5b-lora with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
TrialMatch LoRA adapters (Qwen2.5-1.5B-Instruct)
Educational screening prototype, not a clinical enrollment decision.
Three LoRA adapters trained for one task: given a numbered patient note and one trial criterion (inclusion or exclusion), output a JSON object with the supporting sentence ids and one of the dataset's labels.
{"evidence_sentence_ids":[0,2],"label":"included"}
Files
| Folder | Seed | Validation macro-F1 (best epoch) |
|---|---|---|
seed42/ |
42 | 0.426 |
seed43/ |
43 | 0.440 |
seed44/ |
44 | 0.473 |
Each folder holds adapter_model.safetensors and adapter_config.json.
draft/ holds a separate adapter for the smaller Qwen/Qwen2.5-0.5B-Instruct (revision
7ae557604adf67be50417f59c2c2f167def9a775), trained with the same recipe (seed 42). It is the draft model for
speculative decoding against the 1.5B target and is not meant to be used alone. Validation macro-F1 0.438, accuracy 0.738
(same caveats on the rare classes: it predicted not included 3 times, none correct, and never excluded).
The base model of this folder is the 0.5B model, not the 1.5B one.
Training
- Base model:
Qwen/Qwen2.5-1.5B-Instructat revision989aa7980e4cf806f80c7fef2b1adb7bc71aa306. - Data:
ncbi/TrialGPT-Criterion-Annotationsat revision1cfcafde94a1560a33b4addc1638664fe28fc059(license listed on the dataset page asother). Patient-level split 37/8/8 patients with seed 42; training used the 37 training patients only (602 usable rows). - LoRA:
r=16,alpha=32, dropout 0.05, targetsq_proj,k_proj,v_proj,o_proj. - Optimiser: AdamW, lr 2e-4, cosine schedule with 5% warmup, batch 8, 3 epochs, fp16 base with float32 adapter and a GradScaler, gradient checkpointing.
- Rare-label oversampling: rows labelled
excludedornot includedare repeated 4 times. - Loss covers the assistant tokens only. Prompt, split and training code are in the project repository.
- Checkpoint per seed: the epoch with the highest strict validation macro-F1.
Results so far
Validation only (206 rows from 8 held-out patients). The test split has not been opened.
- Mean validation macro-F1 over the three seeds: 0.446 (SD 0.024). Mean accuracy 0.731. For comparison, a per-type majority baseline scores 0.215 macro-F1 and 0.621 accuracy.
- The rare classes are not learned. The validation split has 2
excludedand 3not includedrows, and across these three adapters only one row of each was ever predicted correctly. Do not read these adapters as able to detect those two labels. - A competing configuration (
r=32, no oversampling) scores 0.434 (SD 0.010) and cannot be told apart from this one.
Limits
- Trained on 37 patients from one dataset. Expect weak transfer to other note styles.
- Labels follow the dataset's annotation scheme, not a clinical judgement.
- Output must be validated: the cited sentence ids can be wrong, and a missing citation never means a label.
- Downloads last month
- -