README.md

---
license: apache-2.0
language:
- en
pipeline_tag: text-generation
library_name: transformers
---
# TD-Llama-OP

## TL;DR
**TD (ToolDial)-Llama-OP (OverallPerformance)** is the same model used in [ToolDial](https://arxiv.org/abs/2503.00564) paper **Overall Performance Task**. We encourage you to use this model to reproduce the results.
 Please refer the **Experiments** of our [github page](https://github.com/holi-lab/ToolDial) to see how our evaluation has proceed.

**[Model Summary]**
- Trained with Q-lora quantization, and LoRA Adapters are merged to original weights.
- Trained for 1 epoch with Adam-8bit optimizer with learning rate 0.00001 and beta 0.9 to 0.995

**[How to load the model]**
```python
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel

device = "cuda:0"

quant_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type='nf4',
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)

## 1. Load the base model (we use llama3-8b-inst) with the given quantization config.
base_model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Meta-Llama-3-8B-Instruct",
    quantization_config=quant_config,
    device_map={"": device},
)
tokenizer = AutoTokenizer.from_pretrained("HOLILAB/td-llama-op")
tokenizer.pad_token_id = tokenizer.eos_token_id

## 2. Load the lora adapter with PeftModel
model = PeftModel.from_pretrained(base_model, "HOLILAB/td-llama-op")

```