File size: 2,874 Bytes
61b0f7e e06c151 ad50998 e06c151 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 | ---
library_name: transformers
tags: []
---
# Llama 2 7B Vanilla
This is the [**Llama 2 7B**](https://huggingface.co/meta-llama/Llama-2-7b-hf) model fine-tuned as the vanilla (unmodified) baseline, trained and evaluated in the paper [ASIDE: Architectural Separation of Instructions and Data in Language Models](https://openreview.net/forum?id=C81TnwHiRM).
## Model Description
This is the vanilla (unmodified) baseline fine-tuned with the same training data and procedure, but without any embedding modification.
## Usage
To use this model, first clone and follow the installation instructions in the official [ASIDE Repository](https://github.com/egozverev/aside/tree/main).
Inside the repository, run the following code snippet [(also provided here as a script)](https://github.com/egozverev/aside/blob/main/experiments/example.py) to do inference with this model.
```python
import torch
import deepspeed
import json
import os
from huggingface_hub import login
from model_api import CustomModelHandler # Import your custom handler
from model_api import format_prompt # Import your prompt formatting function
# Define your instruction and data
instruction_text = "Translate to German."
data_text = "Who is Albert Einstein?"
# Model configuration
hf_token = os.environ["HUGGINGFACE_HUB_TOKEN"]
login(token=hf_token)
embedding_type = "single_emb"
base_model = "meta-llama/Llama-2-7b-hf"
model_path = "Embeddings-Collab/llama_2_7b_single_emb_emb_SFTv110_from_base_run_19"
# Initialize the model handler
handler = CustomModelHandler(
model_path,
base_model,
base_model,
model_path,
None,
0,
embedding_type=embedding_type,
load_from_checkpoint=True
)
# Initialize DeepSpeed inference engine
engine = deepspeed.init_inference(
model=handler.model,
mp_size=torch.cuda.device_count(), # Number of GPUs
dtype=torch.float16,
replace_method='auto',
replace_with_kernel_inject=False
)
handler.model = engine.module
# Load prompt templates
with open("./data/prompt_templates.json", "r") as f:
templates = json.load(f)
template = templates[0]
instruction_text = format_prompt(instruction_text, template, "system")
data_text = format_prompt(data_text, template, "user")
# Generate output
output, inp = handler.call_model_api_batch([instruction_text], [data_text])
print(output)
```
### Citation
If you use this model, please cite our paper:
```
@inproceedings{
zverev2026aside,
title={{ASIDE}}: Architectural Separation of Instructions and Data in Language Models},
author={Egor Zverev and Evgenii Kortukov and Alexander Panfilov and Alexandra Volkova and Rush Tabesh and Sebastian Lapuschkin and Wojciech Samek and Christoph H. Lampert},
booktitle={The Fourteenth International Conference on Learning Representations},
year={2026},
url={https://openreview.net/forum?id=C81TnwHiRM}
}
```
|