Instructions to use yutaozhu94/INTERS-Mistral-7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use yutaozhu94/INTERS-Mistral-7b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="yutaozhu94/INTERS-Mistral-7b")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("yutaozhu94/INTERS-Mistral-7b")
model = AutoModelForCausalLM.from_pretrained("yutaozhu94/INTERS-Mistral-7b")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use yutaozhu94/INTERS-Mistral-7b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "yutaozhu94/INTERS-Mistral-7b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "yutaozhu94/INTERS-Mistral-7b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/yutaozhu94/INTERS-Mistral-7b

SGLang

How to use yutaozhu94/INTERS-Mistral-7b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "yutaozhu94/INTERS-Mistral-7b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "yutaozhu94/INTERS-Mistral-7b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "yutaozhu94/INTERS-Mistral-7b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "yutaozhu94/INTERS-Mistral-7b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use yutaozhu94/INTERS-Mistral-7b with Docker Model Runner:
```
docker model run hf.co/yutaozhu94/INTERS-Mistral-7b
```

This is the model trained by INTERS. The backbone model is Mistral-7b-v0.1.

INTERS: Unlocking the Power of Large Language Models in Search with Instruction Tuning

Authors: Yutao Zhu, Peitian Zhang, Chenghao Zhang, Yifei Chen, Binyu Xie, Zhicheng Dou, Zheng Liu, and Ji-Rong Wen

📃 ArXiv Paper • 💡 GitHub

Introduction

Large language models (LLMs) have demonstrated impressive capabilities in various natural language processing tasks. Despite this, their application to information retrieval (IR) tasks is still challenging due to the infrequent occurrence of many IR-specific concepts in natural language. While prompt-based methods can provide task descriptions to LLMs, they often fall short in facilitating a comprehensive understanding and execution of IR tasks, thereby limiting LLMs' applicability. To address this gap, in this work, we explore the potential of instruction tuning to enhance LLMs' proficiency in IR tasks. We introduce a novel instruction tuning dataset, INTERS, encompassing 20 tasks across three fundamental IR categories: query understanding, document understanding, and query-document relationship understanding. The data are derived from 43 distinct datasets with manually written templates. Our empirical results reveal that INTERS significantly boosts the performance of various publicly available LLMs, such as LLaMA, Mistral, and Phi, in IR tasks. Furthermore, we conduct extensive experiments to analyze the effects of instruction design, template diversity, few-shot demonstrations, and the volume of instructions on performance.

Prompt

If you want to test the model on some IR tasks, please refer to link for writing prompts.

Downloads last month: 765

Dataset used to train yutaozhu94/INTERS-Mistral-7b

Paper for yutaozhu94/INTERS-Mistral-7b

INTERS: Unlocking the Power of Large Language Models in Search with Instruction Tuning

Paper • 2401.06532 • Published Jan 12, 2024 • 13