deepseek-ai
/

DeepSeek-R1

Text Generation

text-generation-inference

Model card Files Files and versions

Instructions to use deepseek-ai/DeepSeek-R1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use deepseek-ai/DeepSeek-R1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="deepseek-ai/DeepSeek-R1", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("deepseek-ai/DeepSeek-R1", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("deepseek-ai/DeepSeek-R1", trust_remote_code=True, device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
HuggingChat
Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use deepseek-ai/DeepSeek-R1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "deepseek-ai/DeepSeek-R1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deepseek-ai/DeepSeek-R1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/deepseek-ai/DeepSeek-R1

How to use deepseek-ai/DeepSeek-R1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "deepseek-ai/DeepSeek-R1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deepseek-ai/DeepSeek-R1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "deepseek-ai/DeepSeek-R1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deepseek-ai/DeepSeek-R1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use deepseek-ai/DeepSeek-R1 with Docker Model Runner:
```
docker model run hf.co/deepseek-ai/DeepSeek-R1
```

Resources

View closed (40)

Upload comment-sample.xlsx

#133 opened over 1 year ago by

non-reasoning data

#132 opened over 1 year ago by

能不能放一些 4bit的权重，现在手里面的卡都不支持FP8

#131 opened over 1 year ago by

For the universe! DeepPhaser.py DeepCoralX.py and DeepSynapse.py

#129 opened over 1 year ago by

Request: Create distill of Mistral Small 24B

#128 opened over 1 year ago by

which vision model is R1 using for text extraction from image or pdfs.

#127 opened over 1 year ago by

PRII

#126 opened over 1 year ago by

Request: DOI

#125 opened over 1 year ago by

Little brother(s) of big DeepSeek-R1 ?

#124 opened over 1 year ago by

Upload gugagagaggagagagga.pdf

#123 opened over 1 year ago by

Change quant_method to bitsandbytes_4bit

#121 opened over 1 year ago by

Unknown quantization type

#120 opened over 1 year ago by

UPdate config.json

#119 opened over 1 year ago by

keerthanaOfficial2001

所以部署一个671B的模型显存需要多少有什么基准的硬件配置？

#118 opened over 1 year ago by

nodejs

#117 opened over 1 year ago by

Distill Compatibility for PC w/ Ryzen 7 Pro 8840HS w/ 780M Graphics 2x32GB RAM 1TB DDR5 SSD

#115 opened over 1 year ago by

Upload gitattributes.txt

#114 opened over 1 year ago by

Introducing Deepseek's TinyZero

#113 opened over 1 year ago by

DeepSeekModerator

Create Kuch v

#112 opened over 1 year ago by

Request: DOI

#111 opened over 1 year ago by

Hassanabbas2975

quantization fp8 error occuring while using pipeline approach or transformer based approach

#110 opened over 1 year ago by

Deepseek-R1

#109 opened over 1 year ago by

deepseek-r1 源码中采用 MLA 架构的 KV Cache 压缩存储策略的实现似乎与文中说的不一致，这是为什么？代码中似乎没实现这个大优化

#108 opened over 1 year ago by

Eating food in a car

#106 opened over 1 year ago by

Update README.md

#103 opened over 1 year ago by

error while downloading model

#102 opened over 1 year ago by

Upload IMG_20250112_172711.jpg

#101 opened over 1 year ago by

help from italy

#100 opened over 1 year ago by

R1 distill to Mistral Small?

#99 opened over 1 year ago by

Running this model on Google Colab?

#98 opened over 1 year ago by

请问下deepseek的同学，能不能train出一个 stable 的 moe model?

#97 opened over 1 year ago by

How to download DeepSeek-R1 7B parameters

#96 opened over 1 year ago by

HuggingFace version does NOT use efficient MLA caching

#95 opened over 1 year ago by

Found a bug

#93 opened over 1 year ago by

Let's Give Credit Where It’s Due: Adding Source Links to AI Responses

#88 opened over 1 year ago by

When will this be available in Transformers library?

#87 opened over 1 year ago by

cannot regenerate (blank respone)

#86 opened over 1 year ago by

A Bug using hugging face API

#85 opened over 1 year ago by

Do we need an authorization access to use this ?

#84 opened over 1 year ago by

where is the source code for this Model ? - what does they prodoudly say by open-source models?

#83 opened over 1 year ago by

智王发布deepseek-r1懒人包，解压即用Deepseek-r1 Lazy Package, easy to decompress and use

#81 opened over 1 year ago by

model-00078-of-000163.safetensors not marked safe?

#80 opened over 1 year ago by

Create Dare

#79 opened over 1 year ago by

problem with using serverless inference

#78 opened over 1 year ago by

Some weird sensorship on unsensitive topic. 对非敏感话题的奇怪审查。

#77 opened over 1 year ago by

Upload dkfoEtm3H4bMcaI0KEJbq.1023.jpeg

#76 opened over 1 year ago by

Update README.md

#75 opened over 1 year ago by

🚩 Report: Ethical issue(s)

#74 opened over 1 year ago by

Typeofprototype

Deepseek-R1 falls: ZW demon redesigns' Nine Birds' Deepseek-R1沦陷：zw魔改版“九只鸟”

#73 opened over 1 year ago by

Consistency, can Deepseek pass?一致性，deepseek能及格吗？

#71 opened over 1 year ago by