Instructions to use XINLI1997/ReMaRk-Qwen3.5-4B-RedditV2-SFT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use XINLI1997/ReMaRk-Qwen3.5-4B-RedditV2-SFT with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="XINLI1997/ReMaRk-Qwen3.5-4B-RedditV2-SFT")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("XINLI1997/ReMaRk-Qwen3.5-4B-RedditV2-SFT")
model = AutoModelForMultimodalLM.from_pretrained("XINLI1997/ReMaRk-Qwen3.5-4B-RedditV2-SFT", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use XINLI1997/ReMaRk-Qwen3.5-4B-RedditV2-SFT with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "XINLI1997/ReMaRk-Qwen3.5-4B-RedditV2-SFT"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "XINLI1997/ReMaRk-Qwen3.5-4B-RedditV2-SFT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/XINLI1997/ReMaRk-Qwen3.5-4B-RedditV2-SFT

SGLang

How to use XINLI1997/ReMaRk-Qwen3.5-4B-RedditV2-SFT with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "XINLI1997/ReMaRk-Qwen3.5-4B-RedditV2-SFT" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "XINLI1997/ReMaRk-Qwen3.5-4B-RedditV2-SFT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "XINLI1997/ReMaRk-Qwen3.5-4B-RedditV2-SFT" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "XINLI1997/ReMaRk-Qwen3.5-4B-RedditV2-SFT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use XINLI1997/ReMaRk-Qwen3.5-4B-RedditV2-SFT with Docker Model Runner:
```
docker model run hf.co/XINLI1997/ReMaRk-Qwen3.5-4B-RedditV2-SFT
```

ReMaRk-Qwen3.5-4B-RedditV2-SFT

SFT initialisation on Reddit-v2 (movies), released with the paper Many Right Answers, One Training Example: Removing Responder Noise from GRPO for LLM Recommenders (Xin Li, Hao Jiang, Jinghao Guo, Weisi Lin, Chau Yuen). arXiv: 2610.17771.

Code: https://github.com/LiXin97/ReMaRk · All checkpoints and data: ReMaRk collection

Checkpoint


Method	SFT initialisation on Reddit-v2 (movies)
Data	The processed Reddit-v2 release of Rank-GRPO (Zhu et al.), yaochenzhu/reddit-v2; SFT targets come from that release
Base model	Qwen/Qwen3.5-4B-Base
Initialised from	Qwen/Qwen3.5-4B-Base
Selected step	1000 (SFT steps)
Paper reference	Table 2 (SFT row, 4B)
Weights	bf16 safetensors, 1 file, 10.35 GB (5.17 B stored parameters, including the unused vision encoder). The file also stores `lm_head.weight`, a bit-identical copy of the tied input embeddings.

Usage

# pip install "transformers>=5.3.0" accelerate torch
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "XINLI1997/ReMaRk-Qwen3.5-4B-RedditV2-SFT"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16, device_map="auto")

messages = [{"role": "user", "content": "I loved Inception, Arrival and Interstellar. What other movies would you recommend? Give me 20 titles."}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=False, return_tensors="pt", return_dict=True).to(model.device)
output = model.generate(**inputs, max_new_tokens=512, do_sample=False)
print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

The model answers with a recommendation list, one title (year) (or name (year) for music) per line. The exact instruction header wrapped around each Reddit request in training and evaluation, the decoding settings and the evaluation scripts are in the code repository; the request above is only an illustration.

Notes:

Qwen3.5 checkpoints need transformers>=5.3.0. They keep the Qwen3.5 multimodal layout (Qwen3_5ForConditionalGeneration); AutoModelForCausalLM loads the language model only (vLLM loads the folder directly; preprocessor_config.json and video_preprocessor_config.json are included for that). When training with TRL, pass the text tokenizer as processing_class.
The snippet uses non-thinking mode (enable_thinking=False).
trainer_state.json (training curves of the run) is included; no optimizer state.

Training

Supervised fine-tuning of the base model on the SFT targets of Reddit-v2 (movies). Full configurations and scripts are in the code repository.

Evaluation

This checkpoint is reported in the paper, Table 2 (SFT row, 4B); see the paper for its scores.

Limitations

Trained for conversational recommendation in one domain (movies); it is not a general assistant.
Recommendations reflect what the Reddit community recommended and the catalog used for grounding; popular items are favoured and the lists can carry the community's biases.
English only.

License

Apache-2.0, inherited from the base model Qwen/Qwen3.5-4B-Base (see LICENSE). Third-party data keep their own terms (Reddit; UCSD Book Graph: academic use; MusicBrainz: CC0); see the dataset card of yaochenzhu/reddit-v2.

Citation

@misc{li2026remark,
  title         = {Many Right Answers, One Training Example: Removing Responder Noise from GRPO for LLM Recommenders},
  author        = {Li, Xin and Jiang, Hao and Guo, Jinghao and Lin, Weisi and Yuen, Chau},
  year          = {2026},
  eprint        = {2610.17771},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/2610.17771}
}