Temper-1-1.5B

Temper-1-1.5B is Qwen2.5-Coder-1.5B-Instruct fine-tuned for writing Rust. On the standard HumanEval-Rust benchmark it solves 50.5% of tasks on the first try, more than Qwen2.5-Coder-3B at twice its size.

The model was trained on the same 10 000 Rust examples as Temper-1-0.5B. Each example was written by DeepSeek-V4-Flash and passed the compiler and tests. This model had no reinforcement learning.

Website: temper-ai.pages.dev/temper-1.5b

Results

HumanEval-Rust (multiple-rs from MultiPL-E, 156 tasks), %. pass@1 is the share of tasks solved on the first try, pass@10 the share solved in at least one of 10 tries.

Model	Size	pass@1	pass@10
Qwen2.5-Coder-0.5B	0.5B	16.0	27.9
StarCoder2-3B	3B	25.1	37.8
Temper-1-0.5B	0.5B	25.2	38.0
Qwen2.5-Coder-1.5B	1.5B	39.3	56.6
Qwen2.5-Coder-3B	3B	44.7	67.4
Temper-1-1.5B	1.5B	50.5	67.0
Qwen3.5-9B	9B	56.7	73.0
Qwen2.5-Coder-7B	7B	57.3	74.3

Every model ran through bigcode-evaluation-harness at commit 8fc5bae with the BigCode Models Leaderboard settings. The model completes code without a chat template, with temperature 0.2, top_p 0.95, 50 samples per task and 512 tokens including the prompt. Qwen2.5-Coder and StarCoder2 are the base versions. Qwen3.5-9B is the post-trained version. StarCoder2-3B scores 25.1 here and 24.5 on the leaderboard.

The Temper-1-0.5B card reports HumanEval-RS with the task sent as a chat message, so its numbers there differ from this table.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

name = "temper-ai/Temper-1-1.5B"
tokenizer = AutoTokenizer.from_pretrained(name)
model = AutoModelForCausalLM.from_pretrained(name, dtype="auto", device_map="auto")

messages = [{"role": "user", "content": "Write a Rust function that returns the n-th Fibonacci number."}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_dict=True, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

The default generation config samples with temperature 0.7, top_p 0.8, top_k 20 and repetition penalty 1.1.

A GGUF version for llama.cpp, LM Studio and Ollama is in temper-ai/Temper-1-1.5B-GGUF.

Limitations

With ten tries Temper-1-1.5B solves 67.0% of tasks, about the same as Qwen2.5-Coder-3B with 67.4%. The model was trained only on Rust tasks with requests in English. Requests in other languages and questions unrelated to Rust may get made-up answers or refusals. It was not trained for fill-in-the-middle, so it is not a drop-in model for inline completion in an editor. The training data has few tasks on async code and web frameworks.

License

Apache 2.0, same as Qwen2.5-Coder-1.5B-Instruct.