Instructions to use vinello28/Tunatron-3-Nano-4B-BF16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use vinello28/Tunatron-3-Nano-4B-BF16 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="vinello28/Tunatron-3-Nano-4B-BF16", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("vinello28/Tunatron-3-Nano-4B-BF16", trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained("vinello28/Tunatron-3-Nano-4B-BF16", trust_remote_code=True, device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use vinello28/Tunatron-3-Nano-4B-BF16 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "vinello28/Tunatron-3-Nano-4B-BF16" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "vinello28/Tunatron-3-Nano-4B-BF16", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/vinello28/Tunatron-3-Nano-4B-BF16
- SGLang
How to use vinello28/Tunatron-3-Nano-4B-BF16 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "vinello28/Tunatron-3-Nano-4B-BF16" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "vinello28/Tunatron-3-Nano-4B-BF16", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "vinello28/Tunatron-3-Nano-4B-BF16" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "vinello28/Tunatron-3-Nano-4B-BF16", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Unsloth Desktop
- Docker Model Runner
How to use vinello28/Tunatron-3-Nano-4B-BF16 with Docker Model Runner:
docker model run hf.co/vinello28/Tunatron-3-Nano-4B-BF16
Tunatron SAST triage model
Summary
Tunatron is a fine-tuned version of
unsloth/NVIDIA-Nemotron-3-Nano-4Bfor static application security testing (SAST) triage. Given a CWE rule, a candidate sink, and the backward code slice that reaches it, it decides whether the sink is genuinely vulnerable and produces a structured JSON finding (vulnerable,vex_status,severity,reasoning,suggested_fix).
The system prompt is in English, and reasoning and suggested_fix are written in English. Supported code languages: C, C++ and Python.
Key Features
- Targeted Triage: Analyzes backward code slices against CWE vulnerability indicators and safe patterns.
- JSON output: Answers with the JSON object described below. With JSON-schema guided decoding, 6 of 600 Juliet responses and 3 of 80 pygoat responses were still not valid JSON with a boolean
vulnerablefield (for example, cut off at the token limit). - Quantized checkpoints: BF16, FP8 and NVFP4 builds (NVIDIA ModelOpt). Only the NVFP4 build has been evaluated (vLLM 0.33.0, NVIDIA GeForce RTX 5070 Ti (16 GB)). NVFP4 is meant for Blackwell GPUs; no other GPU was tested.
Quickstart with vLLM
Serve a checkpoint (vinello28/Tunatron-3-Nano-4B-BF16, -FP8 or -NVFP4) with the settings used in the benchmark (vLLM 0.33.0):
vllm serve vinello28/Tunatron-3-Nano-4B-NVFP4 --served-model-name sast-nemotron --max-model-len 8192 --gpu-memory-utilization 0.80 --max-num-seqs 8 --port 8000 --trust-remote-code --mamba_ssm_cache_dtype float32
The model was trained on chat-formatted prompts with thinking disabled, and the benchmark asks for JSON-schema guided decoding at temperature 0. Send the same request:
import json
import urllib.request
SCHEMA = json.loads("""
{
"type": "object",
"properties": {
"vulnerable": {"type": "boolean"},
"cwe_id": {"type": "string"},
"severity": {"type": "string", "enum": ["NONE", "LOW", "MEDIUM", "HIGH", "CRITICAL"]},
"sink_function": {"type": "string"},
"sink_file": {"type": "string"},
"sink_line": {"type": "integer"},
"vex_status": {"type": "string", "enum": ["affected", "not_affected"]},
"reasoning": {"type": "string"},
"suggested_fix": {"type": "string"}
},
"required": ["vulnerable", "cwe_id", "severity", "sink_function", "sink_file", "sink_line", "vex_status", "reasoning", "suggested_fix"],
"additionalProperties": false
}
""")
SYSTEM = "You are a SAST security analyzer. You receive a CWE rule, a candidate sink and the code trace (backward slice) that reaches it. Decide whether the sink is really vulnerable to that CWE: untrusted data must reach the sink without adequate validation or sanitization. Answer ONLY with a JSON object that conforms to the requested schema. Write \"reasoning\" and \"suggested_fix\" in English, citing files and lines."
USER = (
"## CWE rule\n"
"CWE-78 — OS Command Injection\n"
"Description: An operating system command is built or executed using untrusted input without neutralizing shell metacharacters.\n"
"Vulnerability indicators:\n"
"- User input concatenated or interpolated into a string passed to system/popen/os.system/subprocess\n"
"- subprocess with shell=True and arguments built from external data\n"
"- Unvalidated file names or parameters used as an argument of an executable\n"
"Safe patterns:\n"
"- Argument list without a shell (subprocess.run([...], shell=False)) or execv with a fixed argv\n"
"- Strict allowlist of permitted values before execution\n"
"\n"
"## Candidate sink\n"
"- Function: system\n"
"- File: app.c\n"
"- Line: 43\n"
"\n"
"## Code trace (from the outermost caller to the sink)\n"
"### app.c · run_ping (lines 40-44, depth 0)\n"
"```c\n"
" 40 | void run_ping(const char *host) {\n"
" 41 | char cmd[256];\n"
" 42 | snprintf(cmd, sizeof cmd, \"ping -c 1 %s\", host);\n"
" 43 | system(cmd);\n"
" 44 | }\n"
"```\n"
"\n"
"## Task\n"
"Decide whether the sink `system` is vulnerable to CWE-78. If it is not, answer with \"vulnerable\": false, \"vex_status\": \"not_affected\" and \"severity\": \"NONE\"."
)
body = {
"model": "sast-nemotron",
"messages": [{"role": "system", "content": SYSTEM}, {"role": "user", "content": USER}],
"temperature": 0,
"max_tokens": 768,
"response_format": {"type": "json_schema", "json_schema": {"name": "vex_finding", "strict": True, "schema": SCHEMA}},
"chat_template_kwargs": {"enable_thinking": False},
}
request = urllib.request.Request("http://localhost:8000/v1/chat/completions", json.dumps(body).encode(),
{"Content-Type": "application/json"})
with urllib.request.urlopen(request) as response:
print(json.load(response)["choices"][0]["message"]["content"])
Output
The answer is a JSON object with exactly these fields (severity is one of NONE, LOW, MEDIUM, HIGH, CRITICAL; vex_status is one of affected, not_affected). A sink that is not vulnerable comes with vulnerable: false, vex_status: not_affected and severity: NONE. reasoning and suggested_fix are free text.
{
"vulnerable": true,
"cwe_id": "CWE-78",
"severity": "HIGH",
"sink_function": "system",
"sink_file": "app.c",
"sink_line": 43,
"vex_status": "affected",
"reasoning": "...",
"suggested_fix": "..."
}
Intended use and limitations
- Triage Assistant: Intended as a triage aid for human reviewers, not as a replacement for a SAST engine or a formal security audit.
- Specific Prompt Format: It expects the prompt format of this project (CWE rule, candidate sink, numbered backward slice); other prompt styles are not evaluated.
- Free-text rationale:
reasoningandsuggested_fixare free text written by a 4B-parameter model. They can state things that are not in the code (for example CVE identifiers), so verify them before you rely on them or quote them in a report. - Trained Languages: Trained and evaluated on C, C++ and Python functions from public vulnerability datasets; results may not transfer to other languages or to proprietary code.
- License and Attributions: See the License and attributions section for base model and dataset terms.
Training
- Base model:
unsloth/NVIDIA-Nemotron-3-Nano-4B - Method: DoRA (rank 64, alpha 64) on
q_proj,k_proj,v_proj,o_proj,up_proj,down_proj,in_proj - Recipe (the defaults of
ai-pipeline/train.py): optimizer adamw_8bit; learning rate 0.0002 (cosine, warmup ratio 0.03); 2 epochs; effective batch size 8; max sequence length 8192; seed 3407 - Training code: https://github.com/BlockchainzGurus/tunatron, branch
dev - Hardware, training time, loss curves and package versions of this run are not recorded in this card.
Data
| Split | Examples | Vulnerable | Not vulnerable |
|---|---|---|---|
| train | 5,261 | 2,722 | 2,539 |
| val | 369 | 202 | 167 |
| test | 3,266 | 1,146 | 2,120 |
- Sources (train): secvuleval 2,021, primevul 1,190, juliet-nist 1,080, cvefixes-python 373, codeql-tests 274, pyvul 149, codeql-python 94, pyvul-graph 54, py-synth 26
- Languages (train): c 3,636, cpp 929, python 696
- The prompts and the target rationale are in English.
Evaluation
Measured on the NVFP4 checkpoint (vinello28/Tunatron-3-Nano-4B-NVFP4) with vLLM 0.33.0 on an NVIDIA GeForce RTX 5070 Ti (16 GB): temperature 0, JSON-schema guided decoding, thinking disabled, backward slices up to 5 callers deep. The BF16 and FP8 checkpoints were not measured. The positive class is "vulnerable"; an invalid output counts as "not vulnerable". The Python sets are very small (17 and 16 examples).
| Set | Prompt | Examples | Invalid | Precision | Recall | F1 | FPR |
|---|---|---|---|---|---|---|---|
| Juliet twin pairs | with callers | 600 | 6 | 0.503 | 0.563 | 0.531 | 0.557 |
| Juliet twin pairs | sink only | 600 | 5 | 0.488 | 0.403 | 0.442 | 0.423 |
| Python, CodeQL tests | with callers | 17 | 0 | 0.647 | 1.000 | 0.786 | 1.000 |
| Python, CodeQL tests | sink only | 17 | 0 | 0.647 | 1.000 | 0.786 | 1.000 |
| Python, CVEfixes/PyVul | with callers | 16 | 1 | 0.533 | 1.000 | 0.696 | 0.875 |
| Python, CVEfixes/PyVul | sink only | 16 | 0 | 0.533 | 1.000 | 0.696 | 0.875 |
Twin pairs (Juliet): two functions with the same sink code and opposite labels, so only the callers tell them apart. Accuracy is the share of correct verdicts over the paired examples (0.50 is chance); "both correct" is the share of pairs where both twins are right. The length-controlled subset keeps the pairs whose prompts differ in length by at most 15%. Among the 233 pairs whose prompts differ in length, the vulnerable twin is the longer one in 86.3%.
| Prompt | Pairs | Accuracy | Both correct | Length-controlled pairs | Both correct (length-controlled) |
|---|---|---|---|---|---|
| with callers | 300 | 0.503 | 0.097 | 224 | 0.120 |
| sink only | 300 | 0.490 | 0.017 | 224 | 0.009 |
Frozen list (adeyosemanputra/pygoat at 2aa4cc14): 80 sinks, 69 safe and 11 vulnerable. 61 of the 69 safe sinks (88.4%) were still flagged as vulnerable, and 11 of the 11 vulnerable sinks were found. 3 responses were cut off at 768 tokens, 3 were invalid, and 4 named a CVE identifier in reasoning.
Efficiency
Measured on the NVFP4 checkpoint only, with vLLM 0.33.0 on an NVIDIA GeForce RTX 5070 Ti (16 GB), served with the settings of the quickstart (--gpu-memory-utilization 0.80, --max-num-seqs 8).
| Format | Weights in VRAM (GiB) | KV cache (tokens) | Generation throughput (tok/s) | Disk (GB) |
|---|---|---|---|---|
| nvfp4 | 3.4 | 161,716 | 1,230 | 3.6 |
Throughput is the mean of vLLM's periodic log while at least 6 of the 8 request slots were busy during the evaluation above, so it is a rough figure. BF16 and FP8 were not measured.
Files and reproduction
Each format is published as its own repository, all with this same card:
vinello28/Tunatron-3-Nano-4B-BF16: merged BF16 model (full precision).vinello28/Tunatron-3-Nano-4B-FP8: FP8 W8A8 quantization with FP8 KV cache (NVIDIA ModelOpt).vinello28/Tunatron-3-Nano-4B-NVFP4: NVFP4 quantization for Blackwell Tensor Cores (NVIDIA ModelOpt).
A repository holds model.safetensors, the configuration, the tokenizer and the chat template, the model code (modeling_nemotron_h.py, configuration_nemotron_h.py), LICENSE, this card (README.md) and its charts (model_card_assets/).
The notebook also writes the unmerged DoRA adapter, the training log, the package versions, the trained commit and the raw benchmark output next to the checkpoints, but these are not uploaded to the repositories.
To reproduce: run notebooks/train_colab.ipynb on a GPU runtime. The training section runs ai-pipeline/train.py, section 10 runs quantize.py, and section 11 runs benchmark.py hardware and builds this card.
License and attributions
This model is a Derivative Model of NVIDIA Nemotron 3 Nano 4B and is distributed under the NVIDIA Nemotron Open Model License; the license text is available at that address.
Licensed by NVIDIA Corporation under the NVIDIA Nemotron Model License.
The training, validation and test data come from the public sources below. The license column is what each source states. The code in these datasets was collected from open-source projects, and each snippet remains under the license of its original project. Check these terms before you reuse the model or the data in a way that depends on them.
| Source | License stated by the source | Reference |
|---|---|---|
| PrimeVul | MIT | Ding et al., "Vulnerability Detection with Code Language Models: How Far Are We?", arXiv:2403.18624 (2024); repository DLVulDet/PrimeVul, mirror colin/PrimeVul |
| SecVulEval | MIT (declared in the dataset metadata) | Ahmed et al., "SecVulEval: Benchmarking LLMs for Real-World C/C++ Vulnerability Detection", arXiv:2505.19828 (2025); dataset arag0rn/SecVulEval |
| CVEfixes | Data: CC BY 4.0; code: MIT | Bhandari, Naseer and Moonen, "CVEfixes: Automated Collection of Vulnerabilities and Their Fixes from Open-Source Software", PROMISE '21 (ACM); used through the merged dataset janhvi-ag/cvefixes_pyvul_merged, which states no license of its own. The pyvul-graph examples also read the callers of each function from the project repositories at the fix commit; each fragment stays under the license of its project |
| PyVul | No license stated by its authors (the repository has no LICENSE file) | Quan, Wang, Li, Zhuo, Chen and Du, "An Empirical Study of Vulnerabilities in Python Packages and Their Detection", arXiv:2509.04260 (2025); repository billquan/PyVul, used through the same merged dataset |
| Juliet C/C++ 1.3 | Not subject to copyright in the United States; CC0 1.0 where NIST holds rights abroad | NSA Center for Assured Software, distributed by NIST SARD (https://samate.nist.gov/SARD/test-suites/112) |
| CodeQL security tests | MIT (GitHub, Inc.) | Test files of github/codeql |
| Synthetic Python packages | AGPL-3.0-or-later, like this repository (generated by its own templates, no third-party content) | ai-pipeline/synth_python.py in this repository |
- Downloads last month
- 701
Model tree for vinello28/Tunatron-3-Nano-4B-BF16
Base model
nvidia/NVIDIA-Nemotron-Nano-12B-v2-Base