Tunatron SAST triage model

Base model: Nemotron 3 Nano 4B Fine-tuning: DoRA Task: SAST triage Formats: BF16 / FP8 / NVFP4 License: NVIDIA Nemotron Open Model

Python: pipeline PyTorch: training Hugging Face: Transformers / PEFT / TRL vLLM: serving NVIDIA: ModelOpt quantization Rust: sast-engine slicer

Quickstart

Summary

Tunatron is a fine-tuned version of unsloth/NVIDIA-Nemotron-3-Nano-4B for static application security testing (SAST) triage. Given a CWE rule, a candidate sink, and the backward code slice that reaches it, it decides whether the sink is genuinely vulnerable and produces a structured JSON finding (vulnerable, vex_status, severity, reasoning, suggested_fix).

The system prompt is in English, and reasoning and suggested_fix are written in English. Supported code languages: C, C++ and Python.

Key Features

  • Targeted Triage: Analyzes backward code slices against CWE vulnerability indicators and safe patterns.
  • JSON output: Answers with the JSON object described below. With JSON-schema guided decoding, 6 of 600 Juliet responses and 3 of 80 pygoat responses were still not valid JSON with a boolean vulnerable field (for example, cut off at the token limit).
  • Quantized checkpoints: BF16, FP8 and NVFP4 builds (NVIDIA ModelOpt). Only the NVFP4 build has been evaluated (vLLM 0.33.0, NVIDIA GeForce RTX 5070 Ti (16 GB)). NVFP4 is meant for Blackwell GPUs; no other GPU was tested.

Quickstart with vLLM

Serve a checkpoint (vinello28/Tunatron-3-Nano-4B-BF16, -FP8 or -NVFP4) with the settings used in the benchmark (vLLM 0.33.0):

vllm serve vinello28/Tunatron-3-Nano-4B-NVFP4 --served-model-name sast-nemotron --max-model-len 8192 --gpu-memory-utilization 0.80 --max-num-seqs 8 --port 8000 --trust-remote-code --mamba_ssm_cache_dtype float32

The model was trained on chat-formatted prompts with thinking disabled, and the benchmark asks for JSON-schema guided decoding at temperature 0. Send the same request:

import json
import urllib.request

SCHEMA = json.loads("""
{
  "type": "object",
  "properties": {
    "vulnerable": {"type": "boolean"},
    "cwe_id": {"type": "string"},
    "severity": {"type": "string", "enum": ["NONE", "LOW", "MEDIUM", "HIGH", "CRITICAL"]},
    "sink_function": {"type": "string"},
    "sink_file": {"type": "string"},
    "sink_line": {"type": "integer"},
    "vex_status": {"type": "string", "enum": ["affected", "not_affected"]},
    "reasoning": {"type": "string"},
    "suggested_fix": {"type": "string"}
  },
  "required": ["vulnerable", "cwe_id", "severity", "sink_function", "sink_file", "sink_line", "vex_status", "reasoning", "suggested_fix"],
  "additionalProperties": false
}
""")

SYSTEM = "You are a SAST security analyzer. You receive a CWE rule, a candidate sink and the code trace (backward slice) that reaches it. Decide whether the sink is really vulnerable to that CWE: untrusted data must reach the sink without adequate validation or sanitization. Answer ONLY with a JSON object that conforms to the requested schema. Write \"reasoning\" and \"suggested_fix\" in English, citing files and lines."

USER = (
    "## CWE rule\n"
    "CWE-78 — OS Command Injection\n"
    "Description: An operating system command is built or executed using untrusted input without neutralizing shell metacharacters.\n"
    "Vulnerability indicators:\n"
    "- User input concatenated or interpolated into a string passed to system/popen/os.system/subprocess\n"
    "- subprocess with shell=True and arguments built from external data\n"
    "- Unvalidated file names or parameters used as an argument of an executable\n"
    "Safe patterns:\n"
    "- Argument list without a shell (subprocess.run([...], shell=False)) or execv with a fixed argv\n"
    "- Strict allowlist of permitted values before execution\n"
    "\n"
    "## Candidate sink\n"
    "- Function: system\n"
    "- File: app.c\n"
    "- Line: 43\n"
    "\n"
    "## Code trace (from the outermost caller to the sink)\n"
    "### app.c · run_ping (lines 40-44, depth 0)\n"
    "```c\n"
    "   40 | void run_ping(const char *host) {\n"
    "   41 |     char cmd[256];\n"
    "   42 |     snprintf(cmd, sizeof cmd, \"ping -c 1 %s\", host);\n"
    "   43 |     system(cmd);\n"
    "   44 | }\n"
    "```\n"
    "\n"
    "## Task\n"
    "Decide whether the sink `system` is vulnerable to CWE-78. If it is not, answer with \"vulnerable\": false, \"vex_status\": \"not_affected\" and \"severity\": \"NONE\"."
)

body = {
    "model": "sast-nemotron",
    "messages": [{"role": "system", "content": SYSTEM}, {"role": "user", "content": USER}],
    "temperature": 0,
    "max_tokens": 768,
    "response_format": {"type": "json_schema", "json_schema": {"name": "vex_finding", "strict": True, "schema": SCHEMA}},
    "chat_template_kwargs": {"enable_thinking": False},
}
request = urllib.request.Request("http://localhost:8000/v1/chat/completions", json.dumps(body).encode(),
                                 {"Content-Type": "application/json"})
with urllib.request.urlopen(request) as response:
    print(json.load(response)["choices"][0]["message"]["content"])

Output

The answer is a JSON object with exactly these fields (severity is one of NONE, LOW, MEDIUM, HIGH, CRITICAL; vex_status is one of affected, not_affected). A sink that is not vulnerable comes with vulnerable: false, vex_status: not_affected and severity: NONE. reasoning and suggested_fix are free text.

{
  "vulnerable": true,
  "cwe_id": "CWE-78",
  "severity": "HIGH",
  "sink_function": "system",
  "sink_file": "app.c",
  "sink_line": 43,
  "vex_status": "affected",
  "reasoning": "...",
  "suggested_fix": "..."
}

Intended use and limitations

  • Triage Assistant: Intended as a triage aid for human reviewers, not as a replacement for a SAST engine or a formal security audit.
  • Specific Prompt Format: It expects the prompt format of this project (CWE rule, candidate sink, numbered backward slice); other prompt styles are not evaluated.
  • Free-text rationale: reasoning and suggested_fix are free text written by a 4B-parameter model. They can state things that are not in the code (for example CVE identifiers), so verify them before you rely on them or quote them in a report.
  • Trained Languages: Trained and evaluated on C, C++ and Python functions from public vulnerability datasets; results may not transfer to other languages or to proprietary code.
  • License and Attributions: See the License and attributions section for base model and dataset terms.

Training

  • Base model: unsloth/NVIDIA-Nemotron-3-Nano-4B
  • Method: DoRA (rank 64, alpha 64) on q_proj, k_proj, v_proj, o_proj, up_proj, down_proj, in_proj
  • Recipe (the defaults of ai-pipeline/train.py): optimizer adamw_8bit; learning rate 0.0002 (cosine, warmup ratio 0.03); 2 epochs; effective batch size 8; max sequence length 8192; seed 3407
  • Training code: https://github.com/BlockchainzGurus/tunatron, branch dev
  • Hardware, training time, loss curves and package versions of this run are not recorded in this card.

Data

Split Examples Vulnerable Not vulnerable
train 5,261 2,722 2,539
val 369 202 167
test 3,266 1,146 2,120
  • Sources (train): secvuleval 2,021, primevul 1,190, juliet-nist 1,080, cvefixes-python 373, codeql-tests 274, pyvul 149, codeql-python 94, pyvul-graph 54, py-synth 26
  • Languages (train): c 3,636, cpp 929, python 696
  • The prompts and the target rationale are in English.

Evaluation

Measured on the NVFP4 checkpoint (vinello28/Tunatron-3-Nano-4B-NVFP4) with vLLM 0.33.0 on an NVIDIA GeForce RTX 5070 Ti (16 GB): temperature 0, JSON-schema guided decoding, thinking disabled, backward slices up to 5 callers deep. The BF16 and FP8 checkpoints were not measured. The positive class is "vulnerable"; an invalid output counts as "not vulnerable". The Python sets are very small (17 and 16 examples).

Set Prompt Examples Invalid Precision Recall F1 FPR
Juliet twin pairs with callers 600 6 0.503 0.563 0.531 0.557
Juliet twin pairs sink only 600 5 0.488 0.403 0.442 0.423
Python, CodeQL tests with callers 17 0 0.647 1.000 0.786 1.000
Python, CodeQL tests sink only 17 0 0.647 1.000 0.786 1.000
Python, CVEfixes/PyVul with callers 16 1 0.533 1.000 0.696 0.875
Python, CVEfixes/PyVul sink only 16 0 0.533 1.000 0.696 0.875

Twin pairs (Juliet): two functions with the same sink code and opposite labels, so only the callers tell them apart. Accuracy is the share of correct verdicts over the paired examples (0.50 is chance); "both correct" is the share of pairs where both twins are right. The length-controlled subset keeps the pairs whose prompts differ in length by at most 15%. Among the 233 pairs whose prompts differ in length, the vulnerable twin is the longer one in 86.3%.

Prompt Pairs Accuracy Both correct Length-controlled pairs Both correct (length-controlled)
with callers 300 0.503 0.097 224 0.120
sink only 300 0.490 0.017 224 0.009

Frozen list (adeyosemanputra/pygoat at 2aa4cc14): 80 sinks, 69 safe and 11 vulnerable. 61 of the 69 safe sinks (88.4%) were still flagged as vulnerable, and 11 of the 11 vulnerable sinks were found. 3 responses were cut off at 768 tokens, 3 were invalid, and 4 named a CVE identifier in reasoning.

Efficiency

Measured on the NVFP4 checkpoint only, with vLLM 0.33.0 on an NVIDIA GeForce RTX 5070 Ti (16 GB), served with the settings of the quickstart (--gpu-memory-utilization 0.80, --max-num-seqs 8).

Format Weights in VRAM (GiB) KV cache (tokens) Generation throughput (tok/s) Disk (GB)
nvfp4 3.4 161,716 1,230 3.6

Throughput is the mean of vLLM's periodic log while at least 6 of the 8 request slots were busy during the evaluation above, so it is a rough figure. BF16 and FP8 were not measured.

Files and reproduction

Each format is published as its own repository, all with this same card:

  • vinello28/Tunatron-3-Nano-4B-BF16: merged BF16 model (full precision).
  • vinello28/Tunatron-3-Nano-4B-FP8: FP8 W8A8 quantization with FP8 KV cache (NVIDIA ModelOpt).
  • vinello28/Tunatron-3-Nano-4B-NVFP4: NVFP4 quantization for Blackwell Tensor Cores (NVIDIA ModelOpt).

A repository holds model.safetensors, the configuration, the tokenizer and the chat template, the model code (modeling_nemotron_h.py, configuration_nemotron_h.py), LICENSE, this card (README.md) and its charts (model_card_assets/).

The notebook also writes the unmerged DoRA adapter, the training log, the package versions, the trained commit and the raw benchmark output next to the checkpoints, but these are not uploaded to the repositories.

To reproduce: run notebooks/train_colab.ipynb on a GPU runtime. The training section runs ai-pipeline/train.py, section 10 runs quantize.py, and section 11 runs benchmark.py hardware and builds this card.

License and attributions

This model is a Derivative Model of NVIDIA Nemotron 3 Nano 4B and is distributed under the NVIDIA Nemotron Open Model License; the license text is available at that address.

Licensed by NVIDIA Corporation under the NVIDIA Nemotron Model License.

The training, validation and test data come from the public sources below. The license column is what each source states. The code in these datasets was collected from open-source projects, and each snippet remains under the license of its original project. Check these terms before you reuse the model or the data in a way that depends on them.

Source License stated by the source Reference
PrimeVul MIT Ding et al., "Vulnerability Detection with Code Language Models: How Far Are We?", arXiv:2403.18624 (2024); repository DLVulDet/PrimeVul, mirror colin/PrimeVul
SecVulEval MIT (declared in the dataset metadata) Ahmed et al., "SecVulEval: Benchmarking LLMs for Real-World C/C++ Vulnerability Detection", arXiv:2505.19828 (2025); dataset arag0rn/SecVulEval
CVEfixes Data: CC BY 4.0; code: MIT Bhandari, Naseer and Moonen, "CVEfixes: Automated Collection of Vulnerabilities and Their Fixes from Open-Source Software", PROMISE '21 (ACM); used through the merged dataset janhvi-ag/cvefixes_pyvul_merged, which states no license of its own. The pyvul-graph examples also read the callers of each function from the project repositories at the fix commit; each fragment stays under the license of its project
PyVul No license stated by its authors (the repository has no LICENSE file) Quan, Wang, Li, Zhuo, Chen and Du, "An Empirical Study of Vulnerabilities in Python Packages and Their Detection", arXiv:2509.04260 (2025); repository billquan/PyVul, used through the same merged dataset
Juliet C/C++ 1.3 Not subject to copyright in the United States; CC0 1.0 where NIST holds rights abroad NSA Center for Assured Software, distributed by NIST SARD (https://samate.nist.gov/SARD/test-suites/112)
CodeQL security tests MIT (GitHub, Inc.) Test files of github/codeql
Synthetic Python packages AGPL-3.0-or-later, like this repository (generated by its own templates, no third-party content) ai-pipeline/synth_python.py in this repository
Downloads last month
701
Safetensors
Model size
4B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for vinello28/Tunatron-3-Nano-4B-BF16

Papers for vinello28/Tunatron-3-Nano-4B-BF16