ThreatSage-12B 🔍

ThreatSage-12B is an open-source cybersecurity-specialized language model fine-tuned from Google Gemma 4 12B. Built for the security community, it excels at threat intelligence analysis, MITRE ATT&CK classification, penetration testing report interpretation, and incident response reasoning.

Key Features

• 🎯 Threat intelligence analysis — APT campaigns, IOCs, TTPs
• 🛡️ MITRE ATT&CK v16 tactic and technique classification
• 📋 Penetration testing report analysis and remediation advice
• 🔍 Incident response guidance and threat hunting support
• 👁️ Multimodal: analyze SIEM screenshots, network diagrams, security dashboards
• 🌍 Multilingual: English and Spanish

Model Details

Property	Value
Base model	google/gemma-4-12b-it
Architecture	Decoder-only, encoder-free multimodal
Parameters	12B
Context window	256K tokens
Modalities	Text, Image, Audio, Video
Adapter	LoRA (r=32, alpha=64)
Precision	BF16
License	Apache 2.0

Training Data

Dataset	Samples	Focus
reloading0101/threat-intelligence-dataset	52,279	CTI, APTs, IOCs, campaigns
sarahwei/cyber_MITRE_tactic_CTI_dataset_v16	14,008	MITRE ATT&CK v16
CJJones/Synthetic_PenTest_Reports	752	Pentest reports
Total	67,039

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_model = "google/gemma-4-12b-it"
adapter = "Sakeador/ThreatSage-12B"

tokenizer = AutoTokenizer.from_pretrained(base_model)
model = AutoModelForCausalLM.from_pretrained(
    base_model,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
model = PeftModel.from_pretrained(model, adapter)

messages = [
    {"role": "user", "content": "Analyze this Wazuh alert and identify the MITRE ATT&CK technique."}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

Hardware Requirements

Setup	VRAM	Quantization
Minimum	8 GB	Q4
Recommended	16 GB	BF16 (native)
Optimal	3× 16 GB	BF16 tensor-parallel

Training Infrastructure

• Hardware: NVIDIA GeForce RTX 5060 Ti (16 GB, sm_120 Blackwell)
• Framework: Unsloth + SFTTrainer
• Epochs: 3
• Sequence length: 2048

License

Apache 2.0 — same as the base model Gemma 4 12B.

Acknowledgements

Thanks to the open-source community, Google DeepMind for Gemma 4, and all the dataset creators who made this possible.