LFM3-VL-2B 3D Coder โ regular SFT
Supervised fine-tune of LiquidAI/LFM3-VL-2B for generating executable Three.js scenes (and GLB-exportable assets) from natural-language instructions. A quasi-reproduction of ccore's Qwen3.5-4B-3D-Coder study.
This repo is the plain cross-entropy SFT run (loss_type='nll'). An earlier
DFT run on this account showed a step-200 generation collapse (repetitive
reasoning, no HTML), so this run uses regular SFT at a lower learning rate with
a blocking HTML-emission gate every 200 steps. The previous run's artifacts are
preserved untouched in
AlekseyCalvin/LFM3-VL-2B-3D-Coder.
Training configuration
- Base:
LiquidAI/LFM3-VL-2B,transformers 5.22.1,trust_remote_code=True. - Method: TRL
SFTTrainer,loss_type='nll', assistant-only loss via pre-masked labels. ChatML with<think>reasoning preserved. - QLoRA: bitsandbytes NF4 + double quant, bf16 compute. LoRA r=64, ฮฑ=128,
dropout 0, bias none. Language-model targets
q/k/v/o/gate/in_proj/out_proj/w1/w2/w3(incl. LFM3'sgate_proj); vision encoder frozen. 65,874,944 trainable params (3.32%). - Data:
ccore/ThreeJS-3D-Scenes(default, all train) + 1,500 random (seed=42)ccore/ThreeJS-3D-GLB-Assets(conversations) samples.max_length=None, no truncation โ examples longer than 24,576 tokens are excluded (an L4 cannot train longer sequences), the rest kept whole. - Batch: micro-batch 1, grad-accum 4 โ effective batch 4.
- Optimiser: AdamW fused, peak LR 2e-5, cosine, 59 warmup steps, weight decay 0.01, ฮฒ=(0.9, 0.999), ฮต=1e-8, bf16, 1 epoch.
- Hardware: single NVIDIA L4 (24 GB) via Google Colab. Micro-batch 1 and the 24,576-token filter are what fit; Unsloth gradient checkpointing on.
- Persistence: LoRA checkpoints to
checkpoints/step-*/every 10 steps, livestatus.json+metrics.jsonlheartbeat every 10 steps, validation loss every 50 steps, generated HTML samples every 200 steps, auto-resume from the latest Hub checkpoint on any fresh runtime.
Evaluation
Generation evals use the LFM3-VL-2B model-card sampling parameters verbatim:
output_ids = model.generate(
**inputs, do_sample=True,
temperature=0.1, top_p=1.0, top_k=50, min_p=0.0,
repetition_penalty=1.1, max_new_tokens=16384,
)
evals/baseline/ holds samples from the original untrained checkpoint under
the same regime. Every 200 training steps, 2 fresh samples are generated and a
blocking gate counts HTML emission: 0/2 stops the run (COLLAPSE.json).
Incomplete outputs (stuck in <think>, truncated HTML) count as failures.
Repository layout
| Path | Contents |
|---|---|
merged/ |
Final LoRA merged into bf16 base (on completion) |
final_adapter/ |
Final LoRA adapter (on completion) |
checkpoints/step-*/ |
Interim LoRA checkpoints every 10 steps |
status.json |
Live training heartbeat (step, loss, LR, elapsed) |
gen_gate_*.json |
HTML-emission gate results at generation evals |
evals/baseline/ |
Samples from the original, untrained checkpoint |
evals/step-*/ |
Generated Three.js HTML + results at eval steps |
gguf/ |
4-bit Q4_K_M GGUF (on completion) |
training_config.json |
Full machine-readable config |
TECHNICAL_REPORT.md |
Technical report |
Usage
Use the same system prompt as the training data:
Use only the Three.js core ES module from
https://cdnjs.cloudflare.com/ajax/libs/three.js/0.160.0/three.module.min.js.
Do not import addons or legacy examples/js paths; implement any camera
controls directly inside the HTML.
from transformers import AutoModelForImageTextToText, AutoProcessor
import torch
model_id = "AlekseyCalvin/LFM3-VL-2B-3D-Coder-SFT" # after merge; else base + adapter
model = AutoModelForImageTextToText.from_pretrained(
model_id, dtype=torch.bfloat16, device_map="auto", trust_remote_code=True)
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
Results
[TBD โ run in progress. Validation curves in metrics.jsonl, generation
samples in evals/step-*/, gate verdicts in gen_gate_*.json.]