LFM3-VL-2B 3D Coder โ€” regular SFT

Supervised fine-tune of LiquidAI/LFM3-VL-2B for generating executable Three.js scenes (and GLB-exportable assets) from natural-language instructions. A quasi-reproduction of ccore's Qwen3.5-4B-3D-Coder study.

This repo is the plain cross-entropy SFT run (loss_type='nll'). An earlier DFT run on this account showed a step-200 generation collapse (repetitive reasoning, no HTML), so this run uses regular SFT at a lower learning rate with a blocking HTML-emission gate every 200 steps. The previous run's artifacts are preserved untouched in AlekseyCalvin/LFM3-VL-2B-3D-Coder.

Training configuration

  • Base: LiquidAI/LFM3-VL-2B, transformers 5.22.1, trust_remote_code=True.
  • Method: TRL SFTTrainer, loss_type='nll', assistant-only loss via pre-masked labels. ChatML with <think> reasoning preserved.
  • QLoRA: bitsandbytes NF4 + double quant, bf16 compute. LoRA r=64, ฮฑ=128, dropout 0, bias none. Language-model targets q/k/v/o/gate/in_proj/out_proj/w1/w2/w3 (incl. LFM3's gate_proj); vision encoder frozen. 65,874,944 trainable params (3.32%).
  • Data: ccore/ThreeJS-3D-Scenes (default, all train) + 1,500 random (seed=42) ccore/ThreeJS-3D-GLB-Assets (conversations) samples. max_length=None, no truncation โ€” examples longer than 24,576 tokens are excluded (an L4 cannot train longer sequences), the rest kept whole.
  • Batch: micro-batch 1, grad-accum 4 โ†’ effective batch 4.
  • Optimiser: AdamW fused, peak LR 2e-5, cosine, 59 warmup steps, weight decay 0.01, ฮฒ=(0.9, 0.999), ฮต=1e-8, bf16, 1 epoch.
  • Hardware: single NVIDIA L4 (24 GB) via Google Colab. Micro-batch 1 and the 24,576-token filter are what fit; Unsloth gradient checkpointing on.
  • Persistence: LoRA checkpoints to checkpoints/step-*/ every 10 steps, live status.json + metrics.jsonl heartbeat every 10 steps, validation loss every 50 steps, generated HTML samples every 200 steps, auto-resume from the latest Hub checkpoint on any fresh runtime.

Evaluation

Generation evals use the LFM3-VL-2B model-card sampling parameters verbatim:

output_ids = model.generate(
    **inputs, do_sample=True,
    temperature=0.1, top_p=1.0, top_k=50, min_p=0.0,
    repetition_penalty=1.1, max_new_tokens=16384,
)

evals/baseline/ holds samples from the original untrained checkpoint under the same regime. Every 200 training steps, 2 fresh samples are generated and a blocking gate counts HTML emission: 0/2 stops the run (COLLAPSE.json). Incomplete outputs (stuck in <think>, truncated HTML) count as failures.

Repository layout

Path Contents
merged/ Final LoRA merged into bf16 base (on completion)
final_adapter/ Final LoRA adapter (on completion)
checkpoints/step-*/ Interim LoRA checkpoints every 10 steps
status.json Live training heartbeat (step, loss, LR, elapsed)
gen_gate_*.json HTML-emission gate results at generation evals
evals/baseline/ Samples from the original, untrained checkpoint
evals/step-*/ Generated Three.js HTML + results at eval steps
gguf/ 4-bit Q4_K_M GGUF (on completion)
training_config.json Full machine-readable config
TECHNICAL_REPORT.md Technical report

Usage

Use the same system prompt as the training data:

Use only the Three.js core ES module from
https://cdnjs.cloudflare.com/ajax/libs/three.js/0.160.0/three.module.min.js.
Do not import addons or legacy examples/js paths; implement any camera
controls directly inside the HTML.
from transformers import AutoModelForImageTextToText, AutoProcessor
import torch

model_id = "AlekseyCalvin/LFM3-VL-2B-3D-Coder-SFT"  # after merge; else base + adapter
model = AutoModelForImageTextToText.from_pretrained(
    model_id, dtype=torch.bfloat16, device_map="auto", trust_remote_code=True)
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)

Results

[TBD โ€” run in progress. Validation curves in metrics.jsonl, generation samples in evals/step-*/, gate verdicts in gen_gate_*.json.]

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for AlekseyCalvin/LFM3-VL-2B-3D-Coder-SFT

Adapter
(12)
this model