camtr0n/unrender-qwen3-4b-lora-v3

unrender — Qwen3-4B LoRA v3

LoRA adapters that reconstruct compilable LaTeX from a rendered PDF's extracted text, word boxes, and declared font names.

Standard PEFT adapters — load with transformers on any platform.

• Base: Qwen/Qwen3-4B-Instruct-2507, QLoRA over a 4-bit nf4 base
• Adapters: 14.68M params (0.36%), last 16 layers, rank 16, 8192 context
• Data: 210 examples from 308 repos, no eval source in training

Results — 34 held-out scenarios

model	compiled	glyphs lost	same-line	right-align	recall	score
Qwen3-4B base (bf16)	25/34	138	0.584	0.386	0.569	0.2486
+ LoRA v3 (this)	27/34	299	0.641	0.395	0.578	0.2388

Behavior Spec

Given a PDF's extracted text, word boxes, and declared fonts, the model emits a single .tex that compiles on the first XeLaTeX run with zero missing-character warnings. The rendered result matches the source page count, with matched word positions within N = 3pt.

N is calibrated by perturbation, not chosen: the residual error distribution is bimodal (~0 or ~8pt+), so any N in 1–7pt behaves identically.

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-4B-Instruct-2507")
m = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B-Instruct-2507")
m = PeftModel.from_pretrained(m, "camtr0n/unrender-qwen3-4b-lora-v3").merge_and_unload()

Generation: do_sample=True, temperature=0.7, top_p=0.8, top_k=20, and max_new_tokens at least 10000 — outputs are long and a tight cap truncates mid-command.

Input is an extraction, not a raw PDF: page geometry, declared font names, line boxes. See src/unrender/extract.py in the code repo for the exact format.