gwangjin31/cl-docling-smoldocling-m614-lora

SmolDocling M6.14 Common Lisp LoRA

This is a small research adapter for docling-project/SmolDocling-256M-preview, pinned to revision ce51f56c4ebe36e0b1c3a55f67b261ba22a50bf8.

The adapter was trained natively in Common Lisp with cl-docling and cl-transformer-blocks on Apple MLX. Python was an independent processor, PEFT-load, and generation oracle—not a training fallback.

What this artifact demonstrates

• Hugging Face weights loaded in Common Lisp.
• Frozen-vision decoder LoRA training ran in Common Lisp/MLX.
• Common Lisp exported an ordinary 120-tensor PEFT adapter.
• Transformers 5.16.1 and PEFT 0.20.0 loaded every tensor exactly.
• Fresh Python reproduced all 23 retained native outputs: 11,458 exact token IDs, raw decoded text, and stop reasons.

What it does not demonstrate

This is not a generally improved PDF converter. On nine protected final pages, exact page acceptance remained 0/9. The adapter passed the frozen promotion gate by only two character edits and one word edit:

Protected final metric	Base	Adapter
Character edits / 16,465 reference characters	5,252	5,250
Word edits / 2,728 reference words	869	868
Exact pages	0/9	0/9
Exact structure	2/9	2/9
Strict exports	7/9	7/9

Five frozen real-PDF regression pages and nine synthetic coverage pages were unchanged. Two dense final pages reached the same 2,048-token limit before and after training. Do not present this artifact as production OCR, broad document generalization, or evidence that fine-tuning usually improves SmolDocling.

Its useful result is the reproducible engineering round trip: Hugging Face → Common Lisp → native training → standard PEFT → ordinary Python.

Training

• Base: docling-project/SmolDocling-256M-preview at the immutable revision above.
• Data: eight source-backed pages from four public-domain Internet Archive books.
• Validation: seven pages from three different books.
• Protected final: nine pages from three further books, opened only after selection.
• Discovery: allenai/olmOCR-mix-0225 metadata selected source URLs/pages; its response/weak-label field was never read.
• Targets: manually reviewed DocTags derived from source renders.
• Trainable parameters: decoder self-attention q_proj and v_proj LoRA only.
• LoRA: rank 2, alpha 4, zero dropout; vision, connector, and base decoder frozen.
• Schedule: two fixed passes over eight pages, 16 AdamW updates, FP32, Metal.
• Selection: base/step 8/step 16 ranked on validation only; step 16 selected.

The source repository retains the frozen protocols, aggregate evidence, failures, and exact artifact hashes. It does not publish the reviewed page transcriptions.

Python usage

Install compatible releases of torch, transformers, peft, safetensors, and Pillow. The qualification used Torch 2.14.0, Transformers 5.16.1, PEFT 0.20.0, Safetensors 0.8.0, and Pillow 12.3.0.

import json
import torch
from peft import PeftModel
from transformers import AutoProcessor, Idefics3ForConditionalGeneration

base_id = "docling-project/SmolDocling-256M-preview"
revision = "ce51f56c4ebe36e0b1c3a55f67b261ba22a50bf8"

processor = AutoProcessor.from_pretrained(base_id, revision=revision)
base = Idefics3ForConditionalGeneration.from_pretrained(
    base_id,
    revision=revision,
    dtype=torch.float32,
    attn_implementation="eager",
)

# Run this from a cloned/downloaded adapter directory, or replace "." with its path.
model = PeftModel.from_pretrained(base, ".").eval()

Prepare an RGB page image with the base processor and use the task text Convert this page to docling.. The adapter emits DocTags, not final Markdown; strict parsing/export remains the caller's responsibility.

Common Lisp

The artifact originates from cl-docling, but its public PEFT config correctly names the Hub base rather than the author's private local path. Native loading uses a separately verified local-base binding step; see the release instructions in the source repository. No Python inference is hidden inside the Common Lisp API.

License and attribution

The adapter follows the base model's cdla-permissive-2.0 model-card license. Review the pinned base model card and license before redistribution or deployment. The training pages were selected from sources carrying the project's recorded public-domain eligibility metadata; this is not a universal rights warranty.

Reproducibility boundary

The publication package includes cl_docling_release.json with the immutable base revision, factor hash, evidence hashes, aggregate measurements, and package file identities. A Hub upload and fresh remote download are separate release gates; a local package alone is not evidence that either occurred.

No Hub upload occurred during M7.1.