lora_bart_samsum_results

A LoRA adapter for facebook/bart-large-cnn, fine-tuned on the SAMSum dialogue-summarization dataset using QLoRA (4-bit NF4 base + LoRA adapters).

The base model summarizes news articles. This adapter shifts it toward conversational text — chat logs, meeting transcripts, message threads — where the summary needs to describe what people said and agreed rather than condense an article.

Trained as part of a document-assistant project: pop123-ux/doc-assistant-hf, where it powers the summarization tab of a Gradio app alongside a Longformer QA model.

Status: demonstration run. One epoch, no ROUGE evaluation. See Limitations before using this for anything that matters.

Usage

This is an adapter — you load the base model first, then apply it.

import torch
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM, BitsAndBytesConfig
from peft import PeftModel

base = "facebook/bart-large-cnn"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
)

tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForSeq2SeqLM.from_pretrained(base, quantization_config=bnb_config, device_map="auto")
model = PeftModel.from_pretrained(model, "pop123ux/lora_bart_samsum_results")
model.eval()

dialogue = """Hannah: Hey, do you have Betty's number?
Amanda: Lemme check
Amanda: Sorry, can't find it.
Amanda: Ask Larry, he called her last time we were at the park together.
Hannah: I'd rather you texted him.
Amanda: Just text him 🙂"""

inputs = tokenizer(dialogue, return_tensors="pt", truncation=True, max_length=1024).to(model.device)

with torch.no_grad():
    ids = model.generate(
        **inputs,
        max_new_tokens=150,
        min_new_tokens=50,
        num_beams=4,
        no_repeat_ngram_size=3,
        repetition_penalty=1.2,
        length_penalty=2.0,
        early_stopping=True,
    )

print(tokenizer.decode(ids[0], skip_special_tokens=True))

The 4-bit config is optional — the adapter loads onto the fp16/fp32 base model too. It was trained against a 4-bit base, so results are closest to the training run when you keep the quantization.

Training data

knkarthick/samsum — messenger-style dialogues with human-written abstractive summaries.

Split	Examples
train	14,731
validation	818
test	819 (unused)

Preprocessing: the dialogue field was tokenized to a max of 512 tokens and summary to 128 tokens, both truncated, with dynamic padding via DataCollatorForSeq2Seq and label padding set to -100 so pad tokens are excluded from the loss.

Training procedure

Trained in a single Colab session on one T4 GPU.

Quantization and adapter config

BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
)

LoraConfig(r=8, lora_alpha=32, lora_dropout=0.1, task_type=TaskType.SEQ_2_SEQ_LM)

target_modules was left unset, so PEFT applied its BART default (the attention q_proj and v_proj projections). Combined with prepare_model_for_kbit_training and gradient checkpointing, this trains:

1,179,648 of 407,470,080 parameters — 0.29%.

Hyperparameters

learning_rate	5e-05
train_batch_size	4
eval_batch_size	8
gradient_accumulation_steps	4
total_train_batch_size	16
optimizer	adamw_torch_fused, betas=(0.9, 0.999), eps=1e-08
lr_scheduler_type	linear
weight_decay	0.01
num_epochs	1 (921 steps)
mixed_precision	Native AMP (fp16)
seed	42

Results

Training Loss	Epoch	Step	Validation Loss
4.1413	1.0	921	1.5732

Epoch-average training loss was 6.3249; the run took 1,435 s (~24 min) at 10.3 samples/s, and evaluation took 15.3 s at 53.6 samples/s.

No ROUGE or other generation metrics were computed — validation loss is the only measurement here.

Limitations

• Under-trained, and the loss curve says so. One epoch, and training loss stayed noisy and high throughout (individual steps swinging between roughly 4 and 9, epoch average 6.32) while validation loss landed at 1.57. That gap is wide enough to be worth investigating rather than celebrating — a few early steps also logged nan grad norms, which points at fp16 instability. Expect modest gains over the base model at best.
• No generation-quality evaluation. Validation loss is not summary quality. Without ROUGE against the SAMSum test split, there's no evidence this beats stock bart-large-cnn on any task.
• Domain shift is the point, and the cost. SAMSum is short, informal, multi-speaker chat. On formal prose — reports, papers, articles — the unmodified base model is likely the better choice.
• 1,024-token input ceiling, inherited from BART's learned positional embeddings. Longer documents need chunking; the adapter doesn't change this.
• Trained on 512-token inputs, so behaviour on inputs between 512 and 1,024 tokens is extrapolation.
• Pad token quirk. During training the tokenizer's pad_token was set to the EOS token, which moved pad_token_id from BART's usual 1 to 2. This is harmless for single-example inference, but if you batch inputs, set the pad token explicitly and check your attention masks rather than assuming the default.
• English only, following the dataset.

Framework versions

• PEFT 0.19.1
• Transformers 5.13.1
• PyTorch 2.11.0+cu128
• Datasets 4.0.0
• Tokenizers 0.22.2

License

MIT, matching the base model's licence terms for facebook/bart-large-cnn.