yadavkapil7155/Qwen2.5-Coder-3B_QLora

Qwen2.5-Coder-3B: QLoRA Fine-Tuning for Code Generation

A parameter-efficient fine-tuning project that adapts Qwen2.5-Coder-3B-Instruct using QLoRA and coding instruction datasets. The project explores how additional supervised fine-tuning affects code generation, instruction following, and performance on simple programming tasks.

Overview

Large language models can generate code, explain algorithms, and assist with software development. However, fine-tuning can adapt their behavior to specific datasets and task requirements.

In this project, I fine-tuned an existing coding model using 4-bit QLoRA, trained it on coding instruction-response examples, saved the learned LoRA adapter, and tested inference and preliminary coding performance against the original base model.

Important: This project fine-tunes an existing pretrained model; it does not train a language model from scratch.

Project Highlights

• Fine-tuned Qwen2.5-Coder-3B-Instruct using QLoRA.
• Used 4-bit NF4 quantization to reduce GPU memory requirements.
• Prepared a combined coding instruction dataset.
• Trained on 45,000 examples with 2 epochs.
• Used LoRA adapters instead of updating all base-model parameters.
• Saved the final adapter and training checkpoints.
• Ran inference using the base model with the trained adapter.
• Compared the original and fine-tuned models on custom coding tests.

Model Architecture and Training

Parameter	Configuration
Base model	Qwen/Qwen2.5-Coder-3B-Instruct
Fine-tuning method	QLoRA
Quantization	4-bit NF4
Compute dtype	BF16 during training
LoRA rank	16
LoRA alpha	32
LoRA dropout	0.05
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Maximum sequence length	2,048 tokens
Training examples	45,000
Validation examples	2,994
Epochs	2
Learning rate	2e-4
Optimizer	Paged AdamW 8-bit
Gradient accumulation	16
Training hardware	AWS EC2 G5.xlarge with NVIDIA A10G GPU

QLoRA loads the base model in quantized form and trains additional low-rank adapter parameters. The original base weights remain frozen during training.

Dataset

The training dataset was constructed from two sources:

1. CodeAlpaca 20K — programming instructions and corresponding responses.
2. Magicoder-OSS-Instruct-75K — coding instruction-response examples.

Dataset Split

Split	Examples
Training	45,000
Validation	2,994
Total prepared	47,994

The combined dataset includes programming instructions and examples covering multiple programming languages. Dataset metadata and language labels were retained where available.

Training Results

The production training run completed successfully after two epochs.

Metric	Result
Training loss	0.4730
Validation loss	0.6010
Mean token accuracy	82.81%
Training duration	Approximately 14.85 hours
Final optimizer steps	5,626
Adapter size	Approximately 60 MB

These metrics describe the training run and its validation behavior. Mean token accuracy is not equivalent to the percentage of coding problems solved correctly.

Preliminary Evaluation

The original model and fine-tuned model were tested using the same prompts and predefined expected outputs on small, custom Python coding test suites.

Evaluation Results

Evaluation Set	Original Qwen	Fine-Tuned Qwen
Initial 5-problem pilot	12/15 (80.0%)	15/15 (100.0%)
Additional 20-problem pilot	52/60 (86.7%)	54/60 (90.0%)

The second evaluation showed a difference of 3.3 percentage points in favor of the fine-tuned model on this particular test suite.

These results are preliminary, not a definitive benchmark. The test suites are small and custom-built, and some problems overlap with earlier experiments. They do not establish general coding superiority or prove that the observed difference will generalize to unseen tasks.

A more rigorous evaluation should use a larger set of independent programming problems, validated test cases, and an isolated execution environment.

Inference

The trained adapter can be loaded alongside the original base model to generate code.

The inference process is:

1. Load the base Qwen model.
2. Load the tokenizer.
3. Attach the trained LoRA adapter.
4. Format a prompt using the model's chat template.
5. Generate a response.
6. Decode and inspect the generated output.

Example Prompt

Write a Python function to implement binary search
on a sorted list. Return the target index or -1
if the target is not present.

The adapter contains the learned parameter updates; it is not a standalone replacement for the base model.

Project Structure

qwen-coder/
├── data/
│   ├── final/
│   ├── train/
│   └── validation/
├── scripts/
│   └── train_qlora.py
├── checkpoints/
│   └── qwen-coder-qlora/
├── outputs/
│   └── qwen-coder-qlora/
│       ├── adapter_config.json
│       ├── adapter_model.safetensors
│       ├── eval_metrics.json
│       └── training_metrics.json
└── README.md

The exact contents may vary depending on which training checkpoints and generated artifacts are included in the repository.

Technologies Used

• Python 3.11
• PyTorch
• Hugging Face Transformers
• Hugging Face Datasets
• PEFT
• TRL
• bitsandbytes
• Google Colab
• AWS EC2
• Git and GitHub

Reproducing Inference

Install the required dependencies in a compatible Python environment:

pip install torch transformers peft accelerate bitsandbytes

Load the base model and attach the adapter:

import torch
from transformers import (
    AutoTokenizer,
    AutoModelForCausalLM,
    BitsAndBytesConfig,
)
from peft import PeftModel

BASE_MODEL = "Qwen/Qwen2.5-Coder-3B-Instruct"
ADAPTER_PATH = "./outputs/qwen-coder-qlora"

tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)

quant_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True,
)

base_model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    quantization_config=quant_config,
    device_map="auto",
)

model = PeftModel.from_pretrained(
    base_model,
    ADAPTER_PATH,
)

model.eval()

messages = [
    {
        "role": "user",
        "content": (
            "Write a Python function to check whether "
            "a string is a palindrome."
        ),
    }
]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)

with torch.inference_mode():
    outputs = model.generate(
        inputs,
        max_new_tokens=256,
        do_sample=False,
        pad_token_id=tokenizer.eos_token_id,
    )

generated_tokens = outputs[0][inputs.shape[1]:]

print(
    tokenizer.decode(
        generated_tokens,
        skip_special_tokens=True,
    )
)

The code assumes the adapter files are available locally and that the installed libraries and hardware support the selected quantization configuration. BF16 was used for the original training run; FP16 is used here for broader inference compatibility.

Limitations

• The project adapts an existing coding model rather than creating a model from scratch.
• The adapter depends on the compatible base model.
• Preliminary evaluation results are based on small, custom test suites.
• Generated code may contain logical errors or security vulnerabilities and should be tested before use.
• The current project does not claim a production-ready API, Docker deployment, or independently validated benchmark.

Future Improvements

• Expand evaluation to a larger collection of unseen coding problems.
• Test generated code against comprehensive test cases in an isolated sandbox.
• Evaluate on established coding benchmarks where licensing and hardware permit.
• Investigate failures and improve dataset quality.
• Compare additional LoRA configurations and training hyperparameters.
• Build a FastAPI inference service.
• Add Docker packaging and deployment.
• Publish reproducible evaluation results and example outputs.

Conclusion

This project demonstrates an end-to-end QLoRA fine-tuning workflow for an existing code-generation model: dataset preparation, quantized training, adapter saving, inference, and preliminary evaluation.

The initial experiments show promising results on selected custom test cases, while further independent evaluation is needed to determine the generality and significance of any performance improvement.

Acknowledgements

• Qwen team for the Qwen2.5-Coder model.
• Hugging Face for Transformers, Datasets, PEFT, and TRL.
• The contributors to the CodeAlpaca and Magicoder datasets.

License and Dataset Attribution

Review the base model's license and the licenses and terms of both datasets before redistributing model artifacts or datasets. Preserve required attribution and comply with the applicable terms.