Instructions to use f4roukb/denserepr-code-48M-pretrain-EN-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use f4roukb/denserepr-code-48M-pretrain-EN-v1 with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("f4roukb/denserepr-code-48M-pretrain-EN-v1") sentences = [ "That is a happy person", "That is a happy dog", "That is a very happy person", "Today is a sunny day" ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [4, 4] - Notebooks
- Google Colab
- Kaggle
denserepr-code-48M-pretrain-EN-v1
A small dense embedding model for code retrieval: 48M parameters, one 384-d vector per text, cosine similarity, 8,192-token context. It scores 64.84 mean nDCG@10 on CoIR (10 tasks), +11.4 over the model it was trained from.
It was made by continued contrastive pretraining of ibm-granite/granite-embedding-small-english-r2 on ~23M code retrieval pairs with in-batch negatives only. This is the pretraining stage: no hard negatives, no distillation and no fine-tuning stage have been applied yet.
Usage
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("f4roukb/denserepr-code-48M-pretrain-EN-v1")
queries = ["read a csv file into a list of dicts"]
docs = [
"def load(path):\n import csv\n with open(path) as f:\n return list(csv.DictReader(f))",
"def save(rows, path):\n import json\n with open(path, 'w') as f:\n json.dump(rows, f)",
]
q = model.encode(queries)
d = model.encode(docs)
print(model.similarity(q, d)) # the first document scores higher
No prompts or instructions: queries and documents are encoded the same way.
| parameters | 47,662,464 |
| embedding | 384-d, CLS pooling, cosine similarity |
| max sequence length | 8,192 tokens |
| architecture | ModernBERT (from granite-embedding-small-english-r2) |
| weights | fp32 safetensors |
| precision | mixed bf16 / fp32: trained with fp32 weights and bf16 autocast; CoIR scores measured in bf16 |
Results: CoIR (MTEB)
nDCG@10 Γ100, measured at 8,192 tokens with mteb 2.27.8 task revisions. The base model is measured here the same
way.
| task | base | this model |
|---|---|---|
| AppsRetrieval | 13.48 | 30.75 |
| CodeFeedbackMT | 52.20 | 81.19 |
| CodeFeedbackST | 76.84 | 81.26 |
| CodeSearchNetCCRetrieval | 48.42 | 75.97 |
| CodeTransOceanContest | 77.63 | 80.00 |
| CodeTransOceanDL | 33.63 | 34.84 |
| CosQA | 35.40 | 33.84 |
| COIRCodeSearchNetRetrieval | 60.42 | 79.90 |
| StackOverflowQA | 90.04 | 90.97 |
| SyntheticText2SQL | 46.44 | 59.69 |
| mean | 53.45 | 64.84 |
Among the dense (single-vector) models on the MTEB CoIR leaderboard with all 10 tasks, it is the best one under 80M parameters (as of 2026-11-12).
Read these scores with the following in mind:
- Not zero-shot on CoIR. The training data includes the CoIR train splits. The training set was decontaminated against the CoIR test sets (exact match after normalization, in both roles).
Data
| source | content | pairs before cleaning |
|---|---|---|
| lightonai/cornstack | docstring β code: Python, Go, Java, PHP, JavaScript, Ruby | 21.5M |
| codefuse-ai/F2LLM-v2 | OCGI (instruction β Python) and xCodeEval (problem β solution) | 1.1M |
| CoIR-Retrieval train splits | the 10 CoIR tasks | 2.2M |
Cleaning: exact deduplication of normalized pairs, removal of stub code (pass, empty bodies, return true) with
tree-sitter in 14 languages, and decontamination against the CoIR test sets and NanoCoIR. 24.8M pairs β 23.1M.
Limitations
- English queries; code mostly in the six cornstack languages plus SQL.
- Weakest relative to peers on short natural-language queries (CosQA is below the base model).
- A pretraining-stage model: a fine-tuning stage with hard negatives would likely change the scores.
License and attribution
Apache-2.0. This model is a modified version of ibm-granite/granite-embedding-small-english-r2 (Apache-2.0, Β© IBM): its weights were changed by the training described above. It is not an IBM model.
Training data and licenses: cornstack (Apache-2.0), F2LLM-v2 (Apache-2.0; its OCGI and xCodeEval parts are CC-BY-4.0), and the CoIR train splits, whose upstream sources are APPS (MIT), CodeSearchNet, Code-Feedback (Apache-2.0), gretelai synthetic text-to-SQL (Apache-2.0), CodeTransOcean, CoSQA and StackOverflow (CC-BY-SA). Thanks to their authors.
Citation
@misc{denserepr2026,
title = {denserepr-code-48M-pretrain-EN-v1: a small dense embedding model for code retrieval},
author = {Farouk B.},
year = {2026},
url = {https://huggingface.co/f4roukb/denserepr-code-48M-pretrain-EN-v1}
}
- Downloads last month
- 59