Hanno-Labs/dinghy-law-4b-v1

dinghy-law-4b

A 4B-parameter text-embedding model for legal retrieval, fine-tuned from Qwen/Qwen3-Embedding-4B for statute, case-law, and contract text. It is intended for retrieval, reranking, and semantic search over legal corpora, and is the larger sibling of dinghy-law-0.6b.

On MTEB(Law, v1) it scores 71.24 by Mean(Task) nDCG@10. In our reproduction of the benchmark it is the highest-scoring model we measured, edging a 7.6B legal fine-tune and every commercial legal embedding API on the benchmark, at roughly half the parameter count of the next-best model. These are our own measurements against published scores, not an official leaderboard placement (see Evaluation protocol), and Mean(Task) is one view of the benchmark — the MTEB leaderboard also ranks by a rank-aggregation (Borda) that requires the full field of models.

Results

MTEB(Law, v1)

Mean nDCG@10 (Mean(Task)) over the eight MTEB(Law) tasks. This model is measured with a leaderboard-faithful harness (see Evaluation protocol); scores for the other models are their published MTEB(Law) results. The ordering below reflects those numbers, not an official leaderboard submission.

Rank	Model	Params	Mean(Task)
1	dinghy-law-4b (this model)	4.03B	71.24
2	Mira190/Euler-Legal-Embedding-V1	7.57B	70.37
3	voyageai/voyage-law-2 (commercial)	—	65.39
4	codefuse-ai/F2LLM-v2-14B	13.99B	64.66
5	voyageai/voyage-3 (commercial)	—	64.13
6	infly/inf-retriever-v1	7.07B	63.68
7	codefuse-ai/F2LLM-v2-8B	7.57B	63.54

Per-task nDCG@10 in this evaluation:

Task	Score
LegalQuAD	73.96
GerDaLIRSmall	46.91
AILAStatutes	78.56
AILACasedocs	46.19
LeCaRDv2	76.20
LegalBenchConsumerContractsQA	85.56
LegalBenchCorporateLobbying	95.39
LegalSummarization	67.11

• Against the same field, it wins six of the eight tasks head-to-head versus the next-ranked model, trading German-statute (AILAStatutes) and Chinese case-precedent (LeCaRDv2) for wins on German case retrieval (LegalQuAD, GerDaLIRSmall), the LegalBench contract/lobbying tasks, and contract summarization.
• The backbone is unchanged from Qwen3-Embedding-4B. The untuned base already scores near the top of the benchmark; the adaptation described below adds legal specificity — most visibly on case-to-case precedent retrieval — without disturbing the base's general strength.

Intended use and positioning

Top-of-benchmark legal retrieval has generally meant a 7–14B model or a commercial API. dinghy-law-4b reaches the top of the legal benchmark in our measurements at 4B parameters — smaller than the models it surpasses — which lowers the hardware cost of competitive legal-domain retrieval for self-hosted, on-premise, and air-gapped deployments. Where 4B is still too large, the 0.6b sibling runs on a commodity CPU.

Typical uses: first-stage retrieval and reranking for legal RAG, statute and precedent search, clause and contract matching, and clustering or deduplication of legal text. It produces a single dense vector per input; it is a retrieval model, not a generative model, and does not produce legal advice.

Method

The model is a contrastive fine-tune of Qwen3-Embedding-4B, specialized for legal retrieval and then merged back toward the base to preserve general capability.

1. A strong instruction-following base. Qwen3-Embedding-4B is an LLM-derived embedding model that already accepts a task instruction on the query side [Qwen3-Embedding, 2506.05176]. Untuned it already ranks near the top of the legal benchmark, so the adaptation only has to add legal specificity, not general retrieval competence.

2. Contrastive domain fine-tuning with large-batch in-batch negatives. The model is fine-tuned with an InfoNCE objective [1807.03748] and in-batch negatives [DPR, 2004.04906] on query→passage pairs from public legal corpora. A large effective batch — the source of the negatives — is held in memory with gradient caching [GradCache, 2101.06983], which decouples the negative count from GPU memory and sharpens the contrastive signal at 4B scale.

3. Instruction conditioning. Queries carry a task-specific instruction, matching the base model's native interface and the instruction-tuned-embedding line of work [INSTRUCTOR, 2212.09741]. The document side is encoded without an instruction.

4. Case-to-case precedent pairs. Precedent retrieval (finding relevant cases for a case, as in LeCaRDv2 and GerDaLIR) has a different shape from question→passage retrieval, and question-answering-shaped pairs actively hurt it. Positives for these tasks are built as case→most-similar-same-category case pairs, using nearest-neighbor pairing within a shared legal category, which matches the precedent-retrieval objective and lifts case-to-case tasks without cost elsewhere.

5. Multilingual legal coverage. Beyond the English and Indian-statute mix of the 0.6b lineage, the training data adds German case law and legal reference text (Schleswig-Holstein court decisions; German Wikipedia legal articles) and Chinese criminal-case text, broadening the encoder's legal register across its three languages.

6. Uniform soup and weight-space merge-back. Eight epoch checkpoints are averaged into a single model [Model Soups, 2203.05482], then interpolated back toward the base, θ = (1 − α)·θ_base + α·θ_finetuned, at α = 0.6 [WiSE-FT, 2109.01903]. The soup averages diverse epochs, and the merge recovers general retrieval capability in weight space — in a single step, with no retraining — while keeping the legal gains.

The specialization uses standard contrastive methods; the case-to-case pairing and the soup plus merge-back are what let the model add precedent-retrieval and multilingual legal strength while keeping the base's general retrieval competence.

Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("Hanno-Labs/dinghy-law-4b-v1")

query = "What is the punishment for criminal breach of trust by a public servant?"
docs = [
    "Whoever, being in any manner entrusted with property ... commits criminal breach of trust ...",
    "A contract of sale is a contract whereby the seller transfers ... the property in goods ...",
]

# The query-side instruction is applied to queries and omitted from documents.
q_emb = model.encode_query(query)
d_emb = model.encode_document(docs)

scores = model.similarity(q_emb, d_emb)  # cosine
print(scores)

The query-side instruction is stored in the model config (prompts), so encode_query applies it and encode_document omits it. For MTEB(Law) and other retrieval tasks, a task-specific instruction can be passed on the query side (documents are always embedded raw); the leaderboard-faithful per-task instructions are documented in our MTEB submission. The model emits 2560-dimensional embeddings, uses cosine similarity, and encodes sequences up to 8192 tokens by default (extensible to the base model's 32k context via model.max_seq_length).

Loading and serving. These results, and the exact embeddings, come from loading the model through sentence-transformers, which applies the model's final projection head. Runtimes that pool and normalize without a post-pooling projection — vLLM, Text Embeddings Inference, and GGUF/llama.cpp — omit that head; their embeddings agree with the reference to about 0.997 cosine (a small retrieval difference on MTEB(Law)). A GGUF/llama.cpp build is published at dinghy-law-4b-v1-gguf.

Training data

All training sources are permissively licensed or public-domain legal text; each retains its original license. The training data is released as three datasets — legal-retrieval-pairs-v2 (MIT: German Schleswig-Holstein court decisions and Chinese CAIL cases), legal-retrieval-pairs-dewiki (German Wikipedia legal articles, CC-BY-SA-4.0, used with attribution and kept as a separate share-alike dataset), and the 0.6b-lineage legal-retrieval-pairs-v1 — whose cards carry per-source license and attribution tables. Public-domain and CC-BY-4.0 material is used with attribution; no CC-BY-NC (non-commercial) sources are used. Compilation, curation, and pair construction are © 2026 Clause Logic Inc.

Evaluation protocol

This model's MTEB(Law) number is produced with a leaderboard-faithful harness: query-side instruction only, raw documents, EOS token appended, max sequence length 8192, bf16, cosine similarity — matching the recipe the MTEB leaderboard uses for instruction embedders, and run through the same mteb library and InstructSentenceTransformerModel loader a leaderboard reviewer uses. The exported model reproduces the training-time evaluation geometry to 0.9999 cosine, and the number above is the mteb library's own Mean(Task). Scores quoted for other models are their published MTEB(Law) results; comparing our harness measurement against those published numbers is a reproduction, not an official leaderboard submission.

Limitations

• Mean(Task) is one summary of the benchmark; the MTEB leaderboard's default sort is a rank aggregation (Borda) over the full field of models, which this card does not compute.
• The two tasks where the model trails the field are German-statute identification (AILAStatutes) and Chinese case-precedent retrieval (LeCaRDv2), where the top models hold a lead that in-domain data did not close.
• General-domain (non-legal) retrieval carries a small cost relative to the base; the merge weight (α = 0.6) keeps it bounded.
• The model is multilingual only to the extent of its training languages (English, German, Chinese legal text); other languages and jurisdictions are out of distribution.
• It is a retrieval/embedding model. It does not generate text and does not constitute legal advice.

License and attribution

Released under Apache-2.0. Copyright © 2026 Clause Logic Inc. The base model, Qwen3-Embedding-4B, is licensed by its authors under Apache-2.0; this fine-tune does not relicense it.

Citation

@misc{dinghy-law-4b,
  title  = {dinghy-law-4b: a 4B legal text-embedding model},
  author = {Solka, Stephen},
  year   = {2026},
  note   = {Hanno Labs / Clause Logic Inc.},
  howpublished = {\url{https://huggingface.co/Hanno-Labs/dinghy-law-4b-v1}}
}

References

• Qwen3 Embedding — arXiv:2506.05176
• WiSE-FT: Robust fine-tuning of zero-shot models — arXiv:2109.01903
• Model soups — arXiv:2203.05482
• Representation Learning with Contrastive Predictive Coding (InfoNCE) — arXiv:1807.03748
• Dense Passage Retrieval (DPR) — arXiv:2004.04906
• Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup (GradCache) — arXiv:2101.06983
• INSTRUCTOR: instruction-finetuned embeddings — arXiv:2212.09741
• MTEB — arXiv:2210.07316