Instructions to use enesyila/ota-sbert-kadisicilleri with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use enesyila/ota-sbert-kadisicilleri with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("enesyila/ota-sbert-kadisicilleri") sentences = [ "The weather is lovely today.", "It's so sunny outside!", "He drove to the stadium." ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] - Notebooks
- Google Colab
- Kaggle
OTA-SBERTurk-Court (ota-sbert-kadisicilleri)
OTA-SBERTurk-Court is a sentence-embedding model for Latin-transcribed Ottoman Turkish legal text. We started from BERTurk and adapted it to the language of the Istanbul court registers (kadı sicilleri, 1513–1911) in two steps: first unsupervised SimCSE, then training on pairs of case summaries and records with Multiple Negatives Ranking Loss.
The model turns a sentence or short passage into a 768-dimensional vector. Cosine similarity between vectors can be used for semantic search, clustering, and scoring texts against psychological constructs.
| Base model | dbmdz/bert-base-turkish-cased (111M parameters) |
| Output dimension | 768 |
| Max sequence length | 256 tokens (longer inputs are truncated) |
| Pooling | mean |
| Similarity | cosine |
| Language | Ottoman Turkish, Latin transcription |
Usage
from sentence_transformers import SentenceTransformer, util
model = SentenceTransformer("enesyila/ota-sbert-kadisicilleri")
query = "Zeyd, Amr'dan on bin akçe talep eyledi."
records = [
"Mahalleden birisi mahkemeye gelüb komşusundan borcu olan bir miktar parayı istedi.",
"Mezbûr Hind, vakf olunan menzilin tevliyetine nâib nasb olundu.",
]
q = model.encode(query, normalize_embeddings=True)
r = model.encode(records, normalize_embeddings=True)
print(q.shape) # (768,)
print(util.cos_sim(q, r)) # higher = more similar -- the output for the example above is tensor([[0.2620, 0.0773]]).
You don't need an instruction prefix. Inputs longer than 256 tokens are cut off, so for long records it is better to split them into sentences or passages and pool the scores.
Training
We trained the model on the Istanbul Qadi Registers corpus (İSAM; 54,670 records from 100 volumes and 21 courts). Every record comes with a short, one-sentence case summary.
Data
- Cleaning. We dropped records with no text or no summary, texts shorter than 20 characters or with less than 30% alphabetic characters, texts that were only short markup tags, and summaries of 5 characters or fewer. Exact duplicate (summary, text) pairs were also removed. This left 54,535 records.
- An additional test set for expert annotation. The 440 records annotated by the three experts, plus any exact duplicates of them, were kept out of training, validation, and checkpoint selection. The evaluation done through this test set was not covered in this model card.
- Split by summary. The remaining 54,095 records were split 80/10/10 by case summary (seed 42), so a summary never appears in more than one split. This gives 43,108 training, 5,715 validation, and 5,272 test records.
A note on transcription: the texts do not follow the IJMES or İslâm Ansiklopedisi transcription rules. For example, kaf and kef are not distinguished and both are written as k.
Stage 1: SimCSE (unsupervised)
Annotated data is scarce in Ottoman Turkish NLP, so the first stage uses no labels at all. Starting from dbmdz/bert-base-turkish-cased, each training record is paired with itself. Dropout (0.1) makes the two encodings slightly different, and the other records in the batch serve as negatives (SimCSE; Gao et al., 2021). This teaches the encoder the vocabulary and spelling of the registers.
| Setting | Value |
|---|---|
| Data | 43,108 training records (record text only) |
| Loss | CachedMultipleNegativesRankingLoss (scale 20, cosine similarity) |
| Batch size | 128 (gradient caching, mini-batches of 32) |
| Epochs / learning rate | 1 / 1e-5, linear schedule, warm-up ratio 0.1 |
| Max sequence length | 256 tokens |
| Precision | bf16 mixed precision |
| Checkpoint selection | lowest validation loss (checked every 100 steps) |
Stage 2: summary and record pairs (supervised)
In the second stage the Stage 1 model learns to place a case summary close to its own record. The summary is the anchor, its record is the positive, and the other records in the batch are negatives. In this way the model learns to match short modern-Turkish descriptions with long Ottoman texts.
| Setting | Value |
|---|---|
| Data | 43,108 (summary, record) pairs |
| Loss | CachedMultipleNegativesRankingLoss (scale 20, cosine similarity) |
| Batch size | 64 in-batch negatives (mini-batches of 16 only save memory) |
| Epochs / learning rate | 2 / 2e-5, linear schedule, warm-up ratio 0.1, weight decay 0.01 |
| Max sequence length | 256 tokens (the first 256 tokens of each record) |
| Precision | fp32 weights with bf16 mixed precision |
| Checkpoint selection | best validation NDCG@10 (summary-to-record retrieval on 10,000 validation records, checked every 150 steps) |
| Batching | no_duplicates sampler, so a batch never contains the same text twice |
| Seed | 42 |
Neither stage uses an instruction prefix. The released model is the Stage 2 checkpoint. Training used Sentence Transformers 6.2.0, Transformers 5.20.1, and PyTorch 2.11.
Evaluation
We evaluated the model on four tasks that each check something different: whether it can find the right record from a summary (retrieval), whether matched summaries and records sit close together in the embedding space, whether it can tell related from unrelated Ottoman sentences (STS), and whether it keeps psychological constructs apart (QIC). As an external baseline we also report SerBERT, a publicly available Ottoman Turkish BERT-based MLM encoder, on the STS and QIC tasks.
1. Document retrieval (summary → record)
Each case summary is a query, and the model ranks all 5,272 test records by cosine similarity. A record counts as relevant if it belongs to the query summary. When several records share a summary, all of them are relevant. Every model gets the same queries, candidates, and relevance judgments. Recall@k is the share of a summary's records found in the top k, MRR@10 reflects the rank of the first relevant record, and NDCG@10 scores the ranking of all relevant records in the top ten.
| Model | NDCG@10 | MRR@10 | R@1 | R@5 | R@10 |
|---|---|---|---|---|---|
| BERTurk (base) | 0.011 | 0.009 | 0.006 | 0.013 | 0.019 |
| + SimCSE (stage 1) | 0.024 | 0.018 | 0.011 | 0.029 | 0.041 |
| OTA-SBERTurk-Court (stage 2) | 0.898 | 0.883 | 0.843 | 0.925 | 0.946 |
The base model is close to zero because it was never trained to connect short modern-Turkish summaries with Ottoman records. The big jump comes from the summary–record training in stage 2, not from SimCSE alone.
2. Embedding space
On the test split, we checked whether matched texts sit closer together than unmatched ones, and report ROC-AUC for two checks:
- Summary → text: is a summary more similar to its own record than to a record belonging to another summary?
- Cross-document: are two records that share a summary more similar to each other than to a record with a different summary? Records with the same summary are not identical. They are on the same topic, but they may involve different people, places, or decisions.
| Check | AUC |
|---|---|
| Summary → text | 0.997 |
| Cross-document | 0.985 |
A high score on summary–text pairs is expected, because this is exactly what the model was trained on. The high cross-document score is more interesting, since the model never saw those pairs during training.
3. Semantic textual similarity (STS)
We wrote 100 within-topic sentence pairs on legal topics such as debt, land dispute, and embezzlement. Together they give 200 sentences and 19,900 sentence pairs: the 100 matched pairs are positives and the 19,800 cross-topic pairs are negatives. None of these sentences were in the training data. We report retrieval metrics (each sentence is a query for its partner; MRR with 95% bootstrap CIs over topics) and ROC-AUC. The cross-register AUC compares the 100 positives with the negatives. We also report PR-AUC, because 100 positives against 19,800 negatives can give a high ROC-AUC even when precision is low.
| Metric | OTA-SBERTurk-Court | BERTurk (base) | SerBERT |
|---|---|---|---|
| MRR (95% CI) | 0.806 [0.740, 0.866] | 0.331 [0.276, 0.389] | 0.566 [0.502, 0.632] |
| Recall@1 | 0.745 | 0.235 | 0.465 |
| Cross-register ROC-AUC (95% CI) | 0.972 [0.956, 0.985] | 0.683 [0.657, 0.713] | 0.870 [0.841, 0.900] |
| PR-AUC | 0.521 | 0.009 | 0.075 |
4. Questionnaire-item classification (QIC)
This task checks whether the embeddings keep the psychological constructs apart. There are 47 questionnaire items (honor, individualism, collectivism). Each item and its five aligned proxy sentences count as separate samples, 282 in total (150 honor, 66 individualism, 66 collectivism). We train an RBF-kernel SVM (C = 1, balanced class weights, no tuning) on the embeddings and use 10-fold cross-validation grouped by item, so an item and its proxy sentences never end up in both the training and test folds. Values are the mean ± SD across folds.
| Model | Accuracy | Macro-F1 |
|---|---|---|
| OTA-SBERTurk-Court | 89.58% ± 5.57 | 0.838 |
| SerBERT | 85.25% ± 8.35 | 0.781 |
| BERTurk (base) | 85.08% ± 7.74 | 0.777 |
Limitations
- The model was trained and evaluated only on Latin-transcribed records from Istanbul courts, 1513–1911.
- Inputs are cut at 256 tokens. For longer documents, split them into sentences first.
- Larger general-purpose embedding models, such as Qwen3-Embedding 8B, score higher on the STS and questionnaire tasks. OTA-SBERTurk-Court is much smaller and faster than they are, and its gain is over the base model and on retrieval of Ottoman court records.
- Court records are formulaic and written by an institution. Similarity can therefore reflect the legal genre (for example manumission or endowment deeds) instead of the concept you are looking for.
License
CC-BY-NC-4.0 (non-commercial use). The base model BERTurk is released under the MIT license. The training corpus belongs to İSAM / İstanbul Medipol University and is not redistributed.
Contact
Enes Yılandiloğlu: enes.yilandiloglu@helsinki.fi
- Downloads last month
- 70
Model tree for enesyila/ota-sbert-kadisicilleri
Base model
dbmdz/bert-base-turkish-cased