{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Stoicheia — a character-level model for Ancient Greek\n", "\n", "Stoicheia is a 405M-parameter character-level masked-diffusion encoder for Ancient Greek.\n", "Its input is factored into five aligned planes — letters, word/sentence boundaries,\n", "diacritics, capitalization, punctuation — and **any of them can be set to *unknown* at\n", "inference**. One model therefore reads an edited text, bare *scriptio continua*, and a\n", "lacuna of unknown length, changing nothing but its input.\n", "\n", "This notebook runs the whole release end to end on a free Colab GPU (CPU works too, slower):\n", "\n", "1. restore a lacuna whose width is *not known* in advance\n", "2. pick the restoration model that has provably **never read** your document\n", "3. tag and parse a verse of Homer\n", "4. macronize and scan a line of verse\n", "5. score the macronizer against a hand-annotated benchmark\n", "\n", "Every model and dataset used below is public.\n" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "%pip install -q --upgrade transformers huggingface_hub safetensors torch datasets\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 1. Restoring a lacuna of unknown width\n", "\n", "The hard case in epigraphy and papyrology is a break whose extent is uncertain, in text that\n", "carries no accents and no word division. Write `[N±M]` and the model scores every width in\n", "`N-M … N+M` by its own confidence, restoring the letters, the accents and the word boundaries\n", "together.\n" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "import sys, torch\n", "from transformers import AutoModel\n", "from huggingface_hub import hf_hub_download, snapshot_download\n", "\n", "REPO = \"Ericu950/Stoicheia-doc_clean\" # zero exposure to inscriptions or papyri\n", "model = AutoModel.from_pretrained(REPO, trust_remote_code=True).eval()\n", "\n", "# the processor is a plain helper module, not part of the model classes: fetch it first\n", "hf_hub_download(repo_id=REPO, filename=\"processing_char_bert.py\", local_dir=\".\")\n", "from processing_char_bert import CharBertProcessor\n", "proc = CharBertProcessor()\n", "\n", "# John 1:1 as it would reach us on a damaged, unaccented, unspaced witness\n", "damaged = \"εναρχηηνο[5±3]καιολογοςηνπροστονθεον\"\n", "best, width, candidates = proc.restore_elastic(model, damaged, mask_dia_boundary=True)\n", "print(\"restored :\", best)\n", "print(\"width :\", width, \"characters\")\n", "for c in candidates[:5]:\n", " print(\" \", c)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 2. The model that has never read your document\n", "\n", "A single fixed train/test split makes a model useless for exactly the documents an editor\n", "cares about. Ten restoration checkpoints are released instead, one per held-out final digit\n", "of the PHI/TM identifier: whatever inscription or papyrus you are working on, one of the ten\n", "has provably never seen it during fine-tuning, and its backbone never saw a documentary text\n", "at all. A reading proposed by *that* model cannot be a memory of the edition you are checking.\n" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "def model_that_never_read(document_id: str) -> str:\n", " \"\"\"Pick the released checkpoint whose held-out digit matches this document.\"\"\"\n", " digit = str(document_id).strip()[-1]\n", " return f\"Ericu950/Stoicheia-restoration-test{digit}\"\n", "\n", "for phi in [\"PHI 12345\", \"PHI 293\", \"TM 8100\"]:\n", " print(f\"{phi:12s} -> {model_that_never_read(phi)}\")\n", "\n", "# use it exactly like the backbone above\n", "REPO = model_that_never_read(\"PHI 293\")\n", "local = snapshot_download(REPO, allow_patterns=[\"*.py\", \"*.json\"])\n", "sys.path.insert(0, local)\n", "restorer = AutoModel.from_pretrained(REPO, trust_remote_code=True).eval()\n", "\n", "# note the inputs: τηβου and στεφα...ετης carry no accents and no word division.\n", "# accents and spacing are predictions here, not requirements, so the model fills\n", "# the gap and decides where the words end in the same pass.\n", "print(\"\\n\", proc.restore_respaced(restorer, \"ἔδοξεν τηβου-- καὶ τῷ δήμῳ\"))\n", "# -> ἔδοξεν τῇ βουλῇ καὶ τῷ δήμῳ\n", "print(proc.restore_respaced(restorer, \"στεφανῶσαι αὐτὸν χρυσῷ στεφα[3±1]ετης ἕνεκα\"))\n", "# -> στεφανῶσαι αὐτὸν χρυσῷ στεφάνῳ ἀρετῆς ἕνεκα\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 3. Tagging and parsing\n", "\n", "Four heads on one shared backbone — factored XPOS, an edit-script lemmatizer, a UPOS\n", "auxiliary and a biaffine dependency parser — all from a single forward pass.\n" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "from huggingface_hub import snapshot_download\n", "REPO = \"Ericu950/Stoicheia-tagger-parser\"\n", "local = snapshot_download(REPO, allow_patterns=[\"*.json\", \"*.txt\", \"*.py\", \"*.model\"])\n", "sys.path.insert(0, local)\n", "from processing_char_bert_joint import CharBertJointProcessor\n", "\n", "parser_model = AutoModel.from_pretrained(REPO, trust_remote_code=True).eval()\n", "jproc = CharBertJointProcessor.from_pretrained(local)\n", "\n", "words = \"μῆνιν ἄειδε θεὰ Πηληϊάδεω Ἀχιλῆος\".split()\n", "batch = jproc([words])\n", "with torch.no_grad():\n", " out = parser_model(**batch)\n", "rows = jproc.decode(out, batch, ud=True)\n", "\n", "sent = rows[0] if rows and not isinstance(rows[0], dict) else rows\n", "hdr = ('id', 'form', 'lemma', 'upos', 'head', 'deprel')\n", "print('%3s %-12s%-12s%-8s%4s %s' % hdr)\n", "for i, w in enumerate(sent, 1):\n", " print('%3d %-12s%-12s%-8s%4s %s' % (i, w['form'], w['lemma'], w['upos'], w['head'], w['deprel']))\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 4. Vowel length and metre\n", "\n", "Greek orthography never marks vowel length: α, ι and υ — the *dichrona* — are ambiguous.\n", "Recovering it (*macronization*) is lexical knowledge, and it is the prerequisite for scanning\n", "verse. `Stoicheia-meter` does both at once; `Stoicheia-macronizer` does vowel length alone,\n", "slightly better.\n" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "REPO = \"Ericu950/Stoicheia-meter\"\n", "local = snapshot_download(REPO, allow_patterns=[\"*.json\", \"*.txt\", \"*.py\", \"*.model\"])\n", "sys.path.insert(0, local)\n", "from processing_char_bert_meter import CharBertMeterProcessor\n", "\n", "meter_model = AutoModel.from_pretrained(REPO, trust_remote_code=True).eval()\n", "mproc = CharBertMeterProcessor()\n", "\n", "line = \"ἄνδρα μοι ἔννεπε, μοῦσα, πολύτροπον, ὃς μάλα πολλὰ\"\n", "batch = mproc(line)\n", "with torch.no_grad():\n", " out = meter_model(**{k: v for k, v in batch.items() if not k.startswith(\"_\")})\n", "print(\"macronized:\", mproc.decode_macronization(out, batch)) # _ long, ^ short\n", "print(\"scanned :\", mproc.decode_scansion(out, batch)) # [heavy] {light}\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 5. Scoring against the benchmark\n", "\n", "*Norma Syllabarum Graecarum* is a hand-annotated benchmark of macronization and\n", "syllabification. Here we score the dedicated macronizer on its test split — every ambiguous\n", "α/ι/υ position, compared against the gold mark.\n" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "import json, re\n", "from huggingface_hub import hf_hub_download\n", "\n", "REPO = \"Ericu950/Stoicheia-macronizer\"\n", "local = snapshot_download(REPO, allow_patterns=[\"*.json\", \"*.txt\", \"*.py\", \"*.model\"])\n", "sys.path.insert(0, local)\n", "mac_model = AutoModel.from_pretrained(REPO, trust_remote_code=True).eval()\n", "\n", "path = hf_hub_download(\"Ericu950/norma\", \"data/test.jsonl\", repo_type=\"dataset\")\n", "rows = [json.loads(l) for l in open(path, encoding=\"utf-8\")]\n", "rows = [r for r in rows if r[\"task\"] == \"macronize\"][:120] # raise for the full set\n", "\n", "MARKS = re.compile(r\"[_^]\")\n", "n = correct = 0\n", "for r in rows:\n", " gold = r[\"text\"]\n", " raw = MARKS.sub(\"\", gold)\n", " batch = mproc(raw)\n", " with torch.no_grad():\n", " out = mac_model(**{k: v for k, v in batch.items() if not k.startswith(\"_\")})\n", " pred = mproc.decode_macronization(out, batch)\n", " for g, p in zip(gold, pred):\n", " pass\n", " # compare mark-by-mark at the positions the gold marks\n", " gi = pi = 0\n", " while gi < len(gold) and pi < len(pred):\n", " if gold[gi] in \"_^\" and pred[pi] in \"_^\":\n", " n += 1; correct += (gold[gi] == pred[pi]); gi += 1; pi += 1\n", " elif gold[gi] in \"_^\":\n", " n += 1; gi += 1\n", " elif pred[pi] in \"_^\":\n", " pi += 1\n", " else:\n", " gi += 1; pi += 1\n", "print(f\"macronization accuracy on {len(rows)} lines: {100*correct/max(n,1):.2f}% ({n} scored positions)\")\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "\n", "**Everything in the release**\n", "\n", "| | |\n", "|---|---|\n", "| 11 pretrained backbones | ten rotated literary folds + one documentary-clean |\n", "| 10 restoration checkpoints | one per held-out PHI/TM digit |\n", "| tagger-parser, meter, macronizer | fine-tuned from the documentary-clean backbone |\n", "| 5 datasets | pretraining corpus, synthetic augmentation, inscriptions, meter silver, benchmark |\n", "\n", "Training and evaluation code, including the split pipeline that produces the decontamination\n", "guarantee, is in the accompanying code repository.\n" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "name": "python" }, "colab": { "provenance": [], "toc_visible": true }, "accelerator": "GPU" }, "nbformat": 4, "nbformat_minor": 0 }