aivrar-code/distilbart-mnli-12-3-onnx

distilbart-mnli-12-3 (single-file ONNX)

A single-file ONNX export of valhalla/distilbart-mnli-12-3, a distilled BART model (12 encoder layers, 3 decoder layers) fine-tuned on MNLI for natural language inference / zero-shot classification.

I made this export for serp-to-prompt-writer, a Windows SEO tool written in C that calls it through the ONNX Runtime C API to classify the content type of a search query (how-to, comparison, review, recipe, health guide, ...). It is published here so the file has a stable, documented home.

• Format: one self-contained nli.onnx file (~1.03 GB / ~978 MiB, float32, no external data files, opset 18).
• Not quantized, not modified. Logits match the original PyTorch model (max absolute difference about 6e-6 on the sentence pairs I compared).
• There is another ONNX export of the same family from the community: onnx-community/distilbart-mnli-12-3-ONNX; use that one if you want the standard Optimum layout or quantized variants. This repo is simply the single-file version my C app downloads.

Converted by aivrar with AI-assisted tooling.

Inputs and outputs

Inputs (both int64, dynamic shape):

Name	Shape
input_ids	[batch_size, sequence_length]
attention_mask	[batch_size, sequence_length]

Output:

Name	Type	Shape
logits	float32	[batch_size, 3]

Label order

logits[0] = contradiction, logits[1] = neutral, logits[2] = entailment (same as the id2label in the original model config). Apply softmax over the 3 logits; for zero-shot classification use the entailment probability (softmax(logits)[2]).

Tokenizer

The model uses the BART / GPT-2 byte-level BPE tokenizer (vocab size 50265). The vocab.json and merges.txt are those of valhalla/distilbart-mnli-12-3 (this repo does not redistribute them; the C app downloads them from the original repo).

A premise/hypothesis pair is encoded the BART way:

<s> premise </s></s> hypothesis </s>      # ids: 0 ... 2 2 ... 2

with a leading space before the hypothesis text. attention_mask is all ones for a single unpadded sequence.

Usage (Python, onnxruntime)

# pip install onnxruntime tokenizers huggingface_hub numpy
import numpy as np
import onnxruntime as ort
from huggingface_hub import hf_hub_download
from tokenizers import ByteLevelBPETokenizer

vocab = hf_hub_download("valhalla/distilbart-mnli-12-3", "vocab.json")
merges = hf_hub_download("valhalla/distilbart-mnli-12-3", "merges.txt")
tok = ByteLevelBPETokenizer(vocab, merges)

model = hf_hub_download("aivrar-code/distilbart-mnli-12-3-onnx", "nli.onnx")
sess = ort.InferenceSession(model, providers=["CPUExecutionProvider"])

def entailment(premise, hypothesis):
    ids = [0] + tok.encode(premise).ids + [2, 2] + tok.encode(" " + hypothesis).ids + [2]
    ids = np.array([ids], dtype=np.int64)
    logits = sess.run(["logits"], {"input_ids": ids, "attention_mask": np.ones_like(ids)})[0][0]
    p = np.exp(logits - logits.max()); p /= p.sum()
    return float(p[2])  # index 2 = entailment

print(entailment("how to bake sourdough bread at home", "This text is about a cooking recipe."))
print(entailment("how to bake sourdough bread at home", "This text is about a health and medical guide."))

Expected: the first score is high (about 0.68) and the second is near zero.

Use from C

serp-to-prompt-writer (src/nli.c) implements the byte-level BPE tokenizer in plain C, builds the same <s> premise </s></s> hypothesis </s> sequence, feeds input_ids and attention_mask as int64 tensors through the ONNX Runtime C API, and reads logits.

License and attribution

Released under the MIT license. The base model valhalla/distilbart-mnli-12-3 does not declare a license on its Hugging Face page; it is distilled from facebook/bart-large-mnli, which is MIT licensed. If you need a stricter license guarantee, check with the original authors. Credit for the model goes to valhalla (the distilbart-mnli authors) and the original BART authors at Meta AI; this repo only provides the ONNX conversion.

Source and the app that uses it: https://github.com/aivrar/serp-to-prompt-writer