PolDense-400M-GGUF

GGUF conversion of OPI-PIB/PolDense-400M (Polish dense-retrieval embedding model, ModernBERT architecture) for llama.cpp and tools built on it.

This is an unofficial conversion, not affiliated with OPI-PIB. The original weights were modified: converted to GGUF and quantized (F16, Q8_0). Nothing else was changed. For model details, training and benchmarks see the original model card.

Files

file	type	size	sha256
PolDense-400M-F16.gguf	F16	755 MiB	029b9dacc1d87e7cc80bec5d71d617c85e598cc0a5f983627b698969cc4c2dff
PolDense-400M-Q8_0.gguf	Q8_0	402 MiB	f9a944b8b2a5b76246d536f67156cf9c8ce09143ec46dde8312416b66e58a28b

Source: OPI-PIB/PolDense-400M revision 56c65740d4fe82fc5e6c46438b550a7c82b29e1b.

Usage

Two things are required to get correct embeddings:

1. Query prefix. Prefix retrieval queries with "[query]: "; documents get no prefix. For symmetric tasks (semantic similarity) prefix both texts with "[sts]: " (as described in the original card).
2. Whole input in one batch. The model uses bidirectional attention, so llama.cpp must process the whole text in one micro-batch: set -ub (and -b) to at least the longest input in tokens. Typically -c 8192 -b 8192 -ub 8192. llama-server rejects longer inputs with an error. Some wrappers silently split the text into 512-token pieces instead, and the vector then covers only the last piece. One example is qmd ≤ 2.8.3, via node-llama-cpp's default batch size of 512.

Note: GGUF runtimes do not read the prompts from config_sentence_transformers.json, so the prefix must always be added by your code.

llama-server -m PolDense-400M-Q8_0.gguf --embedding --pooling cls -c 8192 -b 8192 -ub 8192

curl -s localhost:8080/v1/embeddings -H 'Content-Type: application/json' \
  -d '{"input": ["[query]: Jak dożyć 100 lat?", "Trzeba zdrowo się odżywiać i uprawiać sport."]}'

Pooling is CLS (stored in the GGUF metadata; --pooling cls only makes it explicit). Normalize vectors and use cosine similarity.

Verification

Cosine similarity between llama-server embeddings (CPU) and the original model run with sentence-transformers (fp32, CPU). The test used 6 short Polish/English texts with and without the query prefix, plus one long Polish text (3052 tokens, from the Polish Wikipedia article "Białystok").

file	min cos, short texts	cos, long text
F16	0.99962	0.99994
Q8_0	0.99853	0.99955

Conversion: convert_hf_to_gguf.py from llama.cpp commit 4a80943 (--outtype f32), then llama-quantize (release b9297) to F16 and Q8_0. Test runtime: llama-server b9297. Reference: transformers 4.56.1, sentence-transformers 5.1.0, torch 2.10.

License and use restrictions

The original model is released under the Gemma Terms of Use, and so is this conversion. A copy of the terms is included in GEMMA_TERMS.txt; see also NOTICE.

Gemma is provided under and subject to the Gemma Terms of Use found at ai.google.dev/gemma/terms.

These files are subject to the use restrictions in Section 3.2 of the Gemma Terms of Use. You must not use them for the restricted uses set forth in the Gemma Prohibited Use Policy, or in violation of applicable laws and regulations. If you redistribute these files or derivatives, you must include these use restrictions as an enforceable provision in your terms. You must also pass on the notice and a copy of the Gemma Terms of Use.