Text Ranking
sentence-transformers
PyTorch
ONNX
Safetensors
OpenVINO
Transformers
xlm-roberta
text-classification
text-embeddings-inference
Instructions to use cross-encoder/mmarco-mMiniLMv2-L12-H384-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use cross-encoder/mmarco-mMiniLMv2-L12-H384-v1 with sentence-transformers:
from sentence_transformers import CrossEncoder model = CrossEncoder("cross-encoder/mmarco-mMiniLMv2-L12-H384-v1") query = "Which planet is known as the Red Planet?" passages = [ "Venus is often called Earth's twin because of its similar size and proximity.", "Mars, known for its reddish appearance, is often referred to as the Red Planet.", "Jupiter, the largest planet in our solar system, has a prominent red spot.", "Saturn, famous for its rings, is sometimes mistaken for the Red Planet." ] scores = model.predict([(query, passage) for passage in passages]) print(scores) - Transformers
How to use cross-encoder/mmarco-mMiniLMv2-L12-H384-v1 with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForSequenceClassification tokenizer = AutoTokenizer.from_pretrained("cross-encoder/mmarco-mMiniLMv2-L12-H384-v1") model = AutoModelForSequenceClassification.from_pretrained("cross-encoder/mmarco-mMiniLMv2-L12-H384-v1", device_map="auto") - Inference
- Notebooks
- Google Colab
- Kaggle
Add new CrossEncoder model
Browse files- .gitattributes +1 -0
- README.md +62 -62
- config.json +36 -34
- onnx/model.onnx +3 -0
- special_tokens_map.json +51 -1
- tokenizer.json +2 -2
- tokenizer_config.json +55 -1
.gitattributes
CHANGED
|
@@ -26,3 +26,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 26 |
*.zstandard filter=lfs diff=lfs merge=lfs -text
|
| 27 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 28 |
model.safetensors filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 26 |
*.zstandard filter=lfs diff=lfs merge=lfs -text
|
| 27 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 28 |
model.safetensors filter=lfs diff=lfs merge=lfs -text
|
| 29 |
+
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -1,63 +1,63 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: apache-2.0
|
| 3 |
-
language:
|
| 4 |
-
- en
|
| 5 |
-
- ar
|
| 6 |
-
- zh
|
| 7 |
-
- nl
|
| 8 |
-
- fr
|
| 9 |
-
- de
|
| 10 |
-
- hi
|
| 11 |
-
- in
|
| 12 |
-
- it
|
| 13 |
-
- ja
|
| 14 |
-
- pt
|
| 15 |
-
- ru
|
| 16 |
-
- es
|
| 17 |
-
- vi
|
| 18 |
-
- multilingual
|
| 19 |
-
datasets:
|
| 20 |
-
- unicamp-dl/mmarco
|
| 21 |
-
base_model:
|
| 22 |
-
- nreimers/mMiniLMv2-L12-H384-distilled-from-XLMR-Large
|
| 23 |
-
pipeline_tag: text-ranking
|
| 24 |
-
library_name: sentence-transformers
|
| 25 |
-
tags:
|
| 26 |
-
- transformers
|
| 27 |
-
---
|
| 28 |
-
# Cross-Encoder for multilingual MS Marco
|
| 29 |
-
|
| 30 |
-
This model was trained on the [MMARCO](https://hf.co/unicamp-dl/mmarco) dataset. It is a machine translated version of MS MARCO using Google Translate. It was translated to 14 languages. In our experiments, we observed that it performs also well for other languages.
|
| 31 |
-
|
| 32 |
-
As a base model, we used the [multilingual MiniLMv2](https://huggingface.co/nreimers/mMiniLMv2-L12-H384-distilled-from-XLMR-Large) model.
|
| 33 |
-
|
| 34 |
-
The model can be used for Information Retrieval: Given a query, encode the query will all possible passages (e.g. retrieved with ElasticSearch). Then sort the passages in a decreasing order. See [SBERT.net Retrieve & Re-rank](https://www.sbert.net/examples/applications/retrieve_rerank/README.html) for more details. The training code is available here: [SBERT.net Training MS Marco](https://github.com/UKPLab/sentence-transformers/tree/master/examples/training/ms_marco)
|
| 35 |
-
|
| 36 |
-
## Usage with SentenceTransformers
|
| 37 |
-
|
| 38 |
-
The usage becomes easy when you have [SentenceTransformers](https://www.sbert.net/) installed. Then, you can use the pre-trained models like this:
|
| 39 |
-
```python
|
| 40 |
-
from sentence_transformers import CrossEncoder
|
| 41 |
-
model = CrossEncoder('model_name')
|
| 42 |
-
scores = model.predict([('Query', 'Paragraph1'), ('Query', 'Paragraph2') , ('Query', 'Paragraph3')])
|
| 43 |
-
```
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
## Usage with Transformers
|
| 49 |
-
|
| 50 |
-
```python
|
| 51 |
-
from transformers import AutoTokenizer, AutoModelForSequenceClassification
|
| 52 |
-
import torch
|
| 53 |
-
|
| 54 |
-
model = AutoModelForSequenceClassification.from_pretrained('model_name')
|
| 55 |
-
tokenizer = AutoTokenizer.from_pretrained('model_name')
|
| 56 |
-
|
| 57 |
-
features = tokenizer(['How many people live in Berlin?', 'How many people live in Berlin?'], ['Berlin has a population of 3,520,031 registered inhabitants in an area of 891.82 square kilometers.', 'New York City is famous for the Metropolitan Museum of Art.'], padding=True, truncation=True, return_tensors="pt")
|
| 58 |
-
|
| 59 |
-
model.eval()
|
| 60 |
-
with torch.no_grad():
|
| 61 |
-
scores = model(**features).logits
|
| 62 |
-
print(scores)
|
| 63 |
```
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
- ar
|
| 6 |
+
- zh
|
| 7 |
+
- nl
|
| 8 |
+
- fr
|
| 9 |
+
- de
|
| 10 |
+
- hi
|
| 11 |
+
- in
|
| 12 |
+
- it
|
| 13 |
+
- ja
|
| 14 |
+
- pt
|
| 15 |
+
- ru
|
| 16 |
+
- es
|
| 17 |
+
- vi
|
| 18 |
+
- multilingual
|
| 19 |
+
datasets:
|
| 20 |
+
- unicamp-dl/mmarco
|
| 21 |
+
base_model:
|
| 22 |
+
- nreimers/mMiniLMv2-L12-H384-distilled-from-XLMR-Large
|
| 23 |
+
pipeline_tag: text-ranking
|
| 24 |
+
library_name: sentence-transformers
|
| 25 |
+
tags:
|
| 26 |
+
- transformers
|
| 27 |
+
---
|
| 28 |
+
# Cross-Encoder for multilingual MS Marco
|
| 29 |
+
|
| 30 |
+
This model was trained on the [MMARCO](https://hf.co/unicamp-dl/mmarco) dataset. It is a machine translated version of MS MARCO using Google Translate. It was translated to 14 languages. In our experiments, we observed that it performs also well for other languages.
|
| 31 |
+
|
| 32 |
+
As a base model, we used the [multilingual MiniLMv2](https://huggingface.co/nreimers/mMiniLMv2-L12-H384-distilled-from-XLMR-Large) model.
|
| 33 |
+
|
| 34 |
+
The model can be used for Information Retrieval: Given a query, encode the query will all possible passages (e.g. retrieved with ElasticSearch). Then sort the passages in a decreasing order. See [SBERT.net Retrieve & Re-rank](https://www.sbert.net/examples/applications/retrieve_rerank/README.html) for more details. The training code is available here: [SBERT.net Training MS Marco](https://github.com/UKPLab/sentence-transformers/tree/master/examples/training/ms_marco)
|
| 35 |
+
|
| 36 |
+
## Usage with SentenceTransformers
|
| 37 |
+
|
| 38 |
+
The usage becomes easy when you have [SentenceTransformers](https://www.sbert.net/) installed. Then, you can use the pre-trained models like this:
|
| 39 |
+
```python
|
| 40 |
+
from sentence_transformers import CrossEncoder
|
| 41 |
+
model = CrossEncoder('model_name')
|
| 42 |
+
scores = model.predict([('Query', 'Paragraph1'), ('Query', 'Paragraph2') , ('Query', 'Paragraph3')])
|
| 43 |
+
```
|
| 44 |
+
|
| 45 |
+
|
| 46 |
+
|
| 47 |
+
|
| 48 |
+
## Usage with Transformers
|
| 49 |
+
|
| 50 |
+
```python
|
| 51 |
+
from transformers import AutoTokenizer, AutoModelForSequenceClassification
|
| 52 |
+
import torch
|
| 53 |
+
|
| 54 |
+
model = AutoModelForSequenceClassification.from_pretrained('model_name')
|
| 55 |
+
tokenizer = AutoTokenizer.from_pretrained('model_name')
|
| 56 |
+
|
| 57 |
+
features = tokenizer(['How many people live in Berlin?', 'How many people live in Berlin?'], ['Berlin has a population of 3,520,031 registered inhabitants in an area of 891.82 square kilometers.', 'New York City is famous for the Metropolitan Museum of Art.'], padding=True, truncation=True, return_tensors="pt")
|
| 58 |
+
|
| 59 |
+
model.eval()
|
| 60 |
+
with torch.no_grad():
|
| 61 |
+
scores = model(**features).logits
|
| 62 |
+
print(scores)
|
| 63 |
```
|
config.json
CHANGED
|
@@ -1,34 +1,36 @@
|
|
| 1 |
-
{
|
| 2 |
-
"
|
| 3 |
-
|
| 4 |
-
|
| 5 |
-
|
| 6 |
-
"
|
| 7 |
-
"
|
| 8 |
-
"
|
| 9 |
-
"
|
| 10 |
-
"
|
| 11 |
-
"
|
| 12 |
-
"
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
"
|
| 17 |
-
"
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
"
|
| 22 |
-
"
|
| 23 |
-
"
|
| 24 |
-
"
|
| 25 |
-
"
|
| 26 |
-
"
|
| 27 |
-
"
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
"
|
| 32 |
-
"
|
| 33 |
-
"
|
| 34 |
-
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"architectures": [
|
| 3 |
+
"XLMRobertaForSequenceClassification"
|
| 4 |
+
],
|
| 5 |
+
"attention_probs_dropout_prob": 0.1,
|
| 6 |
+
"bos_token_id": 0,
|
| 7 |
+
"classifier_dropout": null,
|
| 8 |
+
"eos_token_id": 2,
|
| 9 |
+
"hidden_act": "gelu",
|
| 10 |
+
"hidden_dropout_prob": 0.1,
|
| 11 |
+
"hidden_size": 384,
|
| 12 |
+
"id2label": {
|
| 13 |
+
"0": "LABEL_0"
|
| 14 |
+
},
|
| 15 |
+
"initializer_range": 0.02,
|
| 16 |
+
"intermediate_size": 1536,
|
| 17 |
+
"label2id": {
|
| 18 |
+
"LABEL_0": 0
|
| 19 |
+
},
|
| 20 |
+
"layer_norm_eps": 1e-05,
|
| 21 |
+
"max_position_embeddings": 514,
|
| 22 |
+
"model_type": "xlm-roberta",
|
| 23 |
+
"num_attention_heads": 12,
|
| 24 |
+
"num_hidden_layers": 12,
|
| 25 |
+
"pad_token_id": 1,
|
| 26 |
+
"position_embedding_type": "absolute",
|
| 27 |
+
"sentence_transformers": {
|
| 28 |
+
"activation_fn": "torch.nn.modules.linear.Identity",
|
| 29 |
+
"version": "4.1.0.dev0"
|
| 30 |
+
},
|
| 31 |
+
"torch_dtype": "float32",
|
| 32 |
+
"transformers_version": "4.52.0.dev0",
|
| 33 |
+
"type_vocab_size": 1,
|
| 34 |
+
"use_cache": true,
|
| 35 |
+
"vocab_size": 250002
|
| 36 |
+
}
|
onnx/model.onnx
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3e9a03ed1e966f7c5288dd4230e3d6a9bf5e3a170a06f1f4241c5bca12c6487c
|
| 3 |
+
size 470883696
|
special_tokens_map.json
CHANGED
|
@@ -1 +1,51 @@
|
|
| 1 |
-
{
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"bos_token": {
|
| 3 |
+
"content": "<s>",
|
| 4 |
+
"lstrip": false,
|
| 5 |
+
"normalized": false,
|
| 6 |
+
"rstrip": false,
|
| 7 |
+
"single_word": false
|
| 8 |
+
},
|
| 9 |
+
"cls_token": {
|
| 10 |
+
"content": "<s>",
|
| 11 |
+
"lstrip": false,
|
| 12 |
+
"normalized": false,
|
| 13 |
+
"rstrip": false,
|
| 14 |
+
"single_word": false
|
| 15 |
+
},
|
| 16 |
+
"eos_token": {
|
| 17 |
+
"content": "</s>",
|
| 18 |
+
"lstrip": false,
|
| 19 |
+
"normalized": false,
|
| 20 |
+
"rstrip": false,
|
| 21 |
+
"single_word": false
|
| 22 |
+
},
|
| 23 |
+
"mask_token": {
|
| 24 |
+
"content": "<mask>",
|
| 25 |
+
"lstrip": true,
|
| 26 |
+
"normalized": false,
|
| 27 |
+
"rstrip": false,
|
| 28 |
+
"single_word": false
|
| 29 |
+
},
|
| 30 |
+
"pad_token": {
|
| 31 |
+
"content": "<pad>",
|
| 32 |
+
"lstrip": false,
|
| 33 |
+
"normalized": false,
|
| 34 |
+
"rstrip": false,
|
| 35 |
+
"single_word": false
|
| 36 |
+
},
|
| 37 |
+
"sep_token": {
|
| 38 |
+
"content": "</s>",
|
| 39 |
+
"lstrip": false,
|
| 40 |
+
"normalized": false,
|
| 41 |
+
"rstrip": false,
|
| 42 |
+
"single_word": false
|
| 43 |
+
},
|
| 44 |
+
"unk_token": {
|
| 45 |
+
"content": "<unk>",
|
| 46 |
+
"lstrip": false,
|
| 47 |
+
"normalized": false,
|
| 48 |
+
"rstrip": false,
|
| 49 |
+
"single_word": false
|
| 50 |
+
}
|
| 51 |
+
}
|
tokenizer.json
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3a56def25aa40facc030ea8b0b87f3688e4b3c39eb8b45d5702b3a1300fe2a20
|
| 3 |
+
size 17082734
|
tokenizer_config.json
CHANGED
|
@@ -1 +1,55 @@
|
|
| 1 |
-
{
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"added_tokens_decoder": {
|
| 3 |
+
"0": {
|
| 4 |
+
"content": "<s>",
|
| 5 |
+
"lstrip": false,
|
| 6 |
+
"normalized": false,
|
| 7 |
+
"rstrip": false,
|
| 8 |
+
"single_word": false,
|
| 9 |
+
"special": true
|
| 10 |
+
},
|
| 11 |
+
"1": {
|
| 12 |
+
"content": "<pad>",
|
| 13 |
+
"lstrip": false,
|
| 14 |
+
"normalized": false,
|
| 15 |
+
"rstrip": false,
|
| 16 |
+
"single_word": false,
|
| 17 |
+
"special": true
|
| 18 |
+
},
|
| 19 |
+
"2": {
|
| 20 |
+
"content": "</s>",
|
| 21 |
+
"lstrip": false,
|
| 22 |
+
"normalized": false,
|
| 23 |
+
"rstrip": false,
|
| 24 |
+
"single_word": false,
|
| 25 |
+
"special": true
|
| 26 |
+
},
|
| 27 |
+
"3": {
|
| 28 |
+
"content": "<unk>",
|
| 29 |
+
"lstrip": false,
|
| 30 |
+
"normalized": false,
|
| 31 |
+
"rstrip": false,
|
| 32 |
+
"single_word": false,
|
| 33 |
+
"special": true
|
| 34 |
+
},
|
| 35 |
+
"250001": {
|
| 36 |
+
"content": "<mask>",
|
| 37 |
+
"lstrip": true,
|
| 38 |
+
"normalized": false,
|
| 39 |
+
"rstrip": false,
|
| 40 |
+
"single_word": false,
|
| 41 |
+
"special": true
|
| 42 |
+
}
|
| 43 |
+
},
|
| 44 |
+
"bos_token": "<s>",
|
| 45 |
+
"clean_up_tokenization_spaces": false,
|
| 46 |
+
"cls_token": "<s>",
|
| 47 |
+
"eos_token": "</s>",
|
| 48 |
+
"extra_special_tokens": {},
|
| 49 |
+
"mask_token": "<mask>",
|
| 50 |
+
"model_max_length": 512,
|
| 51 |
+
"pad_token": "<pad>",
|
| 52 |
+
"sep_token": "</s>",
|
| 53 |
+
"tokenizer_class": "XLMRobertaTokenizer",
|
| 54 |
+
"unk_token": "<unk>"
|
| 55 |
+
}
|