Update README.md
Browse files
README.md
CHANGED
|
@@ -42,14 +42,14 @@ You can use OLMo with the standard HuggingFace transformers library:
|
|
| 42 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 43 |
olmo = AutoModelForCausalLM.from_pretrained("allenai/Olmo-3-7B-Instruct")
|
| 44 |
tokenizer = AutoTokenizer.from_pretrained("allenai/Olmo-3-7B-Instruct")
|
| 45 |
-
message = ["
|
| 46 |
inputs = tokenizer(message, return_tensors='pt', return_token_type_ids=False)
|
| 47 |
# optional verifying cuda
|
| 48 |
# inputs = {k: v.to('cuda') for k,v in inputs.items()}
|
| 49 |
# olmo = olmo.to('cuda')
|
| 50 |
response = olmo.generate(**inputs, max_new_tokens=100, do_sample=True, top_k=50, top_p=0.95)
|
| 51 |
print(tokenizer.batch_decode(response, skip_special_tokens=True)[0])
|
| 52 |
-
>> '
|
| 53 |
```
|
| 54 |
|
| 55 |
For faster performance, you can quantize the model using the following method:
|
|
@@ -78,6 +78,31 @@ out = list_repo_refs("allenai/Olmo-3-7B-Instruct")
|
|
| 78 |
branches = [b.name for b in out.branches]
|
| 79 |
```
|
| 80 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 81 |
### Model Description
|
| 82 |
|
| 83 |
- **Developed by:** Allen Institute for AI (Ai2)
|
|
|
|
| 42 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 43 |
olmo = AutoModelForCausalLM.from_pretrained("allenai/Olmo-3-7B-Instruct")
|
| 44 |
tokenizer = AutoTokenizer.from_pretrained("allenai/Olmo-3-7B-Instruct")
|
| 45 |
+
message = ["Who would win in a fight - a dinosaur or a cow named Moo Moo?"]
|
| 46 |
inputs = tokenizer(message, return_tensors='pt', return_token_type_ids=False)
|
| 47 |
# optional verifying cuda
|
| 48 |
# inputs = {k: v.to('cuda') for k,v in inputs.items()}
|
| 49 |
# olmo = olmo.to('cuda')
|
| 50 |
response = olmo.generate(**inputs, max_new_tokens=100, do_sample=True, top_k=50, top_p=0.95)
|
| 51 |
print(tokenizer.batch_decode(response, skip_special_tokens=True)[0])
|
| 52 |
+
>> 'This is a fun and imaginative question! Let’s break it down...'
|
| 53 |
```
|
| 54 |
|
| 55 |
For faster performance, you can quantize the model using the following method:
|
|
|
|
| 78 |
branches = [b.name for b in out.branches]
|
| 79 |
```
|
| 80 |
|
| 81 |
+
## Chat template
|
| 82 |
+
|
| 83 |
+
## Default System Message
|
| 84 |
+
The default system prompt for this model is:
|
| 85 |
+
```
|
| 86 |
+
<|im_start|>system
|
| 87 |
+
You are a helpful function-calling AI assistant.
|
| 88 |
+
You do not currently have access to any functions. <functions></functions><|im_end|>
|
| 89 |
+
```
|
| 90 |
+
|
| 91 |
+
## Chat Format
|
| 92 |
+
|
| 93 |
+
The chat template for this model is formatted as:
|
| 94 |
+
```
|
| 95 |
+
<|im_start|>system
|
| 96 |
+
You are a helpful function-calling AI assistant.
|
| 97 |
+
You do not currently have access to any functions. <functions></functions><|im_end|>
|
| 98 |
+
<|im_start|>user
|
| 99 |
+
Who would win in a fight - a dinosaur or a cow named Moo Moo?<|im_end|>
|
| 100 |
+
<|im_start|>assistant
|
| 101 |
+
This is a fun and imaginative question! Let’s break it down...
|
| 102 |
+
Moo Moo the cow would certinaly win.
|
| 103 |
+
<|endoftext|>
|
| 104 |
+
```
|
| 105 |
+
|
| 106 |
### Model Description
|
| 107 |
|
| 108 |
- **Developed by:** Allen Institute for AI (Ai2)
|