Update README.md
Browse files
README.md
CHANGED
|
@@ -17,9 +17,9 @@ license: other
|
|
| 17 |
</div>
|
| 18 |
<!-- header end -->
|
| 19 |
|
| 20 |
-
# TII's Falcon 7B
|
| 21 |
|
| 22 |
-
These files are GGML format model files for [TII's Falcon 7B
|
| 23 |
|
| 24 |
These files will **not** work in llama.cpp, text-generation-webui or KoboldCpp.
|
| 25 |
|
|
@@ -56,9 +56,11 @@ Compiling on Windows: developer cmp-nct notes: 'I personally compile it using VS
|
|
| 56 |
|
| 57 |
Once compiled you can then use `bin/falcon_main` just like you would use llama.cpp. For example:
|
| 58 |
```
|
| 59 |
-
bin/falcon_main -t 8 -ngl 100 -b 1 -m falcon-
|
| 60 |
```
|
| 61 |
|
|
|
|
|
|
|
| 62 |
You can specify `-ngl 100` regardles of your VRAM, as it will automatically detect how much VRAM is available to be used.
|
| 63 |
|
| 64 |
Adjust `-t 8` (the number of CPU cores to use) according to what performs best on your system. Do not exceed the number of physical CPU cores you have.
|
|
@@ -108,7 +110,7 @@ Thank you to all my generous patrons and donaters!
|
|
| 108 |
|
| 109 |
<!-- footer end -->
|
| 110 |
|
| 111 |
-
# Original model card: TII's Falcon 7B
|
| 112 |
|
| 113 |
|
| 114 |
# ✨ Falcon-7B-Instruct
|
|
|
|
| 17 |
</div>
|
| 18 |
<!-- header end -->
|
| 19 |
|
| 20 |
+
# TII's Falcon 7B Instruct GGML
|
| 21 |
|
| 22 |
+
These files are GGML format model files for [TII's Falcon 7B Instruct](https://huggingface.co/tiiuae/falcon-7b-instruct).
|
| 23 |
|
| 24 |
These files will **not** work in llama.cpp, text-generation-webui or KoboldCpp.
|
| 25 |
|
|
|
|
| 56 |
|
| 57 |
Once compiled you can then use `bin/falcon_main` just like you would use llama.cpp. For example:
|
| 58 |
```
|
| 59 |
+
bin/falcon_main -t 8 -ngl 100 -b 1 -m falcon-7b-instruct.ggccv1.q4_0.bin -enc -p "write a story about llamas"
|
| 60 |
```
|
| 61 |
|
| 62 |
+
Parameter `-enc` should automatically use the right prompt template for the model, so you can just enter your desired prompt.
|
| 63 |
+
|
| 64 |
You can specify `-ngl 100` regardles of your VRAM, as it will automatically detect how much VRAM is available to be used.
|
| 65 |
|
| 66 |
Adjust `-t 8` (the number of CPU cores to use) according to what performs best on your system. Do not exceed the number of physical CPU cores you have.
|
|
|
|
| 110 |
|
| 111 |
<!-- footer end -->
|
| 112 |
|
| 113 |
+
# Original model card: TII's Falcon 7B Instruct
|
| 114 |
|
| 115 |
|
| 116 |
# ✨ Falcon-7B-Instruct
|