Instructions to use Felladrin/gguf-prem-1B-chat with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Felladrin/gguf-prem-1B-chat with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf Felladrin/gguf-prem-1B-chat:Q4_K_M # Run inference directly in the terminal: llama cli -hf Felladrin/gguf-prem-1B-chat:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf Felladrin/gguf-prem-1B-chat:Q4_K_M # Run inference directly in the terminal: llama cli -hf Felladrin/gguf-prem-1B-chat:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf Felladrin/gguf-prem-1B-chat:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf Felladrin/gguf-prem-1B-chat:Q4_K_M
Use Docker
docker model run hf.co/Felladrin/gguf-prem-1B-chat:Q4_K_M
- Llama new
- LM Studio
- Jan
- Ollama
How to use Felladrin/gguf-prem-1B-chat with Ollama:
ollama run hf.co/Felladrin/gguf-prem-1B-chat:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use Felladrin/gguf-prem-1B-chat with Docker Model Runner:
docker model run hf.co/Felladrin/gguf-prem-1B-chat:Q4_K_M
- Lemonade
How to use Felladrin/gguf-prem-1B-chat with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Felladrin/gguf-prem-1B-chat:Q4_K_M
Run and chat with the model
lemonade run user.gguf-prem-1B-chat-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Upload folder using huggingface_hub
Browse files- .gitattributes +11 -0
- prem-1B-chat.F16.gguf +3 -0
- prem-1B-chat.Q2_K.gguf +3 -0
- prem-1B-chat.Q3_K_M.gguf +3 -0
- prem-1B-chat.Q3_K_S.gguf +3 -0
- prem-1B-chat.Q4_0.gguf +3 -0
- prem-1B-chat.Q4_K_M.gguf +3 -0
- prem-1B-chat.Q4_K_S.gguf +3 -0
- prem-1B-chat.Q5_K_M.gguf +3 -0
- prem-1B-chat.Q5_K_S.gguf +3 -0
- prem-1B-chat.Q6_K.gguf +3 -0
- prem-1B-chat.Q8_0.gguf +3 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,14 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
prem-1B-chat.F16.gguf filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
prem-1B-chat.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
prem-1B-chat.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
| 39 |
+
prem-1B-chat.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
| 40 |
+
prem-1B-chat.Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
|
| 41 |
+
prem-1B-chat.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
| 42 |
+
prem-1B-chat.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
| 43 |
+
prem-1B-chat.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
| 44 |
+
prem-1B-chat.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
| 45 |
+
prem-1B-chat.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
|
| 46 |
+
prem-1B-chat.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
prem-1B-chat.F16.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1aa704da4e3d771d46e6e9b505c6a709bd78d7e01d464ba3d5102877ef9d34e9
|
| 3 |
+
size 2201050304
|
prem-1B-chat.Q2_K.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:dd2f35016f91d3055013ce3491e09003aa2cb6bb7927517d3b0858a754ec8c8c
|
| 3 |
+
size 432140672
|
prem-1B-chat.Q3_K_M.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4ac1581cd5c181f0f475c437ce8abdee48a60ecf90c8a32d6ab987737516b07f
|
| 3 |
+
size 548414656
|
prem-1B-chat.Q3_K_S.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c8690e9df99bab4c99cbd02638cf125f5bc9033bb8d5a4e073fda63af92625bc
|
| 3 |
+
size 499352768
|
prem-1B-chat.Q4_0.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d2f1d720d5c32e30b78ce706e5a92594761f6a32a52f34e409d385bb90d8c2dc
|
| 3 |
+
size 636737792
|
prem-1B-chat.Q4_K_M.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d08887ec88b65ce7f5efdbf8132558350cf6984cb79d7eb69438a8c2a9f1f742
|
| 3 |
+
size 667826432
|
prem-1B-chat.Q4_K_S.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a9dedeeb41e8a6cd6e282cbeda8b39d36203747e2ba4b29ab427ffd8c7b2bdad
|
| 3 |
+
size 639883520
|
prem-1B-chat.Q5_K_M.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:190656fba733cea19ad34ff2701313f7141173b11a914de369e60098eb1f2b86
|
| 3 |
+
size 782056704
|
prem-1B-chat.Q5_K_S.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ca1c6726a8d111938aca1962417c8b835727568cc180ed3e5ab9d14622947cb3
|
| 3 |
+
size 766041344
|
prem-1B-chat.Q6_K.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4a66604c99694fcde3c7d00806d295c761565f962be434e44d88d42811fc43a9
|
| 3 |
+
size 903426368
|
prem-1B-chat.Q8_0.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6348cbe1e0d84195f391278fa6dc67607f22c544f9a30c7e04fca9eef7e4c2c8
|
| 3 |
+
size 1169825984
|