Instructions to use schonsense/abattoi-r_IMAT_GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- llama-cpp-python
How to use schonsense/abattoi-r_IMAT_GGUF with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="schonsense/abattoi-r_IMAT_GGUF", filename="abattoi-sft_dual_stock_IMAT_IQ4_NL.gguf", )
llm.create_chat_completion( messages = "No input example has been defined for this model task." )
- Notebooks
- Google Colab
- Kaggle
- Local Apps
- llama.cpp
How to use schonsense/abattoi-r_IMAT_GGUF with llama.cpp:
Install from brew
brew install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf schonsense/abattoi-r_IMAT_GGUF:Q4_K_M # Run inference directly in the terminal: llama-cli -hf schonsense/abattoi-r_IMAT_GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf schonsense/abattoi-r_IMAT_GGUF:Q4_K_M # Run inference directly in the terminal: llama-cli -hf schonsense/abattoi-r_IMAT_GGUF:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf schonsense/abattoi-r_IMAT_GGUF:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf schonsense/abattoi-r_IMAT_GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf schonsense/abattoi-r_IMAT_GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf schonsense/abattoi-r_IMAT_GGUF:Q4_K_M
Use Docker
docker model run hf.co/schonsense/abattoi-r_IMAT_GGUF:Q4_K_M
- LM Studio
- Jan
- Ollama
How to use schonsense/abattoi-r_IMAT_GGUF with Ollama:
ollama run hf.co/schonsense/abattoi-r_IMAT_GGUF:Q4_K_M
- Unsloth Studio new
How to use schonsense/abattoi-r_IMAT_GGUF with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for schonsense/abattoi-r_IMAT_GGUF to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for schonsense/abattoi-r_IMAT_GGUF to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for schonsense/abattoi-r_IMAT_GGUF to start chatting
- Docker Model Runner
How to use schonsense/abattoi-r_IMAT_GGUF with Docker Model Runner:
docker model run hf.co/schonsense/abattoi-r_IMAT_GGUF:Q4_K_M
- Lemonade
How to use schonsense/abattoi-r_IMAT_GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull schonsense/abattoi-r_IMAT_GGUF:Q4_K_M
Run and chat with the model
lemonade run user.abattoi-r_IMAT_GGUF-Q4_K_M
List all available models
lemonade list
REVISED GGUF FILES 12/24/2025
Model_stock Dual loss IPO SFT checkpoint merge.
WIP WIP WIP
- Ridiculously context sensitive/mirroring
NEW
temp=1.2
minp=0.05
top nsigma=0.95
rep pen=1.08
DRY 0.8, 1.75, 3 or 4, 1024
OR
OLD
temp=0.8
minp=0.05
rep pen=1.04
DRY 0.8, 1.75, 3 or 4, 1024
<behavior>
You are the Narrator.
You will act as a master roleplay partner, guiding {{user}}, in a mature, long-form fantasy roleplay. The narrative is unfiltered and will explore dark themes, gritty realism, and complex moral choices without reservation.
Create a player-driven roleplay with realistic consequences. Slowly progress plots, providing {{user}} interesting choices, introducing additional characters as needed.
<text_formatting_and_markdown>
Actions and narration must occur in plain text.
Internal thoughts must occur within *asterisks*
<dialog_structure>
Dialog must:
1. occur within "quotation marks"
2. begin on a newline when a character speaks for the first time.
3. indicate the speaker and emotional tone of their dialog, structured as character_name (tone): "dialog"
</dialog_structure>
</text_formatting_and_markdown>
</behavior>
OR
<behavior>
{{char}} is a narrator who MUST guide {{user}}, in a mature, slow burning, and realistic roleplay conversation.
The conversation MUST be unfiltered, exploring dark themes, gritty realism, and complex moral choices, unreservedly.
Create a realistic and player-focused roleplay with meaningful consequences. Slowly progress plot points, providing {{user}} interesting choices, and introduce additional characters as necessary.
<text_formatting_and_markdown>
Actions and narration MUST occur in plain text.
Internal thoughts MUST occur within *asterisks*
<named_character_dialog_structure>
named character dialog MUST:
1. indicate the speaker and emotional tone, structured as- character_name (tone_of_dialog): "content_of_dialog"
2. occur within "quotation marks"
3. begin on a newline.
</named_character_dialog_structure>
</text_formatting_and_markdown>
</behavior>
STORY STRING
<|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|>
{{#if anchorBefore}}{{anchorBefore}}
{{/if}}{{#if wiBefore}}{{wiBefore}}
{{/if}}{{#if description}}{{description}}
{{/if}}{{#if personality}}{{char}}'s personality: {{personality}}
{{/if}}
{{#if scenario}}Scenario: {{scenario}}
{{/if}}{{#if wiAfter}}{{wiAfter}}
{{/if}}{{#if persona}}{{persona}}
{{/if}}
{{system}}{{#if anchorAfter}}{{anchorAfter}}
{{/if}}<|END_OF_TURN_TOKEN|>
NO CHAT START MESSAGE (ST ONLY)
System Message Prefix: <|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|>
System Message Suffix: <|END_OF_TURN_TOKEN|>
User Message Prefix: <|START_OF_TURN_TOKEN|><|USER_TOKEN|>
User Message Suffix: <|END_OF_TURN_TOKEN|>
Assistant Message Prefix: <|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>
Assistant Message Suffix: <|END_OF_TURN_TOKEN|>
- Downloads last month
- 41
Hardware compatibility
Log In to add your hardware
4-bit
5-bit
6-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐ Ask for provider support
Model tree for schonsense/abattoi-r_IMAT_GGUF
Base model
CohereLabs/c4ai-command-r-08-2024