Instructions to use MarinaraSpaghetti/NemoMix-Unleashed-12B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use MarinaraSpaghetti/NemoMix-Unleashed-12B-GGUF with Transformers:

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("MarinaraSpaghetti/NemoMix-Unleashed-12B-GGUF", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use MarinaraSpaghetti/NemoMix-Unleashed-12B-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf MarinaraSpaghetti/NemoMix-Unleashed-12B-GGUF
# Run inference directly in the terminal:
llama cli -hf MarinaraSpaghetti/NemoMix-Unleashed-12B-GGUF

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf MarinaraSpaghetti/NemoMix-Unleashed-12B-GGUF
# Run inference directly in the terminal:
llama cli -hf MarinaraSpaghetti/NemoMix-Unleashed-12B-GGUF

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf MarinaraSpaghetti/NemoMix-Unleashed-12B-GGUF
# Run inference directly in the terminal:
./llama-cli -hf MarinaraSpaghetti/NemoMix-Unleashed-12B-GGUF

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf MarinaraSpaghetti/NemoMix-Unleashed-12B-GGUF
# Run inference directly in the terminal:
./build/bin/llama-cli -hf MarinaraSpaghetti/NemoMix-Unleashed-12B-GGUF

Use Docker

docker model run hf.co/MarinaraSpaghetti/NemoMix-Unleashed-12B-GGUF

LM Studio
Jan
Ollama
How to use MarinaraSpaghetti/NemoMix-Unleashed-12B-GGUF with Ollama:
```
ollama run hf.co/MarinaraSpaghetti/NemoMix-Unleashed-12B-GGUF
```
Unsloth Desktop
Docker Model Runner
How to use MarinaraSpaghetti/NemoMix-Unleashed-12B-GGUF with Docker Model Runner:
```
docker model run hf.co/MarinaraSpaghetti/NemoMix-Unleashed-12B-GGUF
```

Lemonade

How to use MarinaraSpaghetti/NemoMix-Unleashed-12B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull MarinaraSpaghetti/NemoMix-Unleashed-12B-GGUF

Run and chat with the model

lemonade run user.NemoMix-Unleashed-12B-GGUF-{{QUANT_TAG}}

List all available models

lemonade list

Atomic Chat

Having a strange issue while using Nemo with Jan

by lovelyme - opened Jun 27, 2025

Discussion

lovelyme

Jun 27, 2025

•

edited Jun 27, 2025

Nemo worked great on an earlier version of Jan, but now since Jan received an update, it's broken. No matter what I ask it, it's only response is about the Tekken video game. Sometimes in other languages it talks about Tekken nonstop. It's been annoying me for the past days. It sucks because your AI model is my favorite of all my AI models. I've posted screenshots of what I mean.

A guy who had a similar issue pointed to this thread, but I wasn't sure how to apply that information to Nemo. Here it is: https://huggingface.co/unsloth/Mistral-Small-3.2-24B-Instruct-2506-GGUF/discussions/3#685939d254cf0f0546295b27

Tekken Stan Nemo pictures below. This sucks too. I really like Nemo.

MarinaraSpaghetti

Owner Jun 27, 2025

Not sure how I have anything to do with this, I ain’t a Jan developer. You need to contact them, not me.
My guess is they probably messed up the tokenizer, given its name being „Tekken”.

MarinaraSpaghetti changed discussion status to closed Jun 27, 2025

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment