Instructions to use seyuyo/diktalo-e2b-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use seyuyo/diktalo-e2b-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf seyuyo/diktalo-e2b-GGUF:BF16
# Run inference directly in the terminal:
llama cli -hf seyuyo/diktalo-e2b-GGUF:BF16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf seyuyo/diktalo-e2b-GGUF:BF16
# Run inference directly in the terminal:
llama cli -hf seyuyo/diktalo-e2b-GGUF:BF16

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf seyuyo/diktalo-e2b-GGUF:BF16
# Run inference directly in the terminal:
./build/bin/llama cli -hf seyuyo/diktalo-e2b-GGUF:BF16

Use Docker

docker model run hf.co/seyuyo/diktalo-e2b-GGUF:BF16

Llama new
LM Studio
Jan

vLLM

How to use seyuyo/diktalo-e2b-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "seyuyo/diktalo-e2b-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "seyuyo/diktalo-e2b-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/seyuyo/diktalo-e2b-GGUF:BF16

Ollama
How to use seyuyo/diktalo-e2b-GGUF with Ollama:
```
ollama run hf.co/seyuyo/diktalo-e2b-GGUF:BF16
```
Unsloth Desktop

How to use seyuyo/diktalo-e2b-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf seyuyo/diktalo-e2b-GGUF:BF16

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "seyuyo/diktalo-e2b-GGUF:BF16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use seyuyo/diktalo-e2b-GGUF with Docker Model Runner:
```
docker model run hf.co/seyuyo/diktalo-e2b-GGUF:BF16
```

Lemonade

How to use seyuyo/diktalo-e2b-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull seyuyo/diktalo-e2b-GGUF:BF16

Run and chat with the model

lemonade run user.diktalo-e2b-GGUF-BF16

List all available models

lemonade list

Hermes Agent

How to use seyuyo/diktalo-e2b-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf seyuyo/diktalo-e2b-GGUF:BF16

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default seyuyo/diktalo-e2b-GGUF:BF16

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use seyuyo/diktalo-e2b-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf seyuyo/diktalo-e2b-GGUF:BF16

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "seyuyo/diktalo-e2b-GGUF:BF16" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

diktalo-e2b (GGUF)

A small fine-tuned Gemma 4 E2B model that turns Hungarian voice dictation transcripts into structured notes (JSON).

The smaller one. Faster and lighter, meant for weaker phones. Output quality is a bit worse than the E4B version, but for short and medium dictations it works fine.

There is also a E4B version. Code and training data: github.com/seyuyo/diktalo

What it does

You record something on your phone (meeting notes, todo list, shopping list, whatever), Whisper turns it into text, and this model makes a clean note out of it:

{
  "cim": "short title",
  "osszefoglalo": "1-3 sentence summary",
  "pontok": ["main points"],
  "teendok": ["todos, empty list if none"],
  "datumok": ["mentioned dates / deadlines with context"]
}

(The field names are Hungarian: cim = title, osszefoglalo = summary, pontok = points, teendok = todos, datumok = dates.)

It was also trained to deal with typical speech-to-text problems:

filler words (ööö, hát, izé) are dropped
self-corrections like "on Tuesday... I mean Wednesday" are resolved
misheard / nonsense words are not copied over and not made up, the model either fixes them from context or leaves them out

It's a single-task model, not a chatbot. Don't expect it to answer general questions.

Prompt

Important: use exactly this prompt (character for character), with the transcript appended at the end. This is what the model was trained on, if you change it the results get worse.

Az alábbi szöveg egy magyar nyelvű hangfelvétel gépi átirata. Strukturáld az alábbi JSON formátumba. A mezők tartalma nyelvtanilag helyes, természetes magyar legyen. Ha egy szó vagy kifejezés az átiratban nem egyértelmű vagy furcsán hangzik, NE találj ki hozzá jelentést. Ha a szövegkörnyezetből egyértelműen kiderül, mire utal (pl. egy korábban már említett dologra), azt használd; ha nem derül ki, és a szó szerinti átvétel értelmetlen vagy nyelvtanilag hibás mondatot eredményezne, hagyd ki teljesen — soha ne másolj be szó szerint egy zajos/értelmetlen kifejezést csak azért, mert az átiratban szerepelt. Csak a JSON-t add vissza, semmi mást.

{
  "cim": "rövid, tömör cím",
  "osszefoglalo": "1-3 mondatos összefoglaló",
  "pontok": ["a tartalom fő pontjai listában"],
  "teendok": ["konkrét teendők, ha vannak, egyébként üres lista"],
  "datumok": ["említett dátumok/határidők kontextussal, ha vannak"]
}

Átirat:
<your transcript here>

Thinking mode is disabled, the model answers with the JSON right away.

Files

File	Description
`gemma-4-e2b-it.Q4_K_M.gguf`	the model, Q4_K_M quantization (~3.4 GB)
`gemma-4-e2b-it.BF16-mmproj.gguf`	vision/audio projector from the base model, not needed for text only
`Modelfile`	Ollama Modelfile with the Gemma 4 chat template

Running with Ollama

Download the GGUF and the Modelfile into one folder, then:

ollama create diktalo-e2b -f Modelfile
ollama run diktalo-e2b

Running with llama.cpp

llama-cli -m gemma-4-e2b-it.Q4_K_M.gguf --temp 0.3 -c 8192

Training

Base model: unsloth/gemma-4-E2B-it
QLoRA (4-bit) with Unsloth, on a single RTX 5060 Ti 16GB
LoRA r=8, alpha=8, 1 epoch, learning rate 1e-4, only the language layers were tuned
Loss computed only on the assistant answer (the JSON), not on the prompt
500 synthetic examples, generated with DeepSeek: realistic Whisper-style transcripts on random everyday topics (work meetings, doctor visits, car repair, travel planning, etc.) with different speaking styles and some transcription noise

Limitations

Trained only on synthetic data, so real transcripts can still trip it up sometimes
Hungarian only
Very long dictations (5+ minutes) can lose some details
Always check dates and names in the output, it can still get them wrong