Instructions to use webmp3/Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-3bit-352E-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use webmp3/Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-3bit-352E-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf webmp3/Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-3bit-352E-GGUF:IQ3_XXS
# Run inference directly in the terminal:
llama cli -hf webmp3/Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-3bit-352E-GGUF:IQ3_XXS

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf webmp3/Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-3bit-352E-GGUF:IQ3_XXS
# Run inference directly in the terminal:
llama cli -hf webmp3/Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-3bit-352E-GGUF:IQ3_XXS

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf webmp3/Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-3bit-352E-GGUF:IQ3_XXS
# Run inference directly in the terminal:
./build/bin/llama cli -hf webmp3/Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-3bit-352E-GGUF:IQ3_XXS

Use Docker

docker model run hf.co/webmp3/Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-3bit-352E-GGUF:IQ3_XXS

Llama new
LM Studio
Jan

vLLM

How to use webmp3/Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-3bit-352E-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "webmp3/Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-3bit-352E-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "webmp3/Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-3bit-352E-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/webmp3/Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-3bit-352E-GGUF:IQ3_XXS

Ollama
How to use webmp3/Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-3bit-352E-GGUF with Ollama:
```
ollama run hf.co/webmp3/Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-3bit-352E-GGUF:IQ3_XXS
```
Unsloth Desktop

How to use webmp3/Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-3bit-352E-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf webmp3/Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-3bit-352E-GGUF:IQ3_XXS

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "webmp3/Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-3bit-352E-GGUF:IQ3_XXS"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use webmp3/Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-3bit-352E-GGUF with Docker Model Runner:
```
docker model run hf.co/webmp3/Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-3bit-352E-GGUF:IQ3_XXS
```

Lemonade

How to use webmp3/Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-3bit-352E-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull webmp3/Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-3bit-352E-GGUF:IQ3_XXS

Run and chat with the model

lemonade run user.Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-3bit-352E-GGUF-IQ3_XXS

List all available models

lemonade list

Hermes Agent

How to use webmp3/Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-3bit-352E-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf webmp3/Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-3bit-352E-GGUF:IQ3_XXS

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default webmp3/Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-3bit-352E-GGUF:IQ3_XXS

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use webmp3/Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-3bit-352E-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf webmp3/Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-3bit-352E-GGUF:IQ3_XXS

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "webmp3/Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-3bit-352E-GGUF:IQ3_XXS" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Sakura — Qwen3.8-Flash-Next, ISTA 3-bit cut with Darwin-R3 tensors, 352 of 512 experts (experimental)

Experimental: no demonstrated advantage over the unmodified cut

In our own coding tests this build scores the same as the unmodified ISTA K352 IQ3_XXS cut within the noise of single runs, and its held-out German/English perplexity is about 3 % higher than the unmodified cut's (see the tables below). It is published for people who want to try the Darwin-R3 weights on the 352-expert layout, not because it was found to be better.

GPU memory: 30.0 GiB dedicated + 11.2 GiB shared after loading (32k context, q8_0 KV, MTP head), about 41 GiB of the Vulkan pool — not for a 32 GB card

It runs on a 64 GB Strix Halo (unified memory). For a 32 GB card use the 2.5-bit sibling Sakura-Qwen3.8-Flash-Next-Swift-2.5bit-352E-32GB-GGUF.

An experimental GGUF of Qwen3.8-Flash-Next: ISTA-DASLab's 3-bit GSQ-RCO quantization cut to 352 of 512 routed experts per layer (chosen on German + English routing, no re-quantization), with 496 non-expert tensors (attention, shared experts, hyper-connection weights) replaced by those of Darwin-180B-RSI-R3 in Q8_0. Size 58.66 GiB in two shards. Independent community build, not an official Qwen, ISTA-DASLab or FINAL-Bench release. Overview of all our Flash-Next builds: Sakura-Qwen3.8-Flash-Next-Overview.

Part of the Sakura Micro line.

Provenance

Base: Qwen/Qwen3.8-Flash-Next (arch qwen4exp, 48 layers, 512 experts, 10 active).
Quantization and experts: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF, IQ3_XXS tier, expert-sliced (see NOTICE.md; kept experts in kept_experts_K352.json).
Grafted tensors: Darwin-180B-RSI-R3 from FINAL-Bench/POCKET-Darwin-180B-GGUF. According to FINAL-Bench, R3 is a second round of self-training that changes only attention paths and shared experts (LoRA, merged) and leaves the 512 routed experts and the router unchanged; their card reports a small gain on a held-out SuperGPQA set (+1.03 over round 1) and no significant gain on GPQA Diamond (+0.25). We did not reproduce those benchmarks.
Tools: prune_raw_gguf.py (expert slicing) and graft_tensors.py (tensor graft) are in the Overview repository; the list of grafted tensors is grafted_tensors.json.

Files

File	Size
`Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-K352-IQ3_XXS-00001-of-00002.gguf`	31.84 GiB (dense weights, grafted tensors and all kept experts)
`Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-K352-IQ3_XXS-00002-of-00002.gguf`	26.82 GiB (the large per-layer token table, identical to the source file)

SHA256SUMS lists the checksums. Load shard 1; llama.cpp finds shard 2 itself. Use -lm dio --lazy-mode on (or your build's equivalent) so the large token table is read lazily instead of held in memory.

Measured results (own tests; unmodified ISTA K352 IQ3_XXS cut vs this build)

Same runtime for both: Unsloth fork qwen4exp/mtp (commit 6fcaa16, Vulkan, built by us), shared MTP head mtp-Qwen3.8-Flash-Next-shared-Q4_K_M.gguf with n-max 2, -b 2048 -ub 2048, 32k context, KV q8_0, thinking budget 10000, max 12288 tokens, temperature 0.6, top-p 0.95, top-k 20, seeds 1000 + task number, one run each, own frozen task set (LeetCode/AtCoder-style).

Run	passed	not passed (task numbers)	generated tokens	wall time
unmodified ISTA cut, tasks 1-8	8/8	-	40,445	25 min
this build, tasks 1-8	8/8	-	39,524	24 min
unmodified ISTA cut, tasks 9-20	11/12	15	100,341	62 min
this build, tasks 9-20	11/12	10	89,952	53 min
unmodified ISTA cut, tasks 21-32	9/12	21, 23, 27	112,663	68 min
this build, tasks 21-32	10/12	21, 27	105,334	60 min

Task 15 (the one the unmodified cut fails on tasks 9-20) was repeated with two more seeds on both builds: 0 of 3 on each. The one-task differences elsewhere are inside the noise of single runs.

Check	unmodified ISTA cut	this build
Language gate, 14 German + 14 English prompts, share of wrong-language words	DE 0.004 / EN 0.0	DE 0.001 / EN 0.003 (gate: each <= 0.06)
Loops / CJK answers	0 / 0	0 / 0
Held-out perplexity (German mix / English mix)	2.0155 / 1.8087	2.0771 / 1.8559
Decode / prefill speed (AMD Strix Halo, 64 GB, Radeon 8060S, Windows, Vulkan, MTP head)	-	31.4 tok/s decode, 236 tok/s prefill

The language gate and the perplexity were measured with the official llama.cpp build b11259 (Vulkan, no speculation); coding tasks and speed with the fork above. Single runs on one machine; they are indicators, not a benchmark suite. For comparison, the Swift 1.5 fine-tune cut (same architecture and size) reaches about 34.5 tok/s with the same fork and head, also 11/12 on tasks 9-20 and 9/12 on tasks 21-32.

Known limits

No demonstrated advantage (see the box above). The higher precision of the grafted tensors (Q8_0) and Darwin's training cannot be separated in our tests.
The Darwin benchmarks (GPQA, AIME, MMLU-Pro) belong to FINAL-Bench's own models and were not measured by us for this build.
Cutting experts without retraining costs some quality; the size of that cost was checked on German/English text and a small coding set only.
Speculation with the shared MTP head needs a llama.cpp fork; official llama.cpp runs the file without speculation. Vision was not tested (no mmproj is included).
Licenses: the weights inherit the Qwen Community License 1.0, included as LICENSE. Read it before commercial use.

Run example

llama-server -m Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-K352-IQ3_XXS-00001-of-00002.gguf --jinja -fa on -ngl 99 -c 32768 -ctk q8_0 -ctv q8_0 -b 512 -ub 512 -lm dio --lazy-mode on --device Vulkan0

(Official llama.cpp b11259, Vulkan, without speculation. For the MTP head use the fork named above.)

中文说明 · 樱花 (Simplified Chinese)

English above. 本节为上文的中文翻译(Sakura = 樱花 yīnghuā);完整的独立中文版见 README_zh.md。

Sakura — Qwen3.8-Flash-Next,嫁接了 Darwin-R3 张量的 ISTA 3 比特裁剪版,512 个专家中保留 352 个(实验性)

实验性:相对于未修改的裁剪版本,没有证明有优势

在我们自己的编程测试中,这个构建的得分与未修改的 ISTA K352 IQ3_XXS 裁剪版本相同(在单次运行的噪声范围内),而且它在留出的德语/英语上的困惑度比未修改的裁剪版本 高约 3 %(见下表)。发布它是为了让想在 352 专家布局上试用 Darwin-R3 权重的人使用,而不是因为发现它更好。

GPU 内存:加载后 30.0 GiB 专用内存 + 11.2 GiB 共享内存(32k 上下文,q8_0 KV,MTP 头),约占 Vulkan 池的 41 GiB — 不适用于 32 GB 显卡

它可以在 64 GB 的 Strix Halo(统一内存)上运行。对于 32 GB 的显卡,请使用 2.5 比特的同系列版本 Sakura-Qwen3.8-Flash-Next-Swift-2.5bit-352E-32GB-GGUF。

这是 Qwen3.8-Flash-Next 的一个实验性 GGUF:ISTA-DASLab 的 3 比特 GSQ-RCO 量化,裁剪为每层 512 个路由专家中保留 352 个(依据德语 + 英语的路由情况选出,没有重新量化), 并把 496 个非专家张量(注意力、共享专家、超连接权重)替换为 Darwin-180B-RSI-R3 的对应张量(Q8_0)。大小 58.66 GiB,分为两个分片。这是独立的社区构建版本,不是 Qwen、ISTA-DASLab 或 FINAL-Bench 的官方发布。我们所有 Flash-Next 构建的概览:Sakura-Qwen3.8-Flash-Next-Overview。

属于 Sakura Micro 系列。

来源

基础模型:Qwen/Qwen3.8-Flash-Next(架构 qwen4exp,48 层,512 个专家,激活 10 个)。
量化和专家:ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF,IQ3_XXS 档位,已进行专家切片(见 NOTICE.md;保留的专家见 kept_experts_K352.json)。
嫁接的张量:来自 FINAL-Bench/POCKET-Darwin-180B-GGUF 的 Darwin-180B-RSI-R3。据 FINAL-Bench 称,R3 是第二轮自训练,只改变注意力路径和共享专家(LoRA,已合并),512 个路由专家和路由器保持不变;他们的卡片报告了在留出的 SuperGPQA 集上有小幅提升(比第一轮高 +1.03),而在 GPQA Diamond 上没有显著提升(+0.25)。我们没有复现这些基准测试。
工具:prune_raw_gguf.py(专家切片)和 graft_tensors.py(张量嫁接)在 Overview 仓库中;被嫁接张量的列表是 grafted_tensors.json。

文件

文件	大小
`Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-K352-IQ3_XXS-00001-of-00002.gguf`	31.84 GiB(稠密权重、嫁接的张量和所有保留的专家)
`Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-K352-IQ3_XXS-00002-of-00002.gguf`	26.82 GiB(巨大的逐层 token 表,与源文件相同)

SHA256SUMS 列出了校验和。加载分片 1;llama.cpp 会自己找到分片 2。请使用 -lm dio --lazy-mode on(或你所用构建中的等效选项),使巨大的 token 表被懒加载读取,而不是常驻内存。

测量结果(自己的测试;未修改的 ISTA K352 IQ3_XXS 裁剪版本对比本构建)

两者使用相同的运行时:Unsloth 分支 qwen4exp/mtp(提交 6fcaa16,Vulkan,由我们自己构建),共享 MTP 头 mtp-Qwen3.8-Flash-Next-shared-Q4_K_M.gguf,n-max 2,-b 2048 -ub 2048,32k 上下文,KV q8_0,思考预算 10000,最多 12288 个 token,temperature 0.6,top-p 0.95,top-k 20,随机种子 1000 + 任务编号,各运行一次,使用自己的冻结任务集(LeetCode/AtCoder 风格)。

运行	通过	未通过(任务编号)	生成的 token 数	耗时
未修改的 ISTA 裁剪版本,任务 1-8	8/8	-	40,445	25 分钟
本构建,任务 1-8	8/8	-	39,524	24 分钟
未修改的 ISTA 裁剪版本,任务 9-20	11/12	15	100,341	62 分钟
本构建,任务 9-20	11/12	10	89,952	53 分钟
未修改的 ISTA 裁剪版本,任务 21-32	9/12	21, 23, 27	112,663	68 分钟
本构建,任务 21-32	10/12	21, 27	105,334	60 分钟

任务 15(未修改的裁剪版本在任务 9-20 中失败的那一个)在两个构建上又用另外两个随机种子重复了一次:各自都是 3 次中 0 次通过。其他地方相差一个任务的差异处于单次运行的噪声范围内。

检查项	未修改的 ISTA 裁剪版本	本构建
语言关卡,14 个德语 + 14 个英语提示,使用了错误语言的词所占比例	DE 0.004 / EN 0.0	DE 0.001 / EN 0.003(关卡:各 <= 0.06)
循环 / CJK 回答	0 / 0	0 / 0
留出困惑度(德语混合 / 英语混合)	2.0155 / 1.8087	2.0771 / 1.8559
解码 / 预填充速度(AMD Strix Halo,64 GB,Radeon 8060S,Windows,Vulkan,MTP 头)	-	31.4 tok/s 解码,236 tok/s 预填充

语言关卡和困惑度是用官方 llama.cpp 构建 b11259(Vulkan,无推测解码)测得的;编程任务和速度使用上述分支。单次运行,在一台机器上;它们是指示性的,不是基准测试套件。作为对比,Swift 1.5 微调裁剪版本(相同架构和大小)在相同的分支和头下达到约 34.5 tok/s,在任务 9-20 上同样是 11/12,在任务 21-32 上是 9/12。

已知局限

没有证明有优势(见上方的方框)。在我们的测试中,无法把被嫁接张量更高的精度(Q8_0)与 Darwin 的训练区分开。
Darwin 的基准(GPQA、AIME、MMLU-Pro)属于 FINAL-Bench 自己的模型,我们没有为这个构建测量过。
不经再训练就裁剪专家会损失一些质量;这一损失的大小只在德语/英语文本和一个小型编程集上检查过。
与共享 MTP 头一起使用的推测解码需要 llama.cpp 分支;官方 llama.cpp 在没有推测解码的情况下运行该文件。视觉能力未经测试(未包含 mmproj)。
许可证:权重继承 Qwen Community License 1.0,以 LICENSE 随附。商业使用之前请先阅读。

运行示例

llama-server -m Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-K352-IQ3_XXS-00001-of-00002.gguf --jinja -fa on -ngl 99 -c 32768 -ctk q8_0 -ctv q8_0 -b 512 -ub 512 -lm dio --lazy-mode on --device Vulkan0

(官方 llama.cpp b11259,Vulkan,无推测解码。若要使用 MTP 头,请使用上面提到的分支。)

Downloads last month: 142

GGUF

Model size

139B params

Architecture

qwen4exp

Hardware compatibility

3-bit

Model tree for webmp3/Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-3bit-352E-GGUF

FINAL-Bench/Darwin-180B-RSI-R3

ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF

Merge model

this model

Collection including webmp3/Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-3bit-352E-GGUF

Sakura Micro

Collection

Sakura Micro: compact derivatives of large models for one local machine (e.g. 64 GB Strix Halo), with measured quality and speed. • 9 items • Updated Oct 5

Sakura — Qwen3.8-Flash-Next, ISTA 3-bit cut with Darwin-R3 tensors, 352 of 512 experts (experimental)

Experimental: no demonstrated advantage over the unmodified cut

GPU memory: 30.0 GiB dedicated + 11.2 GiB shared after loading (32k context, q8_0 KV, MTP head), about 41 GiB of the Vulkan pool — not for a 32 GB card

Provenance

Files

Measured results (own tests; unmodified ISTA K352 IQ3_XXS cut vs this build)

Known limits

Run example

Sakura — Qwen3.8-Flash-Next,嫁接了 Darwin-R3 张量的 ISTA 3 比特裁剪版,512 个专家中保留 352 个(实验性)

实验性:相对于未修改的裁剪版本,没有证明有优势

GPU 内存:加载后 30.0 GiB 专用内存 + 11.2 GiB 共享内存(32k 上下文,q8_0 KV,MTP 头),约占 Vulkan 池的 41 GiB — 不适用于 32 GB 显卡

来源

文件

测量结果(自己的测试;未修改的 ISTA K352 IQ3_XXS 裁剪版本 对比 本构建)

已知局限

运行示例

Model tree for webmp3/Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-3bit-352E-GGUF

Collection including webmp3/Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-3bit-352E-GGUF

测量结果(自己的测试;未修改的 ISTA K352 IQ3_XXS 裁剪版本对比本构建)