Instructions to use webmp3/Sakura-Qwen3.8-Flash-Next-Swift-3bit-352E-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use webmp3/Sakura-Qwen3.8-Flash-Next-Swift-3bit-352E-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf webmp3/Sakura-Qwen3.8-Flash-Next-Swift-3bit-352E-GGUF:IQ3_XXS # Run inference directly in the terminal: llama cli -hf webmp3/Sakura-Qwen3.8-Flash-Next-Swift-3bit-352E-GGUF:IQ3_XXS
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf webmp3/Sakura-Qwen3.8-Flash-Next-Swift-3bit-352E-GGUF:IQ3_XXS # Run inference directly in the terminal: llama cli -hf webmp3/Sakura-Qwen3.8-Flash-Next-Swift-3bit-352E-GGUF:IQ3_XXS
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf webmp3/Sakura-Qwen3.8-Flash-Next-Swift-3bit-352E-GGUF:IQ3_XXS # Run inference directly in the terminal: ./build/bin/llama cli -hf webmp3/Sakura-Qwen3.8-Flash-Next-Swift-3bit-352E-GGUF:IQ3_XXS
Use Docker
docker model run hf.co/webmp3/Sakura-Qwen3.8-Flash-Next-Swift-3bit-352E-GGUF:IQ3_XXS
- Llama new
- LM Studio
- Jan
- vLLM
How to use webmp3/Sakura-Qwen3.8-Flash-Next-Swift-3bit-352E-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "webmp3/Sakura-Qwen3.8-Flash-Next-Swift-3bit-352E-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "webmp3/Sakura-Qwen3.8-Flash-Next-Swift-3bit-352E-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/webmp3/Sakura-Qwen3.8-Flash-Next-Swift-3bit-352E-GGUF:IQ3_XXS
- Ollama
How to use webmp3/Sakura-Qwen3.8-Flash-Next-Swift-3bit-352E-GGUF with Ollama:
ollama run hf.co/webmp3/Sakura-Qwen3.8-Flash-Next-Swift-3bit-352E-GGUF:IQ3_XXS
- Unsloth Desktop
- Pi
How to use webmp3/Sakura-Qwen3.8-Flash-Next-Swift-3bit-352E-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf webmp3/Sakura-Qwen3.8-Flash-Next-Swift-3bit-352E-GGUF:IQ3_XXS
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "webmp3/Sakura-Qwen3.8-Flash-Next-Swift-3bit-352E-GGUF:IQ3_XXS" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use webmp3/Sakura-Qwen3.8-Flash-Next-Swift-3bit-352E-GGUF with Docker Model Runner:
docker model run hf.co/webmp3/Sakura-Qwen3.8-Flash-Next-Swift-3bit-352E-GGUF:IQ3_XXS
- Lemonade
How to use webmp3/Sakura-Qwen3.8-Flash-Next-Swift-3bit-352E-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull webmp3/Sakura-Qwen3.8-Flash-Next-Swift-3bit-352E-GGUF:IQ3_XXS
Run and chat with the model
lemonade run user.Sakura-Qwen3.8-Flash-Next-Swift-3bit-352E-GGUF-IQ3_XXS
List all available models
lemonade list
- Hermes Agent
How to use webmp3/Sakura-Qwen3.8-Flash-Next-Swift-3bit-352E-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf webmp3/Sakura-Qwen3.8-Flash-Next-Swift-3bit-352E-GGUF:IQ3_XXS
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default webmp3/Sakura-Qwen3.8-Flash-Next-Swift-3bit-352E-GGUF:IQ3_XXS
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use webmp3/Sakura-Qwen3.8-Flash-Next-Swift-3bit-352E-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf webmp3/Sakura-Qwen3.8-Flash-Next-Swift-3bit-352E-GGUF:IQ3_XXS
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "webmp3/Sakura-Qwen3.8-Flash-Next-Swift-3bit-352E-GGUF:IQ3_XXS" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
- Sakura โ Qwen3.8-Flash-Next (Swift 1.5), 352 of 512 experts, IQ3_XXS
- Sakura โ Qwen3.8-Flash-Next (Swift 1.5),512 ไธชไธๅฎถไธญไฟ็ 352 ไธช,IQ3_XXS
Sakura โ Qwen3.8-Flash-Next (Swift 1.5), 352 of 512 experts, IQ3_XXS
GPU memory: about 31.4 GiB of weights โ too big for a 32 GB card
This 3-bit build keeps about 31.4 GiB of weights on the GPU (the 26.8 GiB per-layer token table stays on disk/RAM via lazy loading). It runs on a 64 GB Strix Halo (unified memory: in our test 30.4 GiB dedicated plus 10.1 GiB shared memory with 32k context, q8_0 KV cache and the MTP head), but it does not fit a 32 GB GPU. For a 32 GB card use the 2.5-bit sibling Sakura-Qwen3.8-Flash-Next-Swift-2.5bit-352E-32GB-GGUF (about 26 GiB of GPU weights).
An experimental expert-pruned GGUF of UkisAI's Swift 1.5 fine-tune of Qwen3.8-Flash-Next: 352 of 512 routed experts per layer kept (chosen on German + English routing), the rest cut out of the ~3-bit (IQ3_XXS, GSQ-RCO) quantization without re-quantizing. Size 58.19 GiB in two shards. Made for a single 64 GB AMD Strix Halo (Radeon 8060S); German and English are the target languages. Independent community build, not an official Qwen or UkisAI release.
Part of the Sakura Micro line.
All Sakura Flash-Next builds side by side (what differs, measured table, runtime notes, tools): Sakura-Qwen3.8-Flash-Next-Overview.
Provenance
- Base: Qwen/Qwen3.8-Flash-Next (arch
qwen4exp, 48 layers, 512 experts, 10 active). - Fine-tune + quantization: ukisai/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-GGUF (IQ3_XXS tier). Swift is the reason for this build: it uses clearly fewer thinking tokens than the base model.
- What this build changes: expert slicing only (see
NOTICE.md). Kept experts per layer:kept_experts_K352.json. - Expert selection: router statistics of the Swift model itself on a German + English calibration mix (routing mass); an expert is kept if it is hot in
either language. Measured with
llama-expert-stats, a small tool that runs the model and counts router use (much faster than llama-imatrix).
Files
| File | Size |
|---|---|
Sakura-Qwen3.8-Flash-Next-Swift-K352-IQ3_XXS-00001-of-00002.gguf |
34.32 GiB (dense weights, the large per-layer token table and part of the experts) |
Sakura-Qwen3.8-Flash-Next-Swift-K352-IQ3_XXS-00002-of-00002.gguf |
23.87 GiB (rest of the routed experts) |
SHA256SUMS lists the checksums. Load shard 1; llama.cpp finds shard 2 itself. Use -lm dio --lazy-mode on (or your build's equivalent) so the large token table is
read lazily instead of held in memory.
Measured results (own tests, this exact file)
| Check | Result |
|---|---|
| Language gate, 14 German + 14 English prompts: share of wrong-language words | DE 0.000 / EN 0.005 (gate: each <= 0.06) |
| Loops / CJK answers | 0 / 0 |
| Held-out perplexity (German mix / English mix) | 2.0514 / 1.8231 (details below) |
| Decode / prefill speed | 34.5 tok/s decode, 314 tok/s prefill with speculation (Vulkan, see the speed table) |
| Coding tasks (own 8 LeetCode/AtCoder-style tasks) | 8/8 passed (14,443 generated tokens, about 8.6 min) |
| Tasks 9-20 of our frozen 32-task set (harder tasks) | 11/12 passed (98,937 generated tokens), see below |
The language gate and the perplexity were measured with the official llama.cpp build b11259 (Vulkan); speed and coding tasks with the fork build described below.
Held-out perplexity raw
{"de": 2.0514, "en": 1.8231}
llama-perplexity, context 512, 40 chunks of own German and English held-out texts (not in the calibration data). For orientation, same texts: Swift 512-expert IQ2_XS 2.0605 / 1.8245, Swift K352 IQ2_XS 2.0681 / 1.8229, the same recipe cut to K320 (IQ3_XXS) 2.0437 / 1.8284. The differences between these builds span about 1.2 % (German); perplexity cannot separate them. (No perplexity of the unpruned IQ3_XXS: the 76 GB file does not fit the test machine.)
Runtime used for speed and coding tests
The shared MTP head needs a llama.cpp fork. We used github.com/danielhanchen/llama.cpp, branch qwen4exp/mtp, commit 6fcaa16 (21 Sep 2026), built by us for Vulkan
(MSVC, Vulkan SDK 1.4.357: cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DGGML_VULKAN=ON -DBUILD_SHARED_LIBS=ON -DLLAMA_OPENSSL=OFF -DLLAMA_CURL=OFF).
Speed (AMD Strix Halo, 64 GB, Radeon 8060S, Windows, Vulkan)
Same ~1700-token prompt each time, 32k context, KV q8_0, -lm dio --lazy-mode on, shared MTP sidecar mtp-Qwen3.8-Flash-Next-shared-Q4_K_M.gguf (1.78 GiB):
-b/-ub |
Speculation | Prefill | Decode |
|---|---|---|---|
| 2048 | MTP n-max 2 | 314.5 tok/s | 34.5 tok/s |
| 2048 | MTP n-max 3 | 316.7 tok/s | 33.6 tok/s |
| 512 | MTP n-max 3 | 257.5 tok/s | 33.4 tok/s |
| 512 | MTP n-max 2 | 253.4 tok/s | 32.8 tok/s |
Without speculation in the official llama.cpp b11259 (Vulkan) the same file decodes at about 21 tok/s (measured during a coding task). ROCm is not recommended for this file: on Windows the decode collapses to about 5 tok/s as soon as the resident weights (file size minus the token table, here about 31 GiB) exceed the dedicated GPU memory; smaller cuts (K304 and below) stay fast on ROCm but were not evaluated for quality.
Coding check (own tasks, thinking budget 10000, max 12288 tokens, temperature 0.6, top-p 0.95, top-k 20, seeds 1000 + task number, one run each)
- Eight anchor tasks: 8/8 passed, 30 to 34 tok/s decode, including the hardest one of the set (3,664 tokens).
- Tasks 9 to 20 of the frozen 32-task set (the harder part; same questions, grader and seeds for every model we test): 11 of 12 passed. Task 15 failed (a hidden test returned 12 instead of 8). For orientation, in our earlier runs a dense Swift 1.5 27B Q5_K_M solved 6 of these 12 tasks and a 35B-A3B coder 8 of 12. These are not like-for-like numbers: the 27B ran without a thinking budget, this build with a budget of 10,000 tokens, and every task was run once. Task 21 of the 32-task set was not run.
Single runs on one machine; they are indicators, not a benchmark suite. Vision was not tested (no mmproj is included; use the BF16 projector from the Swift repository).
Known limits
- Cutting experts without retraining costs some quality; the size of that cost was checked on German/English text and a small coding set only.
- Speculation needs the fork above. The shared MTP sidecar is not part of this GGUF and does not load in official llama.cpp (b11259 and b11294 fail with
token_embd.weight not found; the upstream loader has no MTP-only path). Unsloth publishes the shared sidecar asMTP/mtp-Qwen3.8-Flash-Next-shared-Q8_0.ggufinunsloth/Qwen3.8-Flash-Next-GGUF; we measured with a Q4_K_M sidecar and did not test the Q8_0 one. Without speculation the file works in official llama.cpp. - An earlier Unsloth Vulkan package (build 10830, commit
0cd7a79f5) gave wrong results with this file in our tests: after the thinking budget ran out, the model kept reasoning in the answer part instead of writing the code (4 of 4 variants failed the same task that the fork and the official b11259 solved). We do not recommend that build for this model. - Licenses: Swift Open License v1.0 (commercial-use threshold, section 5) and Qwen Community License 1.0 (Model-as-a-Service / AI Work Assistant clause). Read both before commercial use.
Run example
llama-server -m Sakura-Qwen3.8-Flash-Next-Swift-K352-IQ3_XXS-00001-of-00002.gguf --jinja -fa on -ngl 99 -c 32768 -ctk q8_0 -ctv q8_0 -b 2048 -ub 2048 -lm dio --lazy-mode on --device Vulkan0 \
--spec-type draft-mtp --spec-draft-model mtp-Qwen3.8-Flash-Next-shared-Q4_K_M.gguf --spec-draft-ngl 99 --spec-draft-n-max 2 --spec-draft-n-min 0
This is the command line used for the speed numbers with the fork build. Leave out the last line to run without speculation.
ไธญๆ่ฏดๆ ยท ๆจฑ่ฑ (Simplified Chinese)
English above. ๆฌ่ไธบไธๆ็ไธญๆ็ฟป่ฏ(Sakura = ๆจฑ่ฑ yฤซnghuฤ);ๅฎๆด็็ฌ็ซไธญๆ็่ง README_zh.mdใ
Sakura โ Qwen3.8-Flash-Next (Swift 1.5),512 ไธชไธๅฎถไธญไฟ็ 352 ไธช,IQ3_XXS
GPU ๅ ๅญ:ๆ้็บฆ 31.4 GiB โ ๅฏน 32 GB ๆพๅกๆฅ่ฏดๅคชๅคง
่ฟไธช 3 ๆฏ็นๆๅปบๅจ GPU ไธไฟ็็บฆ 31.4 GiB ็ๆ้(26.8 GiB ็้ๅฑ token ่กจ้่ฟๆๅ ่ฝฝ็ๅจ็ฃ็/ๅ ๅญไธญ)ใๅฎๅฏไปฅๅจ 64 GB ็ Strix Halo(็ปไธๅ ๅญ:ๅจๆไปฌ็ๆต่ฏไธญ,32k ไธไธๆใq8_0 KV ็ผๅญๅ MTP ๅคด็ๆ ๅตไธๅ ็จ 30.4 GiB ไธ็จๅ ๅญๅ 10.1 GiB ๅ ฑไบซๅ ๅญ)ไธ่ฟ่ก,ไฝ ไธ่ฝ ๆพ่ฟ 32 GB ็ GPUใๅฏนไบ 32 GB ็ๆพๅก,่ฏทไฝฟ็จ 2.5 ๆฏ็น็ๅ็ณปๅ็ๆฌ Sakura-Qwen3.8-Flash-Next-Swift-2.5bit-352E-32GB-GGUF(GPU ๆ้็บฆ 26 GiB)ใ
่ฟๆฏ UkisAI ๅฏน Qwen3.8-Flash-Next ็ Swift 1.5 ๅพฎ่ฐ็ๆฌ็ไธไธชๅฎ้ชๆงไธๅฎถๅชๆ GGUF:ๆฏๅฑ 512 ไธช่ทฏ็ฑไธๅฎถไธญไฟ็ 352 ไธช (ไพๆฎๅพท่ฏญ + ่ฑ่ฏญ็่ทฏ็ฑๆ ๅต้ๅบ),ๅ ถไฝ็ไป็บฆ 3 ๆฏ็น(IQ3_XXS,GSQ-RCO)้ๅไธญ่ขซ่ฃๆ,ๆฒกๆ้ๆฐ้ๅใๅคงๅฐ 58.19 GiB, ๅไธบไธคไธชๅ็ใไธบๅๅฐ 64 GB ็ AMD Strix Halo(Radeon 8060S)ๅถไฝ;็ฎๆ ่ฏญ่จๆฏๅพท่ฏญๅ่ฑ่ฏญใ่ฟๆฏ็ฌ็ซ็็คพๅบๆๅปบ็ๆฌ, ไธๆฏ Qwen ๆ UkisAI ็ๅฎๆนๅๅธใ
ๅฑไบ Sakura Micro ็ณปๅใ
ๆๆ Sakura Flash-Next ๆๅปบ็ๅนถๆๆฏ่พ(ๅบๅซใๅฎๆต่กจๆ ผใ่ฟ่กๆถ่ฏดๆใๅทฅๅ ท):Sakura-Qwen3.8-Flash-Next-Overviewใ
ๆฅๆบ
- ๅบ็กๆจกๅ:Qwen/Qwen3.8-Flash-Next(ๆถๆ
qwen4exp,48 ๅฑ,512 ไธชไธๅฎถ,ๆฟๆดป 10 ไธช)ใ - ๅพฎ่ฐ + ้ๅ:ukisai/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-GGUF(IQ3_XXS ๆกฃไฝ)ใ Swift ๆฏๅถไฝ่ฟไธชๆๅปบ็ๅๅ :ๅฎไฝฟ็จ็ๆ่ token ๆๆพๅฐไบๅบ็กๆจกๅใ
- ๆฌๆๅปบ็ๆนๅจ:ไป
ไธบไธๅฎถๅ็(่ง
NOTICE.md)ใๆฏๅฑไฟ็็ไธๅฎถ:kept_experts_K352.jsonใ - ไธๅฎถ้ๆฉ:Swift ๆจกๅ่ช่บซๅจๅพท่ฏญ + ่ฑ่ฏญๆ กๅๆททๅๆๆฌไธ็่ทฏ็ฑๅจ็ป่ฎก(่ทฏ็ฑ่ดจ้);ๅช่ฆไธไธชไธๅฎถๅจไปปไธ่ฏญ่จไธญๆฏ็ญ็น,ๅฐฑไบไปฅไฟ็ใไฝฟ็จ
llama-expert-statsๆต้,่ฟๆฏไธไธช่ฟ่กๆจกๅๅนถ็ป่ฎก่ทฏ็ฑๅจไฝฟ็จๆ ๅต็ๅฐๅทฅๅ ท(ๆฏ llama-imatrix ๅฟซๅพๅค)ใ
ๆไปถ
| ๆไปถ | ๅคงๅฐ |
|---|---|
Sakura-Qwen3.8-Flash-Next-Swift-K352-IQ3_XXS-00001-of-00002.gguf |
34.32 GiB(็จ ๅฏๆ้ใๅทจๅคง็้ๅฑ token ่กจๅไธ้จๅไธๅฎถ) |
Sakura-Qwen3.8-Flash-Next-Swift-K352-IQ3_XXS-00002-of-00002.gguf |
23.87 GiB(ๅ ถไฝ็่ทฏ็ฑไธๅฎถ) |
SHA256SUMS ๅๅบไบๆ ก้ชๅใๅ ่ฝฝๅ็ 1;llama.cpp ไผ่ชๅทฑๆพๅฐๅ็ 2ใ่ฏทไฝฟ็จ -lm dio --lazy-mode on(ๆไฝ ๆ็จๆๅปบไธญ็็ญๆ้้กน),ไฝฟๅทจๅคง็ token ่กจ่ขซๆๅ ่ฝฝ่ฏปๅ,่ไธๆฏๅธธ้ฉปๅ
ๅญใ
ๆต้็ปๆ(่ชๅทฑ็ๆต่ฏ,่ฟไธ็กฎๅ็ๆไปถ)
| ๆฃๆฅ้กน | ็ปๆ |
|---|---|
| ่ฏญ่จๅ ณๅก,14 ไธชๅพท่ฏญ + 14 ไธช่ฑ่ฏญๆ็คบ:ไฝฟ็จไบ้่ฏฏ่ฏญ่จ็่ฏๆๅ ๆฏไพ | DE 0.000 / EN 0.005(ๅ ณๅก:ๅ <= 0.06) |
| ๅพช็ฏ / CJK ๅ็ญ | 0 / 0 |
| ็ๅบๅฐๆๅบฆ(ๅพท่ฏญๆททๅ / ่ฑ่ฏญๆททๅ) | 2.0514 / 1.8231(่ฏฆ่งไธๆ) |
| ่งฃ็ / ้ขๅกซๅ ้ๅบฆ | ไฝฟ็จๆจๆต่งฃ็ ๆถ 34.5 tok/s ่งฃ็ ,314 tok/s ้ขๅกซๅ (Vulkan,่ง้ๅบฆ่กจ) |
| ็ผ็จไปปๅก(่ชๅทฑ็ 8 ไธช LeetCode/AtCoder ้ฃๆ ผไปปๅก) | 8/8 ้่ฟ(็ๆ 14,443 ไธช token,็บฆ 8.6 ๅ้) |
| ๆไปฌๅป็ป็ 32 ไธชไปปๅก้ไธญ็ไปปๅก 9-20(ๆด้พ็ไปปๅก) | 11/12 ้่ฟ(็ๆ 98,937 ไธช token),่งไธๆ |
่ฏญ่จๅ ณๅกๅๅฐๆๅบฆๆฏ็จๅฎๆน llama.cpp ๆๅปบ b11259(Vulkan)ๆตๅพ็;้ๅบฆๅ็ผ็จไปปๅกไฝฟ็จไธๆๆ่ฟฐ็ๅๆฏๆๅปบใ
็ๅบๅฐๆๅบฆๅๅงๅผ
{"de": 2.0514, "en": 1.8231}
llama-perplexity,ไธไธๆ 512,ๅจ่ชๅทฑ็ๅพท่ฏญๅ่ฑ่ฏญ็ๅบๆๆฌ(ไธๅจๆ กๅๆฐๆฎไธญ)ไธๅ 40 ไธชๅใไฝไธบๅ่,ๅจ็ธๅๆๆฌไธ:Swift 512 ไธๅฎถ IQ2_XS ไธบ 2.0605 / 1.8245,Swift K352 IQ2_XS ไธบ 2.0681 / 1.8229,็ธๅ้ ๆน่ฃๅชๅฐ K320(IQ3_XXS)ไธบ 2.0437 / 1.8284ใ่ฟไบๆๅปบไน้ด็ๅทฎๅผ็บฆไธบ 1.2 %(ๅพท่ฏญ); ๅฐๆๅบฆๆ ๆณๅฐๅฎไปฌๅบๅๅผใ(ๆฒกๆๆชๅชๆ็ IQ3_XXS ็ๅฐๆๅบฆ:76 GB ็ๆไปถๆพไธ่ฟๆต่ฏๆบๅจใ)
็จไบ้ๅบฆๅ็ผ็จๆต่ฏ็่ฟ่กๆถ
ๅ
ฑไบซ MTP ๅคด้่ฆ llama.cpp ๅๆฏใๆไปฌไฝฟ็จไบ github.com/danielhanchen/llama.cpp,ๅๆฏ qwen4exp/mtp,ๆไบค 6fcaa16(2026 ๅนด 9 ๆ 21 ๆฅ),็ฑๆไปฌ่ชๅทฑไธบ Vulkan ๆๅปบ
(MSVC,Vulkan SDK 1.4.357:cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DGGML_VULKAN=ON -DBUILD_SHARED_LIBS=ON -DLLAMA_OPENSSL=OFF -DLLAMA_CURL=OFF)ใ
้ๅบฆ(AMD Strix Halo,64 GB,Radeon 8060S,Windows,Vulkan)
ๆฏๆฌกไฝฟ็จ็ธๅ็็บฆ 1700 ไธช token ็ๆ็คบ,32k ไธไธๆ,KV q8_0,-lm dio --lazy-mode on,ๅ
ฑไบซ MTP ๆ่ทฏๆไปถ mtp-Qwen3.8-Flash-Next-shared-Q4_K_M.gguf(1.78 GiB):
-b/-ub |
ๆจๆต่งฃ็ | ้ขๅกซๅ | ่งฃ็ |
|---|---|---|---|
| 2048 | MTP n-max 2 | 314.5 tok/s | 34.5 tok/s |
| 2048 | MTP n-max 3 | 316.7 tok/s | 33.6 tok/s |
| 512 | MTP n-max 3 | 257.5 tok/s | 33.4 tok/s |
| 512 | MTP n-max 2 | 253.4 tok/s | 32.8 tok/s |
ๅจๅฎๆน llama.cpp b11259(Vulkan)ไธญไธไฝฟ็จๆจๆต่งฃ็ ๆถ,ๅไธไธชๆไปถ็่งฃ็ ้ๅบฆ็บฆไธบ 21 tok/s(ๅจไธๆฌก็ผ็จไปปๅกไธญๆตๅพ)ใ ไธๆจ่ๅจๆญคๆไปถไธไฝฟ็จ ROCm: ๅจ Windows ไธ,ไธๆฆๅธธ้ฉปๆ้(ๆไปถๅคงๅฐๅๅป token ่กจ,่ฟ้็บฆไธบ 31 GiB)่ถ ่ฟไธ็จ GPU ๅ ๅญ,่งฃ็ ้ๅบฆๅฐฑไผ้ชค้ๅฐ็บฆ 5 tok/s;่พๅฐ็่ฃๅช(K304 ๅไปฅไธ)ๅจ ROCm ไธไป็ถๅพๅฟซ,ไฝๆฒกๆ่ฏไผฐๅ ถ่ดจ้ใ
็ผ็จๆฃๆฅ(่ชๅทฑ็ไปปๅก,ๆ่้ข็ฎ 10000,ๆๅค 12288 ไธช token,temperature 0.6,top-p 0.95,top-k 20,้ๆบ็งๅญ 1000 + ไปปๅก็ผๅท,ๅ่ฟ่กไธๆฌก)
- ๅ ซไธช้็นไปปๅก:8/8 ้่ฟ,่งฃ็ 30 ่ณ 34 tok/s,ๅ ๆฌ่ฟไธ็ปไธญๆ้พ็ไธไธช(3,664 ไธช token)ใ
- ๅป็ป็ 32 ไธชไปปๅก้ไธญ็ไปปๅก 9 ่ณ 20(ๆด้พ็้จๅ;ๆไปฌๆต่ฏ็ๆฏไธชๆจกๅ้ฝไฝฟ็จ็ธๅ็้ฎ้ขใ่ฏๅๅจๅ้ๆบ็งๅญ):12 ไธชไธญ้่ฟ 11 ไธชใไปปๅก 15 ๅคฑ่ดฅ(ไธไธช้่ๆต่ฏ่ฟๅไบ 12 ่ไธๆฏ 8)ใ ไฝไธบๅ่,ๅจๆไปฌ่พๆฉ็่ฟ่กไธญ,็จ ๅฏ็ Swift 1.5 27B Q5_K_M ่งฃๅบไบ่ฟ 12 ไธชไปปๅกไธญ็ 6 ไธช,ไธไธช 35B-A3B ็ผ็จๆจกๅ่งฃๅบไบ 12 ไธชไธญ็ 8 ไธชใ่ฟไบไธๆฏๅๅฃๅพ็ๆฐๆฎ:27B ๆฏๅจๆฒกๆๆ่้ข็ฎ็ๆ ๅตไธ่ฟ่ก็,่่ฟไธชๆๅปบ็้ข็ฎๆฏ 10,000 ไธช token,ๅนถไธๆฏไธชไปปๅกๅช่ฟ่กไบไธๆฌกใ32 ไธชไปปๅก้ไธญ็ไปปๅก 21 ๆฒกๆ่ฟ่กใ
ๅๆฌก่ฟ่ก,ๅจไธๅฐๆบๅจไธ;ๅฎไปฌๆฏๆ็คบๆง็,ไธๆฏๅบๅๆต่ฏๅฅไปถใ่ง่ง่ฝๅๆช็ปๆต่ฏ(ๆชๅ ๅซ mmproj;่ฏทไฝฟ็จ Swift ไปๅบไธญ็ BF16 ๆๅฝฑๅจ)ใ
ๅทฒ็ฅๅฑ้
- ไธ็ปๅ่ฎญ็ปๅฐฑ่ฃๅชไธๅฎถไผๆๅคฑไธไบ่ดจ้;่ฟไธๆๅคฑ็ๅคงๅฐๅชๅจๅพท่ฏญ/่ฑ่ฏญๆๆฌๅไธไธชๅฐๅ็ผ็จ้ไธๆฃๆฅ่ฟใ
- ๆจๆต่งฃ็ ้่ฆไธ่ฟฐๅๆฏใ ๅ
ฑไบซ MTP ๆ่ทฏๆไปถไธๅฑไบ่ฟไธช GGUF,ๅจๅฎๆน llama.cpp ไธญๆ ๆณๅ ่ฝฝ(b11259 ๅ b11294 ไผไปฅ
token_embd.weight not foundๅคฑ่ดฅ;ไธๆธธๅ ่ฝฝๅจๆฒกๆไป ๅซ MTP ็่ทฏๅพ)ใUnsloth ๅจunsloth/Qwen3.8-Flash-Next-GGUFไธญๆๅ ฑไบซๆ่ทฏๆไปถๅๅธไธบMTP/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf;ๆไปฌไฝฟ็จ Q4_K_M ๆ่ทฏๆไปถๆต้,ๆฒกๆๆต่ฏ Q8_0 ็้ฃไธชใไธไฝฟ็จๆจๆต่งฃ็ ๆถ,่ฏฅๆไปถๅฏๅจๅฎๆน llama.cpp ไธญ่ฟ่กใ - ่พๆฉ็ Unsloth Vulkan ๅ
(ๆๅปบ 10830,ๆไบค
0cd7a79f5)ๅจๆไปฌ็ๆต่ฏไธญๅฏนๆญคๆไปถ็ปๅบไบ้่ฏฏ็็ปๆ: ๅจๆ่้ข็ฎ็จๅฎไนๅ,ๆจกๅไปๅจๅ็ญ้จๅ็ปง็ปญๆจ็,่ๆฒกๆๅๅบไปฃ็ (4 ไธชๅไฝไธญๆ 4 ไธชๅจๅไธไธชไปปๅกไธๅคฑ่ดฅ,่่ฏฅไปปๅกๅๆฏๅๅฎๆน b11259 ้ฝ่งฃๅบไบ)ใๆไปฌไธๆจ่ๆ่ฟไธชๆๅปบ็จไบ่ฏฅๆจกๅใ - ่ฎธๅฏ่ฏ:Swift Open License v1.0(ๅไธไฝฟ็จ้จๆง,็ฌฌ 5 ่) ไปฅๅ Qwen Community License 1.0(Model-as-a-Service / AI Work Assistant ๆกๆฌพ)ใๅไธไฝฟ็จไนๅ่ฏท้ ่ฏปไธค่ ใ
่ฟ่ก็คบไพ
llama-server -m Sakura-Qwen3.8-Flash-Next-Swift-K352-IQ3_XXS-00001-of-00002.gguf --jinja -fa on -ngl 99 -c 32768 -ctk q8_0 -ctv q8_0 -b 2048 -ub 2048 -lm dio --lazy-mode on --device Vulkan0 \
--spec-type draft-mtp --spec-draft-model mtp-Qwen3.8-Flash-Next-shared-Q4_K_M.gguf --spec-draft-ngl 99 --spec-draft-n-max 2 --spec-draft-n-min 0
่ฟๆฏไฝฟ็จๅๆฏๆๅปบๅพๅบ้ๅบฆๆฐๆฎๆถๆ็จ็ๅฝไปค่กใ็็ฅๆๅไธ่กๅณๅฏๅจไธไฝฟ็จๆจๆต่งฃ็ ็ๆ ๅตไธ่ฟ่กใ
- Downloads last month
- 291
3-bit
Model tree for webmp3/Sakura-Qwen3.8-Flash-Next-Swift-3bit-352E-GGUF
Base model
Qwen/Qwen3.8-Flash-Next