Sakura โ€” Qwen3.8-Flash-Next (Swift 1.5), 352 of 512 experts, IQ3_XXS

Sakura logo

GPU memory: about 31.4 GiB of weights โ€” too big for a 32 GB card

This 3-bit build keeps about 31.4 GiB of weights on the GPU (the 26.8 GiB per-layer token table stays on disk/RAM via lazy loading). It runs on a 64 GB Strix Halo (unified memory: in our test 30.4 GiB dedicated plus 10.1 GiB shared memory with 32k context, q8_0 KV cache and the MTP head), but it does not fit a 32 GB GPU. For a 32 GB card use the 2.5-bit sibling Sakura-Qwen3.8-Flash-Next-Swift-2.5bit-352E-32GB-GGUF (about 26 GiB of GPU weights).

An experimental expert-pruned GGUF of UkisAI's Swift 1.5 fine-tune of Qwen3.8-Flash-Next: 352 of 512 routed experts per layer kept (chosen on German + English routing), the rest cut out of the ~3-bit (IQ3_XXS, GSQ-RCO) quantization without re-quantizing. Size 58.19 GiB in two shards. Made for a single 64 GB AMD Strix Halo (Radeon 8060S); German and English are the target languages. Independent community build, not an official Qwen or UkisAI release.

Part of the Sakura Micro line.

All Sakura Flash-Next builds side by side (what differs, measured table, runtime notes, tools): Sakura-Qwen3.8-Flash-Next-Overview.

Provenance

  • Base: Qwen/Qwen3.8-Flash-Next (arch qwen4exp, 48 layers, 512 experts, 10 active).
  • Fine-tune + quantization: ukisai/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-GGUF (IQ3_XXS tier). Swift is the reason for this build: it uses clearly fewer thinking tokens than the base model.
  • What this build changes: expert slicing only (see NOTICE.md). Kept experts per layer: kept_experts_K352.json.
  • Expert selection: router statistics of the Swift model itself on a German + English calibration mix (routing mass); an expert is kept if it is hot in either language. Measured with llama-expert-stats, a small tool that runs the model and counts router use (much faster than llama-imatrix).

Files

File Size
Sakura-Qwen3.8-Flash-Next-Swift-K352-IQ3_XXS-00001-of-00002.gguf 34.32 GiB (dense weights, the large per-layer token table and part of the experts)
Sakura-Qwen3.8-Flash-Next-Swift-K352-IQ3_XXS-00002-of-00002.gguf 23.87 GiB (rest of the routed experts)

SHA256SUMS lists the checksums. Load shard 1; llama.cpp finds shard 2 itself. Use -lm dio --lazy-mode on (or your build's equivalent) so the large token table is read lazily instead of held in memory.

Measured results (own tests, this exact file)

Check Result
Language gate, 14 German + 14 English prompts: share of wrong-language words DE 0.000 / EN 0.005 (gate: each <= 0.06)
Loops / CJK answers 0 / 0
Held-out perplexity (German mix / English mix) 2.0514 / 1.8231 (details below)
Decode / prefill speed 34.5 tok/s decode, 314 tok/s prefill with speculation (Vulkan, see the speed table)
Coding tasks (own 8 LeetCode/AtCoder-style tasks) 8/8 passed (14,443 generated tokens, about 8.6 min)
Tasks 9-20 of our frozen 32-task set (harder tasks) 11/12 passed (98,937 generated tokens), see below

The language gate and the perplexity were measured with the official llama.cpp build b11259 (Vulkan); speed and coding tasks with the fork build described below.

Held-out perplexity raw
{"de": 2.0514, "en": 1.8231}

llama-perplexity, context 512, 40 chunks of own German and English held-out texts (not in the calibration data). For orientation, same texts: Swift 512-expert IQ2_XS 2.0605 / 1.8245, Swift K352 IQ2_XS 2.0681 / 1.8229, the same recipe cut to K320 (IQ3_XXS) 2.0437 / 1.8284. The differences between these builds span about 1.2 % (German); perplexity cannot separate them. (No perplexity of the unpruned IQ3_XXS: the 76 GB file does not fit the test machine.)

Runtime used for speed and coding tests

The shared MTP head needs a llama.cpp fork. We used github.com/danielhanchen/llama.cpp, branch qwen4exp/mtp, commit 6fcaa16 (21 Sep 2026), built by us for Vulkan (MSVC, Vulkan SDK 1.4.357: cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DGGML_VULKAN=ON -DBUILD_SHARED_LIBS=ON -DLLAMA_OPENSSL=OFF -DLLAMA_CURL=OFF).

Speed (AMD Strix Halo, 64 GB, Radeon 8060S, Windows, Vulkan)

Same ~1700-token prompt each time, 32k context, KV q8_0, -lm dio --lazy-mode on, shared MTP sidecar mtp-Qwen3.8-Flash-Next-shared-Q4_K_M.gguf (1.78 GiB):

-b/-ub Speculation Prefill Decode
2048 MTP n-max 2 314.5 tok/s 34.5 tok/s
2048 MTP n-max 3 316.7 tok/s 33.6 tok/s
512 MTP n-max 3 257.5 tok/s 33.4 tok/s
512 MTP n-max 2 253.4 tok/s 32.8 tok/s

Without speculation in the official llama.cpp b11259 (Vulkan) the same file decodes at about 21 tok/s (measured during a coding task). ROCm is not recommended for this file: on Windows the decode collapses to about 5 tok/s as soon as the resident weights (file size minus the token table, here about 31 GiB) exceed the dedicated GPU memory; smaller cuts (K304 and below) stay fast on ROCm but were not evaluated for quality.

Coding check (own tasks, thinking budget 10000, max 12288 tokens, temperature 0.6, top-p 0.95, top-k 20, seeds 1000 + task number, one run each)

  • Eight anchor tasks: 8/8 passed, 30 to 34 tok/s decode, including the hardest one of the set (3,664 tokens).
  • Tasks 9 to 20 of the frozen 32-task set (the harder part; same questions, grader and seeds for every model we test): 11 of 12 passed. Task 15 failed (a hidden test returned 12 instead of 8). For orientation, in our earlier runs a dense Swift 1.5 27B Q5_K_M solved 6 of these 12 tasks and a 35B-A3B coder 8 of 12. These are not like-for-like numbers: the 27B ran without a thinking budget, this build with a budget of 10,000 tokens, and every task was run once. Task 21 of the 32-task set was not run.

Single runs on one machine; they are indicators, not a benchmark suite. Vision was not tested (no mmproj is included; use the BF16 projector from the Swift repository).

Known limits

  • Cutting experts without retraining costs some quality; the size of that cost was checked on German/English text and a small coding set only.
  • Speculation needs the fork above. The shared MTP sidecar is not part of this GGUF and does not load in official llama.cpp (b11259 and b11294 fail with token_embd.weight not found; the upstream loader has no MTP-only path). Unsloth publishes the shared sidecar as MTP/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf in unsloth/Qwen3.8-Flash-Next-GGUF; we measured with a Q4_K_M sidecar and did not test the Q8_0 one. Without speculation the file works in official llama.cpp.
  • An earlier Unsloth Vulkan package (build 10830, commit 0cd7a79f5) gave wrong results with this file in our tests: after the thinking budget ran out, the model kept reasoning in the answer part instead of writing the code (4 of 4 variants failed the same task that the fork and the official b11259 solved). We do not recommend that build for this model.
  • Licenses: Swift Open License v1.0 (commercial-use threshold, section 5) and Qwen Community License 1.0 (Model-as-a-Service / AI Work Assistant clause). Read both before commercial use.

Run example

llama-server -m Sakura-Qwen3.8-Flash-Next-Swift-K352-IQ3_XXS-00001-of-00002.gguf --jinja -fa on -ngl 99 -c 32768 -ctk q8_0 -ctv q8_0 -b 2048 -ub 2048 -lm dio --lazy-mode on --device Vulkan0 \
  --spec-type draft-mtp --spec-draft-model mtp-Qwen3.8-Flash-Next-shared-Q4_K_M.gguf --spec-draft-ngl 99 --spec-draft-n-max 2 --spec-draft-n-min 0

This is the command line used for the speed numbers with the fork build. Leave out the last line to run without speculation.


ไธญๆ–‡่ฏดๆ˜Ž ยท ๆจฑ่Šฑ (Simplified Chinese)

English above. ๆœฌ่Š‚ไธบไธŠๆ–‡็š„ไธญๆ–‡็ฟป่ฏ‘(Sakura = ๆจฑ่Šฑ yฤซnghuฤ);ๅฎŒๆ•ด็š„็‹ฌ็ซ‹ไธญๆ–‡็‰ˆ่ง README_zh.mdใ€‚

Sakura โ€” Qwen3.8-Flash-Next (Swift 1.5),512 ไธชไธ“ๅฎถไธญไฟ็•™ 352 ไธช,IQ3_XXS

Sakura logo

GPU ๅ†…ๅญ˜:ๆƒ้‡็บฆ 31.4 GiB โ€” ๅฏน 32 GB ๆ˜พๅกๆฅ่ฏดๅคชๅคง

่ฟ™ไธช 3 ๆฏ”็‰นๆž„ๅปบๅœจ GPU ไธŠไฟ็•™็บฆ 31.4 GiB ็š„ๆƒ้‡(26.8 GiB ็š„้€ๅฑ‚ token ่กจ้€š่ฟ‡ๆ‡’ๅŠ ่ฝฝ็•™ๅœจ็ฃ็›˜/ๅ†…ๅญ˜ไธญ)ใ€‚ๅฎƒๅฏไปฅๅœจ 64 GB ็š„ Strix Halo(็ปŸไธ€ๅ†…ๅญ˜:ๅœจๆˆ‘ไปฌ็š„ๆต‹่ฏ•ไธญ,32k ไธŠไธ‹ๆ–‡ใ€q8_0 KV ็ผ“ๅญ˜ๅ’Œ MTP ๅคด็š„ๆƒ…ๅ†ตไธ‹ๅ ็”จ 30.4 GiB ไธ“็”จๅ†…ๅญ˜ๅŠ  10.1 GiB ๅ…ฑไบซๅ†…ๅญ˜)ไธŠ่ฟ่กŒ,ไฝ† ไธ่ƒฝ ๆ”พ่ฟ› 32 GB ็š„ GPUใ€‚ๅฏนไบŽ 32 GB ็š„ๆ˜พๅก,่ฏทไฝฟ็”จ 2.5 ๆฏ”็‰น็š„ๅŒ็ณปๅˆ—็‰ˆๆœฌ Sakura-Qwen3.8-Flash-Next-Swift-2.5bit-352E-32GB-GGUF(GPU ๆƒ้‡็บฆ 26 GiB)ใ€‚

่ฟ™ๆ˜ฏ UkisAI ๅฏน Qwen3.8-Flash-Next ็š„ Swift 1.5 ๅพฎ่ฐƒ็‰ˆๆœฌ็š„ไธ€ไธชๅฎž้ชŒๆ€งไธ“ๅฎถๅ‰ชๆž GGUF:ๆฏๅฑ‚ 512 ไธช่ทฏ็”ฑไธ“ๅฎถไธญไฟ็•™ 352 ไธช (ไพๆฎๅพท่ฏญ + ่‹ฑ่ฏญ็š„่ทฏ็”ฑๆƒ…ๅ†ต้€‰ๅ‡บ),ๅ…ถไฝ™็š„ไปŽ็บฆ 3 ๆฏ”็‰น(IQ3_XXS,GSQ-RCO)้‡ๅŒ–ไธญ่ขซ่ฃๆމ,ๆฒกๆœ‰้‡ๆ–ฐ้‡ๅŒ–ใ€‚ๅคงๅฐ 58.19 GiB, ๅˆ†ไธบไธคไธชๅˆ†็‰‡ใ€‚ไธบๅ•ๅฐ 64 GB ็š„ AMD Strix Halo(Radeon 8060S)ๅˆถไฝœ;็›ฎๆ ‡่ฏญ่จ€ๆ˜ฏๅพท่ฏญๅ’Œ่‹ฑ่ฏญใ€‚่ฟ™ๆ˜ฏ็‹ฌ็ซ‹็š„็คพๅŒบๆž„ๅปบ็‰ˆๆœฌ, ไธๆ˜ฏ Qwen ๆˆ– UkisAI ็š„ๅฎ˜ๆ–นๅ‘ๅธƒใ€‚

ๅฑžไบŽ Sakura Micro ็ณปๅˆ—ใ€‚

ๆ‰€ๆœ‰ Sakura Flash-Next ๆž„ๅปบ็š„ๅนถๆŽ’ๆฏ”่พƒ(ๅŒบๅˆซใ€ๅฎžๆต‹่กจๆ ผใ€่ฟ่กŒๆ—ถ่ฏดๆ˜Žใ€ๅทฅๅ…ท):Sakura-Qwen3.8-Flash-Next-Overviewใ€‚

ๆฅๆบ

  • ๅŸบ็ก€ๆจกๅž‹:Qwen/Qwen3.8-Flash-Next(ๆžถๆž„ qwen4exp,48 ๅฑ‚,512 ไธชไธ“ๅฎถ,ๆฟ€ๆดป 10 ไธช)ใ€‚
  • ๅพฎ่ฐƒ + ้‡ๅŒ–:ukisai/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-GGUF(IQ3_XXS ๆกฃไฝ)ใ€‚ Swift ๆ˜ฏๅˆถไฝœ่ฟ™ไธชๆž„ๅปบ็š„ๅŽŸๅ› :ๅฎƒไฝฟ็”จ็š„ๆ€่€ƒ token ๆ˜Žๆ˜พๅฐ‘ไบŽๅŸบ็ก€ๆจกๅž‹ใ€‚
  • ๆœฌๆž„ๅปบ็š„ๆ”นๅŠจ:ไป…ไธบไธ“ๅฎถๅˆ‡็‰‡(่ง NOTICE.md)ใ€‚ๆฏๅฑ‚ไฟ็•™็š„ไธ“ๅฎถ:kept_experts_K352.jsonใ€‚
  • ไธ“ๅฎถ้€‰ๆ‹ฉ:Swift ๆจกๅž‹่‡ช่บซๅœจๅพท่ฏญ + ่‹ฑ่ฏญๆ กๅ‡†ๆททๅˆๆ–‡ๆœฌไธŠ็š„่ทฏ็”ฑๅ™จ็ปŸ่ฎก(่ทฏ็”ฑ่ดจ้‡);ๅช่ฆไธ€ไธชไธ“ๅฎถๅœจไปปไธ€่ฏญ่จ€ไธญๆ˜ฏ็ƒญ็‚น,ๅฐฑไบˆไปฅไฟ็•™ใ€‚ไฝฟ็”จ llama-expert-stats ๆต‹้‡,่ฟ™ๆ˜ฏไธ€ไธช่ฟ่กŒๆจกๅž‹ๅนถ็ปŸ่ฎก่ทฏ็”ฑๅ™จไฝฟ็”จๆƒ…ๅ†ต็š„ๅฐๅทฅๅ…ท(ๆฏ” llama-imatrix ๅฟซๅพ—ๅคš)ใ€‚

ๆ–‡ไปถ

ๆ–‡ไปถ ๅคงๅฐ
Sakura-Qwen3.8-Flash-Next-Swift-K352-IQ3_XXS-00001-of-00002.gguf 34.32 GiB(็จ ๅฏ†ๆƒ้‡ใ€ๅทจๅคง็š„้€ๅฑ‚ token ่กจๅ’Œไธ€้ƒจๅˆ†ไธ“ๅฎถ)
Sakura-Qwen3.8-Flash-Next-Swift-K352-IQ3_XXS-00002-of-00002.gguf 23.87 GiB(ๅ…ถไฝ™็š„่ทฏ็”ฑไธ“ๅฎถ)

SHA256SUMS ๅˆ—ๅ‡บไบ†ๆ ก้ชŒๅ’Œใ€‚ๅŠ ่ฝฝๅˆ†็‰‡ 1;llama.cpp ไผš่‡ชๅทฑๆ‰พๅˆฐๅˆ†็‰‡ 2ใ€‚่ฏทไฝฟ็”จ -lm dio --lazy-mode on(ๆˆ–ไฝ ๆ‰€็”จๆž„ๅปบไธญ็š„็ญ‰ๆ•ˆ้€‰้กน),ไฝฟๅทจๅคง็š„ token ่กจ่ขซๆ‡’ๅŠ ่ฝฝ่ฏปๅ–,่€Œไธๆ˜ฏๅธธ้ฉปๅ†…ๅญ˜ใ€‚

ๆต‹้‡็ป“ๆžœ(่‡ชๅทฑ็š„ๆต‹่ฏ•,่ฟ™ไธ€็กฎๅˆ‡็š„ๆ–‡ไปถ)

ๆฃ€ๆŸฅ้กน ็ป“ๆžœ
่ฏญ่จ€ๅ…ณๅก,14 ไธชๅพท่ฏญ + 14 ไธช่‹ฑ่ฏญๆ็คบ:ไฝฟ็”จไบ†้”™่ฏฏ่ฏญ่จ€็š„่ฏๆ‰€ๅ ๆฏ”ไพ‹ DE 0.000 / EN 0.005(ๅ…ณๅก:ๅ„ <= 0.06)
ๅพช็Žฏ / CJK ๅ›ž็ญ” 0 / 0
็•™ๅ‡บๅ›ฐๆƒ‘ๅบฆ(ๅพท่ฏญๆททๅˆ / ่‹ฑ่ฏญๆททๅˆ) 2.0514 / 1.8231(่ฏฆ่งไธ‹ๆ–‡)
่งฃ็  / ้ข„ๅกซๅ……้€Ÿๅบฆ ไฝฟ็”จๆŽจๆต‹่งฃ็ ๆ—ถ 34.5 tok/s ่งฃ็ ,314 tok/s ้ข„ๅกซๅ……(Vulkan,่ง้€Ÿๅบฆ่กจ)
็ผ–็จ‹ไปปๅŠก(่‡ชๅทฑ็š„ 8 ไธช LeetCode/AtCoder ้ฃŽๆ ผไปปๅŠก) 8/8 ้€š่ฟ‡(็”Ÿๆˆ 14,443 ไธช token,็บฆ 8.6 ๅˆ†้’Ÿ)
ๆˆ‘ไปฌๅ†ป็ป“็š„ 32 ไธชไปปๅŠก้›†ไธญ็š„ไปปๅŠก 9-20(ๆ›ด้šพ็š„ไปปๅŠก) 11/12 ้€š่ฟ‡(็”Ÿๆˆ 98,937 ไธช token),่งไธ‹ๆ–‡

่ฏญ่จ€ๅ…ณๅกๅ’Œๅ›ฐๆƒ‘ๅบฆๆ˜ฏ็”จๅฎ˜ๆ–น llama.cpp ๆž„ๅปบ b11259(Vulkan)ๆต‹ๅพ—็š„;้€Ÿๅบฆๅ’Œ็ผ–็จ‹ไปปๅŠกไฝฟ็”จไธ‹ๆ–‡ๆ‰€่ฟฐ็š„ๅˆ†ๆ”ฏๆž„ๅปบใ€‚

็•™ๅ‡บๅ›ฐๆƒ‘ๅบฆๅŽŸๅง‹ๅ€ผ
{"de": 2.0514, "en": 1.8231}

llama-perplexity,ไธŠไธ‹ๆ–‡ 512,ๅœจ่‡ชๅทฑ็š„ๅพท่ฏญๅ’Œ่‹ฑ่ฏญ็•™ๅ‡บๆ–‡ๆœฌ(ไธๅœจๆ กๅ‡†ๆ•ฐๆฎไธญ)ไธŠๅ– 40 ไธชๅ—ใ€‚ไฝœไธบๅ‚่€ƒ,ๅœจ็›ธๅŒๆ–‡ๆœฌไธŠ:Swift 512 ไธ“ๅฎถ IQ2_XS ไธบ 2.0605 / 1.8245,Swift K352 IQ2_XS ไธบ 2.0681 / 1.8229,็›ธๅŒ้…ๆ–น่ฃๅ‰ชๅˆฐ K320(IQ3_XXS)ไธบ 2.0437 / 1.8284ใ€‚่ฟ™ไบ›ๆž„ๅปบไน‹้—ด็š„ๅทฎๅผ‚็บฆไธบ 1.2 %(ๅพท่ฏญ); ๅ›ฐๆƒ‘ๅบฆๆ— ๆณ•ๅฐ†ๅฎƒไปฌๅŒบๅˆ†ๅผ€ใ€‚(ๆฒกๆœ‰ๆœชๅ‰ชๆž็š„ IQ3_XXS ็š„ๅ›ฐๆƒ‘ๅบฆ:76 GB ็š„ๆ–‡ไปถๆ”พไธ่ฟ›ๆต‹่ฏ•ๆœบๅ™จใ€‚)

็”จไบŽ้€Ÿๅบฆๅ’Œ็ผ–็จ‹ๆต‹่ฏ•็š„่ฟ่กŒๆ—ถ

ๅ…ฑไบซ MTP ๅคด้œ€่ฆ llama.cpp ๅˆ†ๆ”ฏใ€‚ๆˆ‘ไปฌไฝฟ็”จไบ† github.com/danielhanchen/llama.cpp,ๅˆ†ๆ”ฏ qwen4exp/mtp,ๆไบค 6fcaa16(2026 ๅนด 9 ๆœˆ 21 ๆ—ฅ),็”ฑๆˆ‘ไปฌ่‡ชๅทฑไธบ Vulkan ๆž„ๅปบ (MSVC,Vulkan SDK 1.4.357:cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DGGML_VULKAN=ON -DBUILD_SHARED_LIBS=ON -DLLAMA_OPENSSL=OFF -DLLAMA_CURL=OFF)ใ€‚

้€Ÿๅบฆ(AMD Strix Halo,64 GB,Radeon 8060S,Windows,Vulkan)

ๆฏๆฌกไฝฟ็”จ็›ธๅŒ็š„็บฆ 1700 ไธช token ็š„ๆ็คบ,32k ไธŠไธ‹ๆ–‡,KV q8_0,-lm dio --lazy-mode on,ๅ…ฑไบซ MTP ๆ—่ทฏๆ–‡ไปถ mtp-Qwen3.8-Flash-Next-shared-Q4_K_M.gguf(1.78 GiB):

-b/-ub ๆŽจๆต‹่งฃ็  ้ข„ๅกซๅ…… ่งฃ็ 
2048 MTP n-max 2 314.5 tok/s 34.5 tok/s
2048 MTP n-max 3 316.7 tok/s 33.6 tok/s
512 MTP n-max 3 257.5 tok/s 33.4 tok/s
512 MTP n-max 2 253.4 tok/s 32.8 tok/s

ๅœจๅฎ˜ๆ–น llama.cpp b11259(Vulkan)ไธญไธไฝฟ็”จๆŽจๆต‹่งฃ็ ๆ—ถ,ๅŒไธ€ไธชๆ–‡ไปถ็š„่งฃ็ ้€Ÿๅบฆ็บฆไธบ 21 tok/s(ๅœจไธ€ๆฌก็ผ–็จ‹ไปปๅŠกไธญๆต‹ๅพ—)ใ€‚ ไธๆŽจ่ๅœจๆญคๆ–‡ไปถไธŠไฝฟ็”จ ROCm: ๅœจ Windows ไธŠ,ไธ€ๆ—ฆๅธธ้ฉปๆƒ้‡(ๆ–‡ไปถๅคงๅฐๅ‡ๅŽป token ่กจ,่ฟ™้‡Œ็บฆไธบ 31 GiB)่ถ…่ฟ‡ไธ“็”จ GPU ๅ†…ๅญ˜,่งฃ็ ้€Ÿๅบฆๅฐฑไผš้ชค้™ๅˆฐ็บฆ 5 tok/s;่พƒๅฐ็š„่ฃๅ‰ช(K304 ๅŠไปฅไธ‹)ๅœจ ROCm ไธŠไป็„ถๅพˆๅฟซ,ไฝ†ๆฒกๆœ‰่ฏ„ไผฐๅ…ถ่ดจ้‡ใ€‚

็ผ–็จ‹ๆฃ€ๆŸฅ(่‡ชๅทฑ็š„ไปปๅŠก,ๆ€่€ƒ้ข„็ฎ— 10000,ๆœ€ๅคš 12288 ไธช token,temperature 0.6,top-p 0.95,top-k 20,้šๆœบ็งๅญ 1000 + ไปปๅŠก็ผ–ๅท,ๅ„่ฟ่กŒไธ€ๆฌก)

  • ๅ…ซไธช้”š็‚นไปปๅŠก:8/8 ้€š่ฟ‡,่งฃ็  30 ่‡ณ 34 tok/s,ๅŒ…ๆ‹ฌ่ฟ™ไธ€็ป„ไธญๆœ€้šพ็š„ไธ€ไธช(3,664 ไธช token)ใ€‚
  • ๅ†ป็ป“็š„ 32 ไธชไปปๅŠก้›†ไธญ็š„ไปปๅŠก 9 ่‡ณ 20(ๆ›ด้šพ็š„้ƒจๅˆ†;ๆˆ‘ไปฌๆต‹่ฏ•็š„ๆฏไธชๆจกๅž‹้ƒฝไฝฟ็”จ็›ธๅŒ็š„้—ฎ้ข˜ใ€่ฏ„ๅˆ†ๅ™จๅ’Œ้šๆœบ็งๅญ):12 ไธชไธญ้€š่ฟ‡ 11 ไธชใ€‚ไปปๅŠก 15 ๅคฑ่ดฅ(ไธ€ไธช้š่—ๆต‹่ฏ•่ฟ”ๅ›žไบ† 12 ่€Œไธๆ˜ฏ 8)ใ€‚ ไฝœไธบๅ‚่€ƒ,ๅœจๆˆ‘ไปฌ่พƒๆ—ฉ็š„่ฟ่กŒไธญ,็จ ๅฏ†็š„ Swift 1.5 27B Q5_K_M ่งฃๅ‡บไบ†่ฟ™ 12 ไธชไปปๅŠกไธญ็š„ 6 ไธช,ไธ€ไธช 35B-A3B ็ผ–็จ‹ๆจกๅž‹่งฃๅ‡บไบ† 12 ไธชไธญ็š„ 8 ไธชใ€‚่ฟ™ไบ›ไธๆ˜ฏๅŒๅฃๅพ„็š„ๆ•ฐๆฎ:27B ๆ˜ฏๅœจๆฒกๆœ‰ๆ€่€ƒ้ข„็ฎ—็š„ๆƒ…ๅ†ตไธ‹่ฟ่กŒ็š„,่€Œ่ฟ™ไธชๆž„ๅปบ็š„้ข„็ฎ—ๆ˜ฏ 10,000 ไธช token,ๅนถไธ”ๆฏไธชไปปๅŠกๅช่ฟ่กŒไบ†ไธ€ๆฌกใ€‚32 ไธชไปปๅŠก้›†ไธญ็š„ไปปๅŠก 21 ๆฒกๆœ‰่ฟ่กŒใ€‚

ๅ•ๆฌก่ฟ่กŒ,ๅœจไธ€ๅฐๆœบๅ™จไธŠ;ๅฎƒไปฌๆ˜ฏๆŒ‡็คบๆ€ง็š„,ไธๆ˜ฏๅŸบๅ‡†ๆต‹่ฏ•ๅฅ—ไปถใ€‚่ง†่ง‰่ƒฝๅŠ›ๆœช็ปๆต‹่ฏ•(ๆœชๅŒ…ๅซ mmproj;่ฏทไฝฟ็”จ Swift ไป“ๅบ“ไธญ็š„ BF16 ๆŠ•ๅฝฑๅ™จ)ใ€‚

ๅทฒ็Ÿฅๅฑ€้™

  • ไธ็ปๅ†่ฎญ็ปƒๅฐฑ่ฃๅ‰ชไธ“ๅฎถไผšๆŸๅคฑไธ€ไบ›่ดจ้‡;่ฟ™ไธ€ๆŸๅคฑ็š„ๅคงๅฐๅชๅœจๅพท่ฏญ/่‹ฑ่ฏญๆ–‡ๆœฌๅ’Œไธ€ไธชๅฐๅž‹็ผ–็จ‹้›†ไธŠๆฃ€ๆŸฅ่ฟ‡ใ€‚
  • ๆŽจๆต‹่งฃ็ ้œ€่ฆไธŠ่ฟฐๅˆ†ๆ”ฏใ€‚ ๅ…ฑไบซ MTP ๆ—่ทฏๆ–‡ไปถไธๅฑžไบŽ่ฟ™ไธช GGUF,ๅœจๅฎ˜ๆ–น llama.cpp ไธญๆ— ๆณ•ๅŠ ่ฝฝ(b11259 ๅ’Œ b11294 ไผšไปฅ token_embd.weight not found ๅคฑ่ดฅ;ไธŠๆธธๅŠ ่ฝฝๅ™จๆฒกๆœ‰ไป…ๅซ MTP ็š„่ทฏๅพ„)ใ€‚Unsloth ๅœจ unsloth/Qwen3.8-Flash-Next-GGUF ไธญๆŠŠๅ…ฑไบซๆ—่ทฏๆ–‡ไปถๅ‘ๅธƒไธบ MTP/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf;ๆˆ‘ไปฌไฝฟ็”จ Q4_K_M ๆ—่ทฏๆ–‡ไปถๆต‹้‡,ๆฒกๆœ‰ๆต‹่ฏ• Q8_0 ็š„้‚ฃไธชใ€‚ไธไฝฟ็”จๆŽจๆต‹่งฃ็ ๆ—ถ,่ฏฅๆ–‡ไปถๅฏๅœจๅฎ˜ๆ–น llama.cpp ไธญ่ฟ่กŒใ€‚
  • ่พƒๆ—ฉ็š„ Unsloth Vulkan ๅŒ…(ๆž„ๅปบ 10830,ๆไบค 0cd7a79f5)ๅœจๆˆ‘ไปฌ็š„ๆต‹่ฏ•ไธญๅฏนๆญคๆ–‡ไปถ็ป™ๅ‡บไบ†้”™่ฏฏ็š„็ป“ๆžœ: ๅœจๆ€่€ƒ้ข„็ฎ—็”จๅฎŒไน‹ๅŽ,ๆจกๅž‹ไปๅœจๅ›ž็ญ”้ƒจๅˆ†็ปง็ปญๆŽจ็†,่€Œๆฒกๆœ‰ๅ†™ๅ‡บไปฃ็ (4 ไธชๅ˜ไฝ“ไธญๆœ‰ 4 ไธชๅœจๅŒไธ€ไธชไปปๅŠกไธŠๅคฑ่ดฅ,่€Œ่ฏฅไปปๅŠกๅˆ†ๆ”ฏๅ’Œๅฎ˜ๆ–น b11259 ้ƒฝ่งฃๅ‡บไบ†)ใ€‚ๆˆ‘ไปฌไธๆŽจ่ๆŠŠ่ฟ™ไธชๆž„ๅปบ็”จไบŽ่ฏฅๆจกๅž‹ใ€‚
  • ่ฎธๅฏ่ฏ:Swift Open License v1.0(ๅ•†ไธšไฝฟ็”จ้—จๆง›,็ฌฌ 5 ่Š‚) ไปฅๅŠ Qwen Community License 1.0(Model-as-a-Service / AI Work Assistant ๆกๆฌพ)ใ€‚ๅ•†ไธšไฝฟ็”จไน‹ๅ‰่ฏท้˜…่ฏปไธค่€…ใ€‚

่ฟ่กŒ็คบไพ‹

llama-server -m Sakura-Qwen3.8-Flash-Next-Swift-K352-IQ3_XXS-00001-of-00002.gguf --jinja -fa on -ngl 99 -c 32768 -ctk q8_0 -ctv q8_0 -b 2048 -ub 2048 -lm dio --lazy-mode on --device Vulkan0 \
  --spec-type draft-mtp --spec-draft-model mtp-Qwen3.8-Flash-Next-shared-Q4_K_M.gguf --spec-draft-ngl 99 --spec-draft-n-max 2 --spec-draft-n-min 0

่ฟ™ๆ˜ฏไฝฟ็”จๅˆ†ๆ”ฏๆž„ๅปบๅพ—ๅ‡บ้€Ÿๅบฆๆ•ฐๆฎๆ—ถๆ‰€็”จ็š„ๅ‘ฝไปค่กŒใ€‚็œ็•ฅๆœ€ๅŽไธ€่กŒๅณๅฏๅœจไธไฝฟ็”จๆŽจๆต‹่งฃ็ ็š„ๆƒ…ๅ†ตไธ‹่ฟ่กŒใ€‚

Downloads last month
291
GGUF
Model size
139B params
Architecture
qwen4exp
Hardware compatibility
Log In to add your hardware

3-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for webmp3/Sakura-Qwen3.8-Flash-Next-Swift-3bit-352E-GGUF

Collections including webmp3/Sakura-Qwen3.8-Flash-Next-Swift-3bit-352E-GGUF