Qwen3-Embedding-4B-q4f16_1-MLC-b2

MLC-compiled Qwen3-Embedding-4B (Apache-2.0) in web-llm 0.2.84's format for WebGPU in-browser embedding. Built locally with mlc-llm (tvmjs 0.24.0-dev3 / mlc-ai 0.26.dev246 generation) + emsdk 3.1.56.

  • Quantization: q4f16_1 · context: 2048 · prefill chunk: 2048 · max batch: 2
  • Embedding dim: 2560 (last-token pooling, L2-normalized — done in the compiled model)
  • WebGPU: works on NVIDIA (Dawn); grid/buffer limits sized for batch 2/2048

Files

File Purpose
mlc-chat-config.json model config (ctx 2048, chunk 2048, max_batch 2)
tensor-cache.json / ndarray-cache.json weight manifest (identical bytes)
params_shard_0.bin … params_shard_73.bin quantized weights (~2.1 GB)
tokenizer.json, tokenizer_config.json, vocab.json, merges.txt Qwen3 tokenizer
qwen3-embedding-4b-q4f16_1-ctx2048_batch2-webgpu.wasm the model library (wasm)

Use in web-llm (0.2.84)

const record = {
  model: 'https://huggingface.co/skkut/qwen3-embedding-4b-q4f16_1-MLC-b2/resolve/main/',
  model_id: 'Qwen3-Embedding-4B-q4f16_1-MLC-b8',
  model_lib: 'https://huggingface.co/skkut/qwen3-embedding-4b-q4f16_1-MLC-b2/resolve/main/qwen3-embedding-4b-q4f16_1-ctx2048_batch2-webgpu.wasm',
  model_type: 1, // EmbeddingModelType
  vram_required_MB: 4400,
};

Query-side texts should use the Qwen3 instruction prefix (Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery: …).

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support