tirah321/strata-nvfp4-linux-sm120-cuda13

Strata NVFP4 Engine — Linux (prebuilt) · sm_120 · CUDA 13.3

Pre-compiled Linux binary of the Strata NVFP4 inference engine (fork sergqwer/strata-nvfp4), so Linux users can run Qwen3.8-Flash-Next (NVFP4 quantized) without compiling from source.

Why this exists: upstream Strata and the NVFP4 fork only publish ready-made (prebuilt) engines for Windows. On Linux the engine must be compiled from source (10–20 min, needs CUDA toolkit + cmake/ninja). This repo provides the already-compiled Linux binary.

What's inside strata-linux-sm120-v0.1.44.1.tar.gz

File	Size	Purpose
strata	~86 MB	Main inference engine (CUDA, NVFP4 support)
strata-vision	~8 MB	Vision/image encoder (CPU build)
libstrata_kq.so	~14 MB	K-quant MMQ prompt library
BUILD.json / strata-BUILD.json	small	Engine metadata (version, arch, CUDA dirs)
README.md	—	Usage notes

Build info

Property	Value
Engine version	0.1.44.1 (fork sergqwer/strata-nvfp4)
Compute capability	sm_120 (NVIDIA Blackwell: RTX 50-series / RTX PRO 6000)
CUDA	13.3 (nvcc V13.3.73)
Build OS	Debian 13 (glibc 2.41)
Architecture	x86_64, AVX-512

⚠️ This binary is compiled only for sm_120. For RTX 20/30/40 (sm_75/86/89) or other GPUs you need a different build — compile it yourself with SETUP.py --build (see the upstream repo).

How to use

1. Download

wget https://huggingface.co/tirah321/strata-nvfp4-linux-sm120-cuda13/resolve/main/strata-linux-sm120-v0.1.44.1.tar.gz

or with the hf CLI:

hf download tirah321/strata-nvfp4-linux-sm120-cuda13 strata-linux-sm120-v0.1.44.1.tar.gz

2. Install CUDA 13 runtime libraries

The engine is built with CUDA 13.3 and needs the matching runtime libs (libcudart, libcublas, ...). They are available as pip wheels — no system CUDA install and no root required:

pip install nvidia-cuda-runtime==13.0.96 nvidia-cublas==13.0.2.14

3. Get the model + config

The engine runs the Qwen3.8-Flash-Next NVFP4 GPTQ model. Download the model files (or use setup.py --no-start to fetch/prepare them and generate the strata-*.json config):

• Model repo: Maximilian228/Qwen3.8-Flash-Next-NVFP4-GPTQ-Strata
• The strata-qwen-nvfp4-gptq.json config is produced by Strata's setup.py.

4. Run the server

# Extract the engine into the `engine/` folder of a Strata checkout
mkdir -p /path/to/strata-nvfp4/engine
cd /path/to/strata-nvfp4/engine
tar xzf strata-linux-sm120-v0.1.44.1.tar.gz

# Point LD_LIBRARY_PATH at the CUDA 13 pip libs and start
export LD_LIBRARY_PATH="$(python3 -c 'import nvidia.cuda_runtime, pathlib; print(pathlib.Path(nvidia.cuda_runtime.__file__).parent)')"
cd /path/to/strata-nvfp4
./engine/strata --config strata-qwen-nvfp4-gptq.json --port 8080

The server exposes an OpenAI-compatible API at http://127.0.0.1:8080/v1 and an Anthropic-compatible API at /v1/messages.

Verifying the download

9643203ca5dd0fdd95e0f5b17f1cfdea42d2ca34ca0eef0e76a08ff3e193ffc0  strata-linux-sm120-v0.1.44.1.tar.gz

See strata-linux-sm120-v0.1.44.1.sha256.

Credits

• Engine: Niko1221/Strata (MIT)
• NVFP4 fork: sergqwer/strata-nvfp4
• Model: Qwen/Qwen3.8-Flash-Next (Qwen license)