Instructions to use Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4")
model = AutoModelForMultimodalLM.from_pretrained("Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4

SGLang

How to use Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4 with Docker Model Runner:
```
docker model run hf.co/Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4
```

Swift-Qwen3.8-27B · huihui-style abliterated · NVFP4 (standard HF format)

NVFP4 + FP8 quantized checkpoint of the abliterated Swift model, published in the standard Hugging Face safetensors format. It loads in vLLM and transformers, so standard features such as JSON-schema structured output (vLLM guided decoding) work out of the box.

Specs


Base model	ukisai/Swift-Qwen3.8-27b (rev `1b30aaaf`)
Chain	Swift → huihui-style abliteration → NVFP4/FP8 quantization
Quantization	NVFP4 (MLP gate/up/down) + FP8 (attention, GDN, lm_head) — allocation copied verbatim from unsloth/Qwen3.8-27B-NVFP4 (`recipe.yaml` in this repo), 32 calibration samples
Weight size	21.8 GiB (2 safetensors shards) + MTP module included
Architecture	hybrid mamba/attention, 64 layers, multimodal (vision + MTP)
Built on	Modal, RTX PRO 6000 (sm_120a), vLLM 0.27.1

Usage (vLLM)

pip install vllm==0.27.1
export VLLM_USE_FLASHINFER_SAMPLER=0   # no nvcc needed in the container
vllm serve dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4 \
  --max-model-len 32768 --gpu-memory-utilization 0.9

JSON-schema output:

from vllm import SamplingParams
params = SamplingParams(temperature=1.0, top_p=0.95, top_k=20,
                        guided_json={"type": "object",
                                     "properties": {"answer": {"type": "string"}},
                                     "required": ["answer"]})

Provenance

Source: ukisai/Swift-Qwen3.8-27b @ 1b30aaaf753fe5c1cb51ada2ea0367a53445359c
Abliteration: huihui-style (refusal-direction projection, Arditi et al. 2024 family), transferred by per-tensor weight difference from the Qwen/Qwen3.8-27B ↔ huihui-ai/Huihui-Qwen3.8-27B-abliterated pair — format/weight change, no fine-tuning
Quantization: llm-compressor oneshot, NVFP4 + FP8 allocation copied verbatim from unsloth/Qwen3.8-27B-NVFP4 (quantization_config), 32 calibration samples
Config repair: nested multimodal config restored + quantization_config grafted (llm-compressor flattens it)
Verification: shape preflight + vLLM 0.27.1 smoke on RTX PRO 6000

Also available

The same checkpoint, packaged as a single-file engine artifact for Blackwell (sm_120a) instead of standard safetensors: Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4-NInfer.

License

This repository is a derivative of the Swift checkpoint, whose license is the Swift Open License v1.0 — not Apache. The chain:

Component	Licence
Qwen/Qwen3.8-27B (base model)	Apache-2.0 — Copyright 2026 Alibaba Cloud (`LICENSE-APACHE-2.0`)
ukisai/Swift-Qwen3.8-27b (Swift Contribution)	Swift Open License v1.0 (`LICENSE`)
This repo (abliteration + quantization of Swift)	derivative work — the Swift Contribution contained in it stays under the Swift Open License v1.0

What that means in practice:

Free use, including commercial, while your gross revenue (counting all controlled entities) is below the $1,000,000 per fiscal year threshold; qualified non-profits have no threshold for non-commercial or research use.
Above the threshold: obtain a separate written licence from UkisAI (Swift Enterprise License).
Redistribution: ship both licence files, keep the copyright and attribution notices, and mark files you modified (Swift licence §4–§5).

This is a description of what the licences say, not legal advice.

Intended use and limitations

This is an uncensored model: the upstream abliteration removes the refusal direction, so it will attempt requests a stock instruction-tuned model declines. It is published for research, evaluation and local deployment where that behaviour is understood and wanted.

Use at your own responsibility. Anyone deploying it is responsible for their own safeguards, output handling, and compliance with the licences above and applicable law. The publisher publishes this checkpoint as-is, without warranty of any kind, and is not liable for how it is used or for its outputs. The refusal behaviour — and any capability cost of the abliteration — originates entirely upstream; this repository changes precision and format, not behaviour.

Credit for the base model to Qwen (Alibaba Cloud); for the Swift training to UkisAI; for the abliteration style to huihui-ai; and for the published NVFP4 recipe to unsloth.

Downloads last month: 194

Safetensors

Model size

19B params

Tensor type

BF16

F8_E4M3

Model tree for Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4

Base model

Qwen/Qwen3.8-27B

Finetuned

ukisai/Swift-Qwen3.8-27b

Quantized

(68)

this model