Fast, Tiny, Local Image Generation: Qwen-Image-2512

We are releasing Qwen-Image-2512, our first image-generation release.

We suggest two possible deployment paths:

GGUF models for ComfyUI and stable-diffusion.cpp.
Humming models for vLLM-Omni. Compared to the GGUF setup, this can be more than 2x faster but the whole setup is still experimental.

For both families we provide a setup guide to start sampling images through ComfyUI: GGUF and Humming. Please note that the Humming integration is experimental.

TL;DR

We are releasing Qwen-Image-2512 quantized in two flavours: GGUF for ComfyUI and stable-diffusion.cpp, and Humming for vLLM-Omni. The GGUF path is the stable one; the Humming path is experimental and roughly 2.5x faster.
On an NVIDIA RTX PRO 6000 Blackwell, at 1024 × 1024 with 20 steps, Humming generates an image in about 8–9 seconds and GGUF in about 21–24 seconds.
The models are small enough to run locally: with CPU offload, the most aggressively quantized builds peak at roughly 15–16 GB of VRAM at 1024 × 1024, and under 10 GB at 512 × 512.
Higher numerical precision does not always produce a better image. BF16 can fail where a quantized model succeeds, and a more aggressively quantized model can sometimes produce a more appealing result than a less quantized one.
At sufficiently low precision, degradation becomes obvious. But around 4 bpw and above, many of the models produced images that were difficult to distinguish from BF16 by visual quality alone.
Not sure which file to download? Check the VRAM and speed table below, then browse the curated image comparisons.

Which model should you use?

The GGUF models run through stable-diffusion.cpp or ComfyUI. Humming models run with vLLM-Omni and are substantially faster, but this integration is still experimental. On an NVIDIA RTX PRO 6000 Blackwell, at 1024 × 1024, with 20 steps, the Humming models generate an image in roughly 8–9 seconds, compared with about 21–24 seconds for GGUF.

Path 1 • The safe default

GGUF

Runtime

ComfyUI (through the ComfyUI-GGUF extension) or stable-diffusion.cpp

Compatibility

Runs almost anywhere. Windows, macOS or Linux, on NVIDIA, AMD or Apple Silicon, and on the CPU alone if that is all you have.

Speed

≈21–24 s per 1024 × 1024 image, 20 steps, RTX PRO 6000 Blackwell

Stability

Stable. A well-trodden setup: install ComfyUI, drop three files in place, load our workflow.

Sizes

Six quantizations from Q6_K (6.61 bpw) down to Q3_K_S (3.04 bpw), plus the BF16 baseline

Hugging Face repo → ComfyUI setup guide →

Path 2 • The fast one

Humming

Runtime

vLLM-Omni with Humming kernels, served through an OpenAI-compatible API or from ComfyUI

Compatibility

NVIDIA on Linux only. SM75 or newer, driver ≥ 575. No Windows, macOS or AMD support.

Speed

≈8–9 s per 1024 × 1024 image, 20 steps, RTX PRO 6000 Blackwell

Stability

Experimental. Pinned package versions and our own loader plugin.

Sizes

Six quantized transformers from 6.77 bpw down to 3.07 bpw, plus the BF16 baseline

Hugging Face repo → vLLM-Omni setup guide →

Both families can run entirely on the GPU or use CPU offload to reduce GPU memory requirements. Offloading can make larger models or image sizes accessible on smaller GPUs, at the cost of some additional latency.

Which model should you download?

Choose a runtime.
GGUF for stable-diffusion.cpp or ComfyUI; Humming for maximum speed if you are comfortable using an experimental vLLM-Omni integration.
Choose an image size.
Larger images require more GPU memory.
Check what fits.
The VRAM and speed table below reports peak VRAM for every model at every image size we measured. Use offload if the full model does not fit.
Pick the largest model that fits.
Less aggressive quantization stays numerically closer to the BF16 baseline, although this does not always produce better-looking images. We examine image quality later in the post; as a simple starting point, choose the largest model your hardware can comfortably run.

VRAM and speed by model

Every model we released, measured at the four output sizes we tested. Pick a runtime and an output size: the table shows what the model takes on disk, the peak GPU memory it reaches, and how long one image takes, both with everything resident on the GPU and with CPU offload.

Runtime

Output size

Peak VRAM and end-to-end time per model
Model	Total on disk	All on GPU		With CPU offload
Model	On disk	Peak VRAM	End-to-end	Peak VRAM	End-to-end

Measured on an NVIDIA RTX PRO 6000 Blackwell Workstation Edition · one image · 20 steps · euler · cfg 2.5 · flow-shift 3. The lowest value in each column is highlighted. Hover (or tap, on a touch screen) any underlined VRAM or time value to see what it is made of; vLLM-Omni does not report a trustworthy VRAM split, so only its timings break down. Scroll the table sideways to see the offload columns.

All on GPU keeps the diffusion model, text encoder and VAE resident in GPU memory. With CPU offload means the GGUF runtime holds and runs the text encoder on the CPU, while vLLM-Omni keeps the weights in system RAM and streams them to the GPU as they are needed. With the GGUF runtime you can free more VRAM still by putting the VAE on the CPU as well, but decoding there lengthens the end-to-end time considerably. End-to-end is the complete request (text encode, the 20 denoising steps, and VAE decode), and peak VRAM is the high-water mark across all of it. Every number here was measured with the BF16 text encoder; a quantized text encoder brings the memory footprint down further.

Setting up and running the models

Detailed instructions in the respective tabs below. Qwen-Image is made up of several components: the main diffusion model, which performs most of the image-generation work, plus a text encoder, VAE, tokenizer, and scheduler. The exact packaging and setup depend on whether you use the GGUF or Humming/vLLM-Omni release, so follow the corresponding guide below to download the required components and assemble the complete pipeline.

The GGUF models run with stable-diffusion.cpp and with ComfyUI through the ComfyUI-GGUF extension. This section walks through the ComfyUI setup.

What You Will Do

Running a GGUF model in ComfyUI involves five main steps:

Set up ComfyUI
Create a Python environment, install ComfyUI, and add the ComfyUI-GGUF extension.
Download the required model components
Download a ByteShape quantized diffusion model, the Qwen-Image VAE, and a compatible text encoder.
Start ComfyUI
Launch the ComfyUI server and open its web interface.
Load the example workflow
Import our prebuilt workflow to configure the model and generation pipeline.
Generate an image
Select the downloaded models, enter your prompts, choose the image dimensions, and adjust the sampling settings.

The sections below walk through each step in order.

Set Up the Environment

Create a Python environment, install ComfyUI, and add the ComfyUI-GGUF extension.

Create and activate the environment:

conda create -y -n comfy python=3.13
conda activate comfy

Clone ComfyUI and install its dependencies:

git clone https://github.com/Comfy-Org/ComfyUI.git
cd ComfyUI
pip install -r requirements.txt

Then add the ComfyUI-GGUF extension:

cd custom_nodes
git clone https://github.com/city96/ComfyUI-GGUF
cd ComfyUI-GGUF
pip install -r requirements.txt

Once the environment is ready, return to the root of the ComfyUI repository before downloading the models.

Download the Models

Qwen-Image requires three separate components:

The main diffusion model
The VAE
The text encoder

Main Diffusion Model

Choose your preferred quantization from our available models, then download it to the models/unet directory inside the ComfyUI repository:

curl -L -C - -o models/unet/Qwen-Image-2512-Q3_K_S-3.04bpw.gguf \
  https://huggingface.co/byteshape/Qwen-Image-2512-GGUF/resolve/main/Qwen-Image-2512-Q3_K_S-3.04bpw.gguf

VAE

The VAE is a relatively small model that Qwen-Image uses to decode images from latent space into pixel space:

curl -L -C - -o models/vae/qwen_image_vae.safetensors \
  https://huggingface.co/Comfy-Org/Qwen-Image_ComfyUI/resolve/main/split_files/vae/qwen_image_vae.safetensors

Text Encoder

The text encoder conditions Qwen-Image on your prompt. Qwen-Image uses Qwen2.5-VL-7B-Instruct for this purpose.

We did not quantize the text encoder ourselves because the model is relatively mature, several high-quality quantizations are already available, and it contributes little to the overall inference time.

Choose one of the following options.

Recommended

This is the recommended option and the one used by our workflow template.

Bartowski: Q4_K_L, 5.1 GB

curl -L -C - -o models/text_encoders/Qwen2.5-VL-7B-Instruct-Q4_K_L.gguf \
  https://huggingface.co/bartowski/Qwen_Qwen2.5-VL-7B-Instruct-GGUF/resolve/main/Qwen_Qwen2.5-VL-7B-Instruct-Q4_K_L.gguf

Alternatives

ComfyUI: BF16

curl -L -C - -o models/text_encoders/qwen_2.5_vl_7b.safetensors \
  https://huggingface.co/Comfy-Org/Qwen-Image_ComfyUI/resolve/main/split_files/text_encoders/qwen_2.5_vl_7b.safetensors

Unsloth: UD-Q4_K_XL, 4.8 GB

curl -L -C - -o models/text_encoders/Qwen2.5-VL-7B-Instruct-UD-Q4_K_XL.gguf \
  https://huggingface.co/unsloth/Qwen2.5-VL-7B-Instruct-GGUF/resolve/main/Qwen2.5-VL-7B-Instruct-UD-Q4_K_XL.gguf

Run ComfyUI

From the root of the ComfyUI repository, run:

python main.py

Once it starts, open a browser and go to http://127.0.0.1:8188 to reach the ComfyUI interface.

Load the Example Workflow

If you are new to ComfyUI, you can use the following workflow file to initialize your workspace and begin generating images:

byteshape-Qwen-Image-2512-GGUF-3.04bpw.json ComfyUI workflow

To load it in ComfyUI, press Ctrl+O and select the JSON file.

The loaded workspace with our smallest model should look like this:

The ByteShape Qwen-Image-2512 workflow loaded in ComfyUI, showing the Models, Prompt, and Image Size groups on the left, the sampler in the centre, and the generated image in the Save Image node on the right — Our example workflow in ComfyUI, loaded with the 3.04 bpw GGUF model. Click to enlarge.

Use the controls highlighted in blue to:

Select the required models
Enter positive and negative prompts
Choose the output image dimensions

The sampler controls in the center, let you configure the random seed, number of sampling steps, and other generation parameters.

ℹ️

The Image Size group in the attached workflow is set to 1328 × 1328. Change the width and height there for any other output size. The timings quoted in this post were measured at 1024 × 1024.

The vLLM models run on the vLLM-Omni backend using optimized Humming kernels. This is the faster of the two deployment paths.

Each model package contains the text encoder, VAE, scheduler, tokenizer, and a selection of quantized transformers. You choose a transformer size, place it in the pipeline directory, and start the server.

⚠️

Experimental. We tested this flow with vLLM 0.24.0, vLLM-Omni 0.24.0, vllm-omni-humming 0.3.0, and humming-kernels 0.1.11 on NVIDIA 30-, 40-, and 50-series GPUs, as well as the RTX PRO 6000 Blackwell.

What You Will Do

Running a Humming model involves five main steps:

Set up the environment
Create a Python environment and install vLLM, vLLM-Omni, the Humming kernels, and the ByteShape loader plugin.
Download the model
Download the shared Qwen-Image pipeline and the quantized transformer size you want to use.
Select the transformer
Link or move the chosen transformer into the pipeline directory.
Start the vLLM-Omni server
Serve the model with a single command. Enable CPU offload when needed to reduce GPU memory use.
Generate an image
Send requests through the OpenAI-compatible images API, either directly with curl or through ComfyUI.

The sections below walk through each step in order.

Set Up the Environment

Create a Python environment and install vLLM, vLLM-Omni, and our loader plugin. Linux and NVIDIA only (SM75+, driver ≥ 575).

conda create -y -n qwen-image-humming python=3.12
conda activate qwen-image-humming

Install the torch trio:

# torch trio (cu129)
pip install torch==2.11.0 torchvision==0.26.0 torchaudio==2.11.0 \
  --index-url https://download.pytorch.org/whl/cu129

Then vLLM itself, pinned to those exact torch builds:

# vLLM 0.24.0 (cu129 wheel)
printf 'torch==2.11.0+cu129\ntorchvision==0.26.0+cu129\ntorchaudio==2.11.0+cu129\n' > constraints.txt
PIP_CONSTRAINT=constraints.txt pip install \
  https://github.com/vllm-project/vllm/releases/download/v0.24.0/vllm-0.24.0+cu129-cp38-abi3-manylinux_2_28_x86_64.whl

Then vLLM-Omni and our loader plugin, under the same constraints:

# vLLM-Omni + our loader plugin
PIP_CONSTRAINT=constraints.txt pip install vllm-omni==0.24.0
PIP_CONSTRAINT=constraints.txt pip install vllm-omni-humming==0.3.0

Finally, the Humming kernels:

# humming kernels: must be >= 0.1.11 (older versions produce corrupted images)
pip install --no-deps "humming-kernels>=0.1.11"

vllm-omni-humming (PyPI, source) is our own plugin. We built and packaged it as the one required extra piece for this path, since stock vLLM-Omni cannot read a pre-quantized humming checkpoint on its own. The plugin registers itself and detects our models from their transformer/config.json.

Download the Model

Each package is a shared shell (the pipeline) plus a pool of transformers, one per size. Grab the shell and the size you want with the Hugging Face CLI:

pip install -U huggingface_hub

# the shell + one transformer (here the 3.07 bpw size)
huggingface-cli download byteshape/Qwen-Image-2512-Humming \
  --include "Qwen-Image-2512/*" "quantized_models/3.07bpw/*" \
  --local-dir Qwen-Image-2512-Humming

Move into the downloaded folder:

cd Qwen-Image-2512-Humming

Then link or move the chosen transformer into the pipeline directory:

# symlink it in (instant, no extra disk) ...
ln -sfn ../quantized_models/3.07bpw/transformer Qwen-Image-2512/transformer

# ... or move it in (a real, self-contained model dir)
mv quantized_models/3.07bpw/transformer Qwen-Image-2512/transformer

To change sizes, replace the linked or copied transformer. Transformer folders are named by bit width, such as 6.77bpw, 4.18bpw, and 3.07bpw.

Serve

From the folder root, start the server:

vllm-omni serve Qwen-Image-2512 \
  --omni --served-model-name Qwen/Qwen-Image-2512 \
  --enable-cpu-offload --port 8124

Keep --enable-cpu-offload on cards with ≤ 32 GB (an RTX 5090 needs it); drop it on larger GPUs for full speed. Do not pass --diffusion-quantization-config or set any VLLM_HUMMING_* variables; the plugin drives everything from the checkpoint. The first serve on a new GPU spends a few minutes compiling kernels for your card, then caches them.

Generate

The server exposes the OpenAI images API. Send a request and decode the returned base64 PNG:

curl -s http://127.0.0.1:8124/v1/images/generations \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen-Image-2512",
    "prompt": "A red panda on a mossy log at dawn",
    "size": "1024x1024",
    "num_inference_steps": 20,
    "true_cfg_scale": 2.5,
    "negative_prompt": " ",
    "seed": 1234
  }' \
  | python3 -c "import sys,json,base64; d=json.load(sys.stdin); open('out.png','wb').write(base64.b64decode(d['data'][0]['b64_json']))"

Always include a negative_prompt, even if it is only a space. With true_cfg_scale > 1, omitting it disables classifier-free guidance and produces flatter, lower-contrast images.

If Something Goes Wrong

A HummingConfig ... unexpected keyword argument error, or a shape mismatch on img_mod.1 / txt_mod.1 at load: the plugin isn’t installed in the active environment. Check with pip show vllm-omni-humming.
Grainy or speckled images: the Humming kernels are too old. Reinstall with pip install --no-deps "humming-kernels>=0.1.11".

Still stuck? Reach out through our contact form and we will help you get it running.

Driving the Server from ComfyUI

If you prefer a GUI over curl, you can drive the vLLM-Omni server from ComfyUI. The ComfyUI-vLLM-Omni custom node sends generation requests to a running vLLM-Omni service, local or remote, so ComfyUI is only the front end and vLLM still does the work. This reuses the ComfyUI install from the GGUF section; you just add one custom node.

Install the Node

Copy the node into your ComfyUI installation’s custom_nodes folder:

cd /path/to/ComfyUI/custom_nodes
git clone https://github.com/vllm-project/vllm-omni.git
cp -r vllm-omni/apps/ComfyUI-vLLM-Omni ComfyUI-vLLM-Omni
rm -rf vllm-omni

Restart ComfyUI (python main.py from the ComfyUI folder). A new vLLM-Omni folder should appear in the Node Library sidebar. No extra Python packages are needed beyond ComfyUI’s own.

Start the Server

Start the server as described above, and leave it running while using ComfyUI.

For a local server, ComfyUI will reach it at http://127.0.0.1:8124/v1. If ComfyUI runs on a different machine, use the server’s address instead of 127.0.0.1.

Load the Workflow

Use our prebuilt workflow:

qwen-image-humming-vllm-omni.json ComfyUI workflow

Load the JSON with Ctrl+O, or use Templates → vLLM-Omni → vLLM-Omni Image Generation.

In the Generate Image node, set:

url → http://127.0.0.1:8124/v1
model → Qwen/Qwen-Image-2512
prompt / negative_prompt → your text (negative: at least a space)
width / height → e.g. 1024 × 1024

In the attached Diffusion Sampling Params node, set the seed, num_inference_steps (e.g. 20), and true_cfg_scale (e.g. 2.5), then run the workflow.

Image quality: what to expect

Evaluating quantized image models is difficult because image quality is subjective and does not vary predictably with numerical precision. BF16 does not always produce the best result: a quantized model may follow the prompt more faithfully, and a more aggressively quantized model may sometimes produce a better-looking image than a less quantized one.

Differences across models are often subtle, prompt-dependent, and difficult to separate from normal variation across seeds. A poor output is therefore not necessarily caused by quantization. Clear degradation does appear at sufficiently aggressive levels, but around 4 bits per weight and above, many of the models were difficult to distinguish from BF16 by visual quality alone.

Rather than reducing these behaviours to a single, potentially misleading quality score, we provide a curated set of examples showing:

Cases where BF16 performs best
Cases where a quantized model follows the prompt more faithfully
Cases where a more quantized model produces the preferred image
The point at which quantization-related degradation becomes clearly visible

The goal is to give you a realistic sense of what to expect when using these models and not to suggest that image quality improves or declines in a perfectly predictable way with precision.

Coffee shop promotional poster generated from the curated prompt set

Anthropomorphic cat holding a subway handrail, generated from the curated prompt set

Steampunk aircraft above a Victorian rooftop, generated from the curated prompt set

Explore 24 curated prompts across both model families

Every prompt rendered by all seven GGUF variants and all seven Humming variants, side by side, with our notes on what each row is worth looking at, including the rows where BF16 is the one that fails.

Open the comparison

ℹ️

The set covers both families: the six GGUF quantizations and the six Humming quantizations, each compared against its own BF16 baseline, with identical prompts and sampling parameters throughout. Each family’s examples carry their own curator notes, written against that family’s outputs and its own BF16 baseline.