ShapeLearn-Lite Held Up. ShapeLearn Did Better:
Qwen 3.8 27B

We were a little impatient.

Qwen 3.8 27B was released on August 14, 2026. Four days later, on August 18, we published our first set of GGUFs. We called them ShapeLearn-Lite for a reason: they were produced using a much smaller optimization budget, fewer checks, and much less waiting.

Now the full ShapeLearn models are done, and we have benchmarked them alongside the original Lite set and competing quants.

The good news: ShapeLearn-Lite held up pretty well. We will come back to that later in “ShapeLearn-Lite, in retrospect”.

The better news: the full ShapeLearn models are even better.

TL;DR

• Full ShapeLearn moves the measured quality-speed frontier beyond Lite. All five models in the new release sit on the frontier in each of our six GPU comparisons.
• GPU-5 is our default recommendation wherever it fits, reaching 99.63% of BF16’s aggregate benchmark score. If it does not fit with the context you need, GPU-4 is still very competitive: it reaches 98.72% of BF16 at a much smaller size (11.0 GB instead of 13.1 GB), and it is faster.
• ShapeLearn-Lite also performed better than its KLD ranking suggested: three of its six models sit on the frontier in the Lite-versus-Unsloth Dynamic v3 comparison.
• Speculative Decoding with MTP or DFlash2 increases throughput across every ShapeLearn model and GPU tested. DFlash2 is usually faster but requires more memory and does not support image inputs with llama.cpp. Choose DFlash2 for maximum text-only throughput when memory allows, and MTP when VRAM or multimodal support matters more.

Full ShapeLearn moves the frontier

We are releasing the full ShapeLearn run for Qwen 3.8 27B.

Within this release, larger models yield higher aggregate scores, while smaller models deliver higher throughput. That ordering holds across all six GPUs tested. Because this is a dense model and memory transfers are the bottleneck, lower BPW translates more directly into higher TPS than it does for MoEs.

The per-GPU comparisons also include AtomicChat, Bartowski, ISTA-DASLab, and Unsloth Dynamic v3. Bartowski’s latest models were released after our testing and are not included. Full ShapeLearn is labelled ByteShape in the figures.

All five ShapeLearn models remain on the measured frontier, with GPU-5 achieving the highest aggregate score among the plotted quants. Other teams also contribute competitive points. Notably, ISTA-DASLab’s excellent model (the yellow “d” on the graph below) also sits on the frontier.

By “frontier,” we mean that no other plotted model is both faster and more accurate.

96 GB: RTX Pro 6000

RTX PRO 6000, the GPU with the most memory, lets us show the full range of models tested.

GPU-5 is our default wherever you can fit it: it reaches 90.4 tok/s at 99.63% of the BF16 baseline.

32 GB: RTX 5090

The RTX 5090 tells a similar story, leading to the same recommendations.

Once again GPU-5 is our default choice, reaching 93.7 tok/s. Choose GPU-4 for slightly more context length or slightly better TPS.

24 GB: RTX 4090 and RTX 3090

RTX 4090

The RTX 4090 keeps the same pattern: GPU-5 is the default, reaching 59.2 tok/s.

RTX 3090

Older, but still fast in these measurements.

GPU-4 reaches 49.5 tok/s, compared with 45.8 tok/s for GPU-5. Moving to the larger model costs about 7.5% in throughput, while the aggregate score rises from 98.72% to 99.63% of BF16. That makes GPU-5 the default here as well.

16 GB: RTX 4080 and RTX 5060 Ti

With a tighter VRAM budget, the pragmatic choice is to leave room for the context you need, not just the model weights. These plots contain fewer competing configurations, but all five ShapeLearn models are represented.

RTX 4080

On the RTX 4080, GPU-4 reaches 52.4 tok/s, while GPU-5 reaches 45.7 tok/s.

RTX 5060 Ti

On the RTX 5060 Ti, the corresponding figures are 33.1 tok/s and 29.1 tok/s.

GPU-5 remains the default on both cards when the model, KV cache, and runtime buffers fit within your memory budget. When they do not, GPU-4 is still very competitive: almost 99% of BF16 at a much smaller size, and faster. A model appearing in these measurements does not establish that every context length or serving configuration will fit.

ShapeLearn-Lite, in retrospect

ShapeLearn-Lite uses a smaller optimization budget than full ShapeLearn. It let us get Qwen 3.8 27B onto 12 GB to 24 GB GPUs within a few days.

We released after targeted sanity checks and started the full evaluation afterwards. The full ShapeLearn models were ready before the benchmarking was finished. Evaluating both sets, along with the competing models, is what took most of the time.

Then Unsloth released its Dynamic v3 models. At similar sizes, several had lower KLD than Lite in our measurements. On KLD alone, Lite looked less competitive.

KLD looked decisive

KLD measures divergence between a quantized model’s predicted token distributions and the BF16 reference under a particular evaluation setup. It is useful for diagnosing substantial changes, but lower divergence does not automatically mean better task performance.

For example, Unsloth’s UD-IQ3_S has about 20% lower KLD than the similarly sized smallest Lite model: 0.028759 versus 0.035875. Yet its aggregate benchmark score is lower: 95.55% versus 97.33% of BF16.

If lower KLD were sufficient to rank these models by task performance, the benchmark ordering should have followed it.

It did not.

The point is not that KLD is useless. It is that a fidelity ranking is not a task-performance ranking. This is the distinction explored in our KLD evaluation blog. Our related paper on KLD and quantization fidelity metrics was also recently accepted to the EMNLP Industry Track.

Lite held up

Naturally, we made more plots.

Here, we show the RTX Pro 6000 because it can accommodate the full comparison. Each model’s benchmark score is reused across the GPU plots; the measured throughput and the set of displayed models change.

Leaving the full ShapeLearn models aside for a moment, three of the six ShapeLearn-Lite models sit on the Lite-versus-Unsloth frontier: the three smallest Lite models, the lighter orange bubbles labelled 1-3.

Of the twelve Unsloth v3 models shown, three also sit on that frontier: UD-IQ2_S, UD-Q2_K_XL, and UD-IQ4_XS. UD-IQ4_XS is a strong higher-quality point, while Lite earns its places in the middle of the range.

Add the five full ShapeLearn models back in (the darker orange bubbles), and they take over the entire frontier.

Lite was never meant to be the final result. It still held its own where it mattered.

Speculative Decoding

We also evaluated MTP and DFlash2 with the new models, using 3 draft tokens for MTP and 7 draft tokens for DFlash2. Both methods increased throughput across all five ShapeLearn models on all six GPUs tested.

To measure MTP and DFlash2 performance, we used the sampling parameters recommended by Qwen for thinking mode and evaluated them over a diverse set of agentic coding, mathematics, and general-knowledge requests. The speedup from MTP and DFlash2 relative to standard next-token prediction (NTP) would likely be even larger under greedy decoding. However, temperature-based sampling better reflects realistic usage, so we used it throughout our evaluation.

DFlash2 was faster than MTP in almost all cases. Across the full lineup, DFlash2 achieved 1.34-2.10x the baseline NTP throughput, while MTP achieved 1.28-1.66x.

There is also a memory tradeoff between the two approaches. The embedded quantized MTP weights add only about 250 MB to the model, and if MTP is not used, these weights are not loaded into GPU memory. In comparison, the 4-bit DFlash2 draft model is about 1 GB, so enabling DFlash2 requires roughly 1 GB of additional GPU memory. Packaging MTP as a separate GGUF file would largely eliminate this advantage: the standalone model would need its own MTP embedding and output layers, which are by far its largest tensors, bringing its memory footprint to roughly 1 GB as well. In addition, DFlash2 in llama.cpp currently does not support image inputs, which is an important consideration for multimodal use cases.

The quality axis represents the target-model benchmark scores reported above. These plots do not independently establish quality equivalence between decoding methods.

96 GB: RTX Pro 6000

32 GB: RTX 5090

24 GB: RTX 4090 and RTX 3090

RTX 4090

RTX 3090

16 GB: RTX 4080 and RTX 5060 Ti

RTX 4080

RTX 5060 Ti

Benchmarking Methodology

We evaluate all reported models across a set of instruct and thinking benchmarks.

Instruct benchmarks:

• GSM8K for math
• IFEval for instruction following
• MMLU for general knowledge
• LiveCodeBench V6* for coding
• Multi-IF for multi-turn and multilingual instruction following
• ACEBench for tool use and agentic tasks

Thinking benchmarks:

• ACEBench for tool use and agentic tasks
• Multiple HumanEval for coding
• BFCL V4* for tool calling and agentic tasks

For the thinking benchmarks, we used Qwen 3.8’s medium thinking setting.

For each benchmark, the score of a quantized model is normalized by the score of the corresponding BF16 model. The overall reported score is the average of these normalized benchmark scores.

Our LiveCodeBench V6* evaluation includes problems from January 1, 2024 onward, excluding the 2023 problems. We found the 2023 problems to be relatively easy for current models, with most models achieving very high scores on them. As a result, they provide limited discrimination between models while adding substantial evaluation time.

For BFCL V4*, we evaluate the following eight subsets:

• live_simple
• live_parallel
• live_parallel_multiple
• live_relevance
• multi_turn_base
• multi_turn_miss_func
• multi_turn_miss_param
• multi_turn_long_context

All evaluations were run with llama.cpp b10430. For both instruct and thinking experiments, we use the sampling parameters recommended by Qwen for the corresponding mode.

Conclusion

ShapeLearn-Lite did what it was designed to do. It got useful Qwen 3.8 27B quants onto 12 to 24 GB GPUs quickly, and it held up better than its KLD ranking suggested.

Full ShapeLearn goes further. It improves the measured quality-speed trade-offs over Lite and contributes five frontier models across all six tested GPUs.

GPU-5 is our default recommendation wherever it fits, reaching 99.63% of BF16’s aggregate benchmark score. When memory is tight, GPU-4 is still very competitive: almost 99% of BF16 at a much smaller size, and faster.

KLD remains useful, but it is not a task-performance leaderboard. Fidelity metrics tell us how much the model’s distributions changed under a particular measurement. Benchmarks tell us whether those changes matter on the tasks we tested.

We were impatient. This time, it worked out pretty well.