Blackwell Picks Favorites:
Qwen 3.5 35B A3B

Here's our next ByteShape release: Qwen 3.5 35B A3B. This time it is a Mixture-of-Experts model, not a dense one like the 9B, and the hardware story flips almost entirely.

With the 9B, GPUs were fairly agreeable while CPUs had strong, diverging preferences. For this 35B model it is the reverse: CPUs are surprisingly consistent, while GPUs are much pickier about which quantized models run best on each card.

That is why we are presenting models a bit differently this time. In the GPU charts, we highlight the models that are the best match for each specific card and gray out the ones that are not really the right choice. There is no single GPU pick that works best everywhere, but once you know your hardware, the right options become fairly clear.

We also have a step-by-step tutorial on how to run our models locally with OpenCode.

TL;DR

On CPUs, the picture is clean. ByteShape models trace a very consistent speed versus quality frontier across the i7, Ryzen 9 5900X, Ultra 7, and even the Raspberry Pi 5, so the recommendations barely change from one system to another.

On GPUs, device-specific optimization matters a lot more. The 40-series cards clearly like one set of models, while the Blackwell cards prefer others.

For the impatient:

• RTX 4080: GPU-4 is the obvious balanced pick.
• RTX 4090: GPU-5 would be our recommendation for "best" speed vs. accuracy, while GPU-6 practically matches baseline accuracy.
• RTX 5090 / RTX Pro 6000 Blackwell: GPU-5 remains our recommendation.
• CPUs: CPU-5 is our recommendation for all CPUs, with CPU-6 if you want to stay as close to baseline as possible.

GPUs

Unlike the usual pattern, where the same ByteShape models tend to perform best across all GPUs, Blackwell shows a much stronger preference for specific datatypes. The charts make this clear, so we highlight the models that best match what Blackwell favors, while still releasing models that remain strong choices on older GPUs as well.

RTX 4090 (24 GB)

The 4090 is actually one of the clearest charts here.

GPU-6 is the conservative, practically same-as-baseline accuracy pick. GPU-5, though, holds essentially baseline quality while adding a meaningful speed boost.

Below those, GPU-4 remains a solid balance pick, and GPU-3 is the faster, more aggressive pick if you are willing to give up more quality and free space for more context.

So on the 4090:

• GPU-6 if you want the safest near-baseline choice,
• GPU-5 if you want the more exciting high-end pick,
• GPU-4 if you want a more classic speed/quality balance.

RTX 4080 (16 GB)

Similar to 4090 but with less VRAM therefore fewer options.

GPU-4 is the clear recommendation: it sits around the mid-140s TPS while still landing at roughly 98.5% of baseline quality. That is a very clean trade-off with a maximum of 16K context length.

If you want more speed or bigger context, GPU-3 is the faster option, pushing into the mid-150s TPS range while staying around 96.4% quality. After that, GPU-2 and GPU-1 get increasingly aggressive, and by that point the quality trade-off is much harder to justify unless your priority is simply extracting maximum throughput/minimizing model size to allow for bigger context windows.

So for the 4080, the recommendation is straightforward:

• GPU-4 for the best overall balance,
• GPU-3 if you want the faster, slightly more aggressive option.

RTX 5090 (32 GB)

This is where Blackwell's preferences really start to stand out. In particular, on Blackwell with llama.cpp kernels, IQ datatypes perform better, while KQ datatypes are less competitive. As a result, GPU-4 and GPU-2 do not fall on the ideal quality-performance curve because most of their layers are dominated by KQ datatypes.

On Blackwell, GPU-5 looks excellent. It gets very close to baseline quality while pushing close to the 200 TPS range.

GPU-6 is still there as the high-quality anchor, and it is also fast. But if you are looking at the 5090 chart and asking "what is the new thing here?", the answer is clearly GPU-5.

So for the 5090:

• GPU-5 is the standout,
• GPU-6 is the conservative premium-quality choice.

RTX Pro 6000 Blackwell (96 GB)

Very similar story to the 5090, which is exactly what you would expect.

Again, GPU-5 looks unusually strong here. It stays essentially on top of the quality chart while delivering a large speed advantage, and once again it looks especially well aligned with Blackwell.

GPU-6 is the safe pick if your top priority is hugging the baseline as closely as possible. GPU-4 is still fine. But the interesting result is the same as on the 5090: GPU-5 is the one that makes you stop and look twice.

So, déjà vu, for the RTX Pro 6000 Blackwell (workstation):

• GPU-5 is excellent speed and accuracy,
• GPU-6 is the conservative premium-quality choice.

CPUs

Picking the right quant for CPU proves more straightforward.

Intel Core i7 12700KF

CPU-6 is the near-baseline option. CPU-5 is probably the default pick for most people: still very close to baseline quality, but with a bit more speed. CPU-4 is the more aggressive/balanced option, and after that we get into more noticeable quality vs. speed trade-offs.

So, for the i7:

• CPU-5 is the default recommendation,
• CPU-6 if you want the highest quality,
• CPU-4 if you want more speed without getting too adventurous.

Ryzen 9 5900X

Almost the same story as the i7, which is refreshing.

The frontier is clean, the recommendations are clean, and the overall trade-off pattern looks very familiar. Again, CPU-5 remains a great default choice, CPU-6 is the quality-first choice, and CPU-4 is the more speed-oriented alternative.

Ultra 7 265KF

The Ultra 7 is just good news all around.

Here again, the same ByteShape models sit on the frontier, but the absolute speeds are better. Sounds familiar? CPU-5 is a great default, CPU-6 is near-baseline pick, and CPU-4 or CPU-3 are there if you want to push harder on speed.

Raspberry Pi 5 (16 GB)

It's alive, barely, but still kicking. For Pi deployments, maybe take a look at Qwen3-Coder which is nearly 3x faster.

CPU-4 and CPU-3 are worth a closer look, landing around the 3 TPS range while keeping quality in a range that is still genuinely usable.

Benchmarking Methodology

Generating our models takes little time. What takes disproportionately longer is evaluating all reported models across the following seven benchmarks:

• BFCL_V3 for tool calling
• GSM8K_V for vision + math
• LiveCodeBench V6 and HumanEval for coding
• GSM8K for math
• IFEVAL for instruction following
• MMLU for general knowledge

The reported score is the mean across these benchmarks, with each benchmark normalized to the original model's score.

We evaluated GSM8K_V in both instruct and thinking modes and treated them as separate entries in the average. In practice, we observe that the relative performance between modes remains consistent.

All evaluations were run with llama.cpp b8204.