Happy Canada Day:
Cohere North Mini Code 1.0

Today, we are releasing our next set of ByteShape-compressed models for Cohere North Mini Code 1.0 (30B-A3B).

This release is Canadian through and through: Cohere is Canadian, ByteShape is Canadian, and North Mini Code is a practical, efficient coding model designed for local and private deployment.

TL;DR

• We are releasing ByteShape-compressed GGUF models for Cohere North Mini Code 1.0.
• Across the RTX 4090, RTX 5090, RTX 4080, and RTX 5060 Ti, ByteShape models sit on the faster side of the quality-speed trade-off.
• On 24GB+ GPUs, GPU-3 offers the best practical quality-speed balance, with near-baseline quality and substantially higher throughput.
• On 16GB GPUs, GPU-1 is the better choice when you need 64K+ context without CPU offloading. GPU-2 offers the best quality-speed trade-off with a smaller context window, or when partial CPU offloading is acceptable. It reaches around 98.7% of BF16 baseline average accuracy while running substantially faster than high-quality alternatives.

Benchmarks

The plots below compare average token-generation throughput with average benchmark accuracy, measured on LiveCodeBench V6 and MultiPL-E HumanEval, both in thinking mode. Accuracy is normalized to the BF16 baseline model.

Higher is better on both axes: moving right means more tokens per second, while moving up means higher average accuracy. Bubble size is proportional to model size, so larger bubbles generally indicate less aggressive compression.

The useful region is the upper-right: high accuracy and high throughput. For coding models, this matters especially because a fast model with lower output quality can lose its speed advantage when users need to spend more time fixing generated code.

24GB+ GPUs

We start with the higher-memory GPUs, where the selected model lineup fits comfortably. This family includes the RTX 4090 and RTX 5090, and the picture is much the same on both.

RTX 4090

On the RTX 4090, GPU-4 matches the BF16 baseline on average accuracy while running faster than the strongest Unsloth variants at similar quality (the green bubbles in the charts below). It is the quality-preserving option.

For most users, GPU-3 is the more practical default. It remains close to BF16 quality, at around 99.6% of baseline average accuracy, while delivering substantially higher throughput.

GPU-2 is the more speed-oriented option, reaching around 98.7% of baseline average accuracy. It is useful when throughput matters more than preserving the final fraction of quality.

RTX 5090

The RTX 5090 shows the same pattern at higher absolute throughput. For the RTX 5090, use GPU-3 for the best quality-speed balance.

16GB GPUs

On 16GB GPUs, both GPU-1 and GPU-2 are practical options, depending on your context-length requirements.

GPU-1 fits comfortably on 16GB GPUs while leaving room for 64K+ context.

GPU-2 also fits on 16GB GPUs with a smaller context window. For larger contexts, you may need to offload some layers to the CPU.

That trade-off can still be worthwhile. GPU-2 is faster and higher quality than GPU-1, so even with partial CPU offloading, it may remain the better overall choice for users who prioritize response quality and generation speed over maximum context length.

For most 16GB users:

• choose GPU-1 when you need 64K+ context without CPU offloading,
• choose GPU-2 when you want the best quality-speed trade-off and can use a smaller context window or tolerate some CPU offloading.

RTX 4080

On the RTX 4080, GPU-2 reaches around 98.7% of BF16 average accuracy at roughly 224 tokens per second.

RTX 5060 Ti

The RTX 5060 Ti result is similarly clear. GPU-2 reaches roughly 155 tokens per second while preserving around 98.7% of BF16 baseline average accuracy. Comparable high-quality alternatives are substantially slower, in the mid-120 TPS range.

Conclusion

North Mini Code is already an efficient coding model: 30B total parameters, 3B active, and optimized for software engineering workflows. ByteShape pushes that efficiency further.

For Canada Day, we are happy to release a Canadian model compressed by a Canadian team.

The recommendations are:

• On 24GB+ GPUs, use GPU-3 for the best quality-speed trade-off.
• On 16GB GPUs, use GPU-1 when you need 64K+ context without offloading. Use GPU-2 for the best quality-speed trade-off with a smaller context window, or when partial CPU offloading is acceptable.