Model Details - North Mini Code

Model size

30B total parameters with 3B active at inference time.

Context

256K context window with up to 64K output tokens.

License

Apache 2.0 open weights.

Primary fit

Agentic coding, terminal tasks, and code generation.

Architecture

North Mini Code is a decoder-only sparse Mixture-of-Experts model.

The feed-forward block uses 128 experts.
8 experts are activated per token.
The model alternates sliding-window attention with RoPE and global attention without positional embeddings in a 3:1 ratio.
The MoE router applies a sigmoid activation before top-k expert selection.

Training Approach

North Mini Code was post-trained in two stages:

Supervised fine-tuning (SFT)
Reinforcement learning with verifiable rewards (RLVR)

The post-training pipeline focused on agentic software engineering, terminal tasks, and robust behavior across multiple harnesses rather than overfitting to a single agent framework.

The Hugging Face launch blog says the training pipeline used more than 70k verifiable tasks across roughly 5k repositories.

Why It Works for Agents

Tool use

The model was trained for tool calls inside coding-agent loops.

Interleaved thinking

The model card explicitly recommends preserving generated thinking content across turns.

Harness robustness

Cohere trained against multiple evaluation harnesses rather than a single scaffold.

Local friendliness

The 3B active footprint makes local inference practical compared with similarly strong open models.