North Mini Code home pagelight logo
  1. Deployment
  2. vLLM

Use vLLM when you want a proper local server for coding agents, tool use, or multi-client access.

The Hugging Face model card says to use vLLM main for North Mini Code until a newer packaged release includes the required support. Accurate response parsing also requires cohere_melody>=0.9.0.

1

Install vLLM and Cohere Melody

uv pip install "git+https://github.com/vllm-project/vllm.git"
uv pip install "cohere_melody>=0.9.0"
2

Serve the model

vllm serve CohereLabs/North-Mini-Code-1.0 \
  -tp 2 \
  --max-model-len 320000 \
  --tool-call-parser cohere_command4 \
  --reasoning-parser cohere_command4 \
  --enable-auto-tool-choice
3

Send a chat request

curl -X POST "http://127.0.0.1:8000/v1/chat/completions" \
  -H "Content-Type: application/json" \
  --data '{
    "model": "CohereLabs/North-Mini-Code-1.0",
    "messages": [
      {
        "role": "user",
        "content": "Write a Python function that returns the longest common prefix of a list of strings."
      }
    ],
    "temperature": 1.0,
    "top_p": 0.95
  }'

North Mini Code supports interleaved thinking. The model card strongly recommends passing the model’s generated thinking content forward into later agent turns, along with tool results, for best multi-step performance.

  • Use with OpenCode

  • Why this path

{
  "$schema": "https://opencode.ai/config.json",
  "model": "vllm/CohereLabs/North-Mini-Code-1.0",
  "provider": {
    "vllm": {
      "name": "Local vLLM server",
      "options": {
        "baseURL": "http://127.0.0.1:8000/v1",
        "apiKey": "EMPTY"
      }
    }
  }
}

Agentic Coding Guide

Learn how to define tools with JSON schema and feed tool results back into the chat history.

SGLang alternative

Use SGLang if it already fits the rest of your inference stack better.