North Mini Code home pagelight logo
  1. Deployment
  2. SGLang

Use SGLang if you want an OpenAI-compatible local server and your stack already prefers its runtime.

  • Pip install

  • Docker image

pip install sglang

python3 -m sglang.launch_server \
  --model-path "CohereLabs/North-Mini-Code-1.0" \
  --host 0.0.0.0 \
  --port 30000

curl -X POST "http://localhost:30000/v1/chat/completions" \
  -H "Content-Type: application/json" \
  --data '{
    "model": "CohereLabs/North-Mini-Code-1.0",
    "messages": [
      {
        "role": "user",
        "content": "Write a Python function that groups anagrams."
      }
    ],
    "temperature": 1.0,
    "top_p": 0.95
  }'

The Hugging Face deploy panel shows this route as an OpenAI-compatible server on port 30000, which makes it easy to swap into existing tooling that already expects chat completions.

vLLM guide

Compare SGLang with the North-specific vLLM setup from the model card.

Quantized Local Apps

Move here if you want smaller-footprint local routes instead of a full server runtime.