North Mini Code home pagelight logo
  1. Getting Started
  2. Quickstart

This page is optimized for first output, not final architecture. If you already know your target runtime, jump directly to the deployment section.

Transformers

Best Python-native first run.

vLLM

Best local server for agents and OpenAI-compatible clients.

SGLang

Alternative local server with pip and Docker flows.

Amazon SageMaker

Managed AWS endpoint using Hugging Face’s deploy snippet.

If your goal is agentic coding rather than simple text generation, prefer vLLM or SGLang over a minimal one-file Transformers script. Those runtimes map more cleanly to tool use and multi-step workflows.

​
Minimal Python Example

Use this when you want the shortest path from model download to generated code.

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "CohereLabs/North-Mini-Code-1.0"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
)

messages = [
    {
        "role": "user",
        "content": "Write a Python function that checks whether a string is a palindrome.",
    }
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    do_sample=True,
    temperature=1.0,
    top_p=0.95,
)

print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Recommended sampling settings from the Hugging Face model card: temperature=1.0, top_p=0.95.

  • What to do next

  • Common first-run checks

  • Want a richer local server? Move to /deployment/vllm.
  • Want a simpler container-first path? Move to /deployment/docker-model-runner.
  • Want managed AWS deployment? Move to /deployment/sagemaker.
  • Want a lower-memory variant? Move to /deployment/quantized-local-apps.