North Mini Code home pagelight logo
  1. Deployment
  2. Transformers

Use this path if you want a direct Python integration without standing up a separate inference server.

The current Hugging Face model card says to use a Transformers build that already includes the necessary North Mini Code support. If your installed version does not recognize the chat template or tokenizer behavior, upgrade to a newer release or install from source.

Hugging Face recommends temperature=1.0 and top_p=0.95 for generation with North Mini Code.

  • Direct load

  • Pipeline

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "CohereLabs/North-Mini-Code-1.0"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
)

messages = [
    {"role": "user", "content": "Write a Python function that merges two sorted lists."}
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    do_sample=True,
    temperature=1.0,
    top_p=0.95,
)

print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

​
Why Use This Path

Simple

No separate server is required.

Python-native

Good fit for notebooks, scripts, and custom applications.

Full chat template access

You can format prompts exactly the way North Mini Code expects.

Easy next step

Move to vLLM if you outgrow single-process local inference.

Keep prompts in structured chat-message format and use apply_chat_template(...) rather than hand-building the final string yourself.

vLLM deployment

Upgrade to an OpenAI-compatible server for agent workflows and multi-client usage.

Agentic coding guide

See how tool definitions and tool results should be threaded through the conversation.