Skip to main content

Reasoning Models

Some models produce a chain-of-thought trace before the final answer. The trace is exposed alongside content so you can log it, show it to users, or strip it out.

Supported models​

Reasoning capability varies by model family. See the Model Catalog for the current set of reasoning-capable models. The catalog is the source of truth and is kept up to date as models are added or retired.

How reasoning is returned​

The reasoning trace is not always in the same field. It depends on the model family:

  • reasoning_content on the message: used by OpenAI-style models (for example openai/gpt-oss-120b, the Nemotron family, and Qwen3 thinking variants).
  • reasoning on the message: used by the GLM-5.x family and MiniMax-M2.5.
  • Inline </thinking> blocks inside content: a small number of models emit the trace inline rather than in a separate field. Strip the block out of content if you only want the final answer.

On the Chat Completions API you need to handle whichever of these the model you call returns. The Responses API normalizes the trace into a structured output item with type: "reasoning", regardless of model family, which is easier to handle uniformly.

Reading the trace robustly (Chat Completions)​

python
1234567891011121314151617181920
def extract_reasoning(message) -> tuple[str, str]:
    """Return (reasoning, final_answer) for any reasoning model."""
    reasoning = ""
    content = message.content or ""


    # OpenAI-style / Nemotron / Qwen3 thinking
    reasoning = getattr(message, "reasoning_content", None) or ""


    # GLM-5.x / MiniMax
    if not reasoning:
        reasoning = getattr(message, "reasoning", None) or ""


    # Inline thinking blocks in content (e.g. some Gemma variants)
    import re
    think = re.findall(r"<thinking>(.*?)</thinking>", content, re.DOTALL)
    if think and not reasoning:
        reasoning = "\n".join(t.strip() for t in think)
        content = re.sub(r"<thinking>.*?</thinking>", "", content, flags=re.DOTALL).strip()


    return reasoning, content

Non-streaming response shape​

json
123456789101112
{
  "choices": [
    {
      "message": {
        "role": "assistant",
        "reasoning_content": "Let me work through this step by step...",
        "content": "The answer is 4."
      },
      "finish_reason": "stop"
    }
  ]
}

For GLM-family models the trace is in reasoning instead of reasoning_content; the structure is otherwise the same.

Reasoning effort​

Use reasoning_effort to control how much reasoning the model does before answering. Lower effort is faster and cheaper; higher effort is more thorough for hard problems.

ValueWhen to use
lowSimple tasks where speed matters
mediumDefault for most workloads
highMulti-step math, logic, planning
maxHardest problems; longest latency

reasoning_effort is accepted on both endpoints:

  • Chat Completions (/v1/chat/completions): top-level reasoning_effort field.
  • Responses (/v1/responses): top-level reasoning_effort, or nested as reasoning: { effort: "..." }.

Not every model honors every value identically. The mapping from effort to actual token budget is model-specific; treat the table above as guidance, not a guarantee. Models that are not reasoning-capable will silently ignore reasoning_effort.

python
123456789101112131415161718
from openai import OpenAI


client = OpenAI(
    api_key="sk-your-api-key-here",
    base_url="https://api.inference.nebul.io/v1"
)


# gpt-oss-120b returns the trace in reasoning_content
response = client.chat.completions.create(
    model="openai/gpt-oss-120b",
    messages=[{"role": "user", "content": "What is 15% of 80?"}],
    reasoning_effort="high",
)


msg = response.choices[0].message
reasoning = getattr(msg, "reasoning_content", None) or getattr(msg, "reasoning", None) or ""
print("Reasoning:", reasoning)
print("Answer:", msg.content)

Streaming​

When streaming, reasoning tokens arrive first, followed by the final answer. Which delta field carries the reasoning depends on the model family, same as the non-streaming case: delta.reasoning_content for OpenAI-style models, delta.reasoning for GLM-family and MiniMax.

python
12345678910111213141516171819202122232425262728
from openai import OpenAI


client = OpenAI(
    api_key="sk-your-api-key-here",
    base_url="https://api.inference.nebul.io/v1"
)


stream = client.chat.completions.create(
    model="openai/gpt-oss-120b",
    messages=[{"role": "user", "content": "What is 15% of 80?"}],
    reasoning_effort="medium",
    stream=True,
)


answer_started = False
print("Thinking:")
for chunk in stream:
    delta = chunk.choices[0].delta


    reasoning = getattr(delta, "reasoning_content", None) or getattr(delta, "reasoning", None)
    if reasoning:
        print(reasoning, end="", flush=True)


    if delta.content:
        if not answer_started:
            print("\n\n--- Answer ---\n")
            answer_started = True
        print(delta.content, end="", flush=True)

Notes​

  • Latency: reasoning models take longer to produce the first token because they reason first. Budget for this in your timeouts.
  • Logging the trace: even if you only surface the final answer to users, keep the reasoning trace in your logs. It is the fastest way to debug a wrong answer.
  • Prompting: you do not need to tell a reasoning model to "think step by step". It does so by default. Reserve effort control for reasoning_effort.
Last updated on