Chat Completions

OpenAI-compatible chat completions endpoint for multi-turn conversations, system instructions, function calling, and structured outputs

The Chat Completions endpoint (POST /v1/chat/completions) is the workhorse of the Inference API: send a list of messages, get the model's reply. It works with any chat model in the Model Catalog and supports system instructions, tools, structured output, and streaming.

Every example here reads your key from NEBUL_API_KEY; export it first. The Quick Start covers key setup and first requests in eight languages.

Request

POST https://api.inference.nebul.io/v1/chat/completions

Parameters

Parameter	Type	Required	Description
`model`	String	Yes	The model ID, e.g. `zai-org/GLM-5.3`.
`messages`	Array	Yes	Message objects with `role` and `content`. Roles: `system`, `user`, `assistant`.
`temperature`	Number	No	Sampling temperature (0.0-2.0). Higher values make output more random. Defaults to `1.0`.
`top_p`	Number	No	Nucleus sampling parameter (0.0-1.0). Defaults to `1.0`.
`n`	Integer	No	Number of completions to generate. Defaults to `1`.
`stream`	Boolean	No	Stream the response as SSE chunks. Defaults to `false`.
`stop`	String or Array	No	Stop sequences, as a string or array of strings.
`max_completion_tokens`	Integer	No	Maximum number of tokens to generate.
`presence_penalty`	Number	No	Penalty for token presence (-2.0 to 2.0). Defaults to `0.0`.
`frequency_penalty`	Number	No	Penalty for token frequency (-2.0 to 2.0). Defaults to `0.0`.
`seed`	Integer	No	Random seed for reproducible outputs.
`tools`	Array	No	Tool definitions. See Function Calling & Tools.
`tool_choice`	String or Object	No	Tool choice strategy.
`response_format`	Object	No	Response format constraints. See Structured Output & JSON.
`user`	String	No	Identifier for the end user.

Temperature tuning

Lower temperatures (0.0-0.3) produce more focused, deterministic output; higher values (0.7-1.0) increase variety. Use temperature: 0 for tasks that need factual accuracy.

Example

import os

from openai import OpenAI

client = OpenAI(
    api_key=os.environ["NEBUL_API_KEY"],
    base_url="https://api.inference.nebul.io/v1",
)

response = client.chat.completions.create(
    model="zai-org/GLM-5.3",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is the capital of France?"},
    ],
)

print(response.choices[0].message.content)

The tabs show the minimal call; the sampling parameters from the table above attach in each SDK's syntax (temperature, max_completion_tokens, and so on).

Response

The answer is in choices[0].message.content; token counts are in usage:

{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1677652288,
  "model": "zai-org/GLM-5.3",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "The capital of France is Paris."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 15,
    "completion_tokens": 8,
    "total_tokens": 23
  }
}

Field	Type	Description
`id`	String	Unique identifier for the completion.
`object`	String	Object type, always `"chat.completion"`.
`created`	Integer	Unix timestamp of when the completion was created.
`model`	String	Model ID used for the completion.
`choices`	Array	Completion choices.
`choices[].message.content`	String	The answer text.
`choices[].finish_reason`	String	Why generation stopped: `stop`, `length`, `tool_calls`.
`usage`	Object	Token usage: `prompt_tokens`, `completion_tokens`, `total_tokens`.

Message roles

The messages array carries the conversation:

system sets the assistant's behavior; put it first.
user is the human's turn.
assistant is a previous model reply, included to keep context across turns.

{
  "model": "zai-org/GLM-5.3",
  "messages": [
    { "role": "system", "content": "You are a helpful assistant." },
    { "role": "user", "content": "What is the capital of France?" },
    { "role": "assistant", "content": "The capital of France is Paris." },
    { "role": "user", "content": "What is its population?" }
  ]
}

There is no server-side conversation state: the full history travels with every request, and you pay for the input tokens each time. For long conversations, trim old messages or use prompt caching to cut the cost of the repeated prefix.

Streaming

Set stream: true and tokens arrive as SSE chunks as they are generated. Working examples in eight languages are on the Examples page; in short:

stream = client.chat.completions.create(
    model="zai-org/GLM-5.3",
    messages=[{"role": "user", "content": "Tell me a short story."}],
    stream=True,
)

for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

Capabilities beyond plain chat

Function calling: let the model ask your code to run tools.
Structured output: constrain replies to a JSON schema.
Reasoning models: control how much a model thinks before answering.
Vision: send images as content parts.

Each capability is only a request field away; the deep pages cover them field by field.