Messages API

Anthropic-compatible Messages API endpoint for conversational responses, tool use, streaming, and token counting

The Messages API provides an Anthropic-compatible endpoint for generating conversational responses. It follows the Anthropic Messages API specification, so tools and clients built for the Anthropic ecosystem (Claude Code, Cursor, other agentic coding tools) work without modification.

Overview

The Messages API exposes two endpoints:

/v1/messages: Send messages and receive model-generated responses, with support for multi-turn conversations, tool use, streaming, and structured output.
/v1/messages/count_tokens: Count the number of tokens in a message before sending it, helping you manage context windows and costs.

Messages Endpoint

Endpoint

POST https://api.inference.nebul.io/v1/messages

Headers

Header	Required	Description
`x-api-key`	Yes	Your API key (or use `Authorization: Bearer <key>`).
`anthropic-version`	No	API version. The Anthropic SDKs send `2023-06-01`; requests without it are accepted.
`content-type`	Yes	Must be `application/json`.

Parameters

Parameter	Type	Required	Description
`model`	String	Yes	The model ID to use (e.g., `zai-org/GLM-5.3`).
`messages`	Array	Yes	Array of message objects with `role` and `content`. Roles: `user`, `assistant`.
`max_tokens`	Integer	Yes	Maximum number of tokens to generate in the response.
`system`	String or Array	No	System prompt. Can be a string or an array of content blocks.
`temperature`	Number	No	Sampling temperature (0.0–1.0). Defaults to `1.0`.
`top_p`	Number	No	Nucleus sampling parameter (0.0–1.0). Defaults to `1.0`.
`top_k`	Integer	No	Only sample from the top K options for each subsequent token.
`stop_sequences`	Array	No	Array of strings that will cause the model to stop generating.
`stream`	Boolean	No	Whether to stream responses. Defaults to `false`.
`tools`	Array	No	Array of tool definitions. See Function Calling & Tools.
`tool_choice`	Object	No	Tool choice strategy (`auto`, `any`, `tool`, `none`). See Function Calling & Tools.
`metadata`	Object	No	Optional metadata for the request.
`output_config`	Object	No	Output configuration including `effort` and `format` (JSON schema).

Code Examples

import os

import requests

url = "https://api.inference.nebul.io/v1/messages"
headers = {
    "x-api-key": os.environ["NEBUL_API_KEY"],
    "anthropic-version": "2023-06-01",
    "content-type": "application/json",
}

payload = {
    "model": "zai-org/GLM-5.3",
    "max_tokens": 1024,
    "system": "You are a helpful assistant.",
    "messages": [
        {
            "role": "user",
            "content": "What is the capital of France?"
        }
    ]
}

response = requests.post(url, headers=headers, json=payload)

for block in response.json()["content"]:
    if block["type"] == "text":
        print(block["text"])

Keep your key out of your code

Every example here reads the key from the environment. Treat sk- keys like passwords: anyone holding one can spend your project's budget.

Anthropic SDK

You can also use the official anthropic Python SDK by setting base_url to https://api.inference.nebul.io. See the Using with the Anthropic SDK section below.

Response Format

The API returns a JSON object with the following structure:

{
  "id": "chatcmpl-77126f46ae0ae1ec76857fb174d6747a",
  "type": "message",
  "role": "assistant",
  "content": [
    {
      "type": "thinking",
      "thinking": "The user is asking about the capital of France...",
      "signature": "8a93d07790aa4ff582dbfd045ec24f10"
    },
    {
      "type": "text",
      "text": "The capital of France is Paris."
    }
  ],
  "model": "zai-org/GLM-5.3",
  "stop_reason": "end_turn",
  "stop_sequence": null,
  "usage": {
    "input_tokens": 25,
    "output_tokens": 19
  }
}

Response Fields

Field	Type	Description
`id`	String	Unique identifier for the message.
`type`	String	Object type, always `"message"`.
`role`	String	Role of the response, always `"assistant"`.
`content`	Array	Array of content blocks (text, tool_use, thinking, etc.).
`model`	String	Model ID used for the response.
`stop_reason`	String	Reason for completion (`end_turn`, `max_tokens`, `stop_sequence`, `tool_use`).
`stop_sequence`	String	The stop sequence that triggered completion, if applicable.
`usage`	Object	Token usage statistics.
`usage.input_tokens`	Integer	Number of tokens in the input.
`usage.output_tokens`	Integer	Number of tokens in the output.

Message Roles

The messages array supports the following roles:

user: Represents the user's input or question. The first message must have the user role.
assistant: Represents the model's previous responses. Used for multi-turn conversations.

The system parameter is used separately (not as a message role) to set the assistant's behavior and context.

Content Block Types

Content in messages can be a simple string or an array of structured content blocks:

Text Content

{
  "role": "user",
  "content": "Hello, how are you?"
}

Or as a structured block:

{
  "role": "user",
  "content": [
    { "type": "text", "text": "Describe this image." }
  ]
}

Image Content

{
  "role": "user",
  "content": [
    {
      "type": "image",
      "source": {
        "type": "base64",
        "media_type": "image/jpeg",
        "data": "<base64-encoded-image-data>"
      }
    },
    { "type": "text", "text": "What is in this image?" }
  ]
}

Tool Use Content

When the model requests a tool call, the response contains a tool_use content block:

{
  "type": "tool_use",
  "id": "toolu_01A09q90qw90lq917835lq9",
  "name": "get_weather",
  "input": { "location": "San Francisco, CA" }
}

Tool results are sent back in a user message with a tool_result content block:

{
  "role": "user",
  "content": [
    {
      "type": "tool_result",
      "tool_use_id": "toolu_01A09q90qw90lq917835lq9",
      "content": "72°F, sunny"
    }
  ]
}

Thinking Content

Models with extended thinking enabled (e.g., reasoning models) return a thinking content block before the text response:

{
  "type": "thinking",
  "thinking": "Let me reason through this step by step...",
  "signature": "8a93d07790aa4ff582dbfd045ec24f10"
}

The signature field is an opaque verification hash. You can safely ignore thinking blocks if you only need the final text output.

Multi-turn Conversations

Include previous assistant responses in the messages array to maintain conversation context:

import os

import requests

url = "https://api.inference.nebul.io/v1/messages"
headers = {
    "x-api-key": os.environ["NEBUL_API_KEY"],
    "anthropic-version": "2023-06-01",
    "content-type": "application/json",
}

payload = {
    "model": "zai-org/GLM-5.3",
    "max_tokens": 1024,
    "messages": [
        {
            "role": "user",
            "content": "What is the capital of France?"
        },
        {
            "role": "assistant",
            "content": "The capital of France is Paris."
        },
        {
            "role": "user",
            "content": "What is its population?"
        }
    ]
}

response = requests.post(url, headers=headers, json=payload)
print(response.json())

Streaming Responses

Enable streaming by setting stream: true. The response will be sent as a series of Server-Sent Events (SSE) with event types: message_start, content_block_start, content_block_delta, content_block_stop, message_delta, message_stop, and ping.

Content block delta types include text_delta for text output and thinking_delta for reasoning model thinking output.

import os

import requests
import json

url = "https://api.inference.nebul.io/v1/messages"
headers = {
    "x-api-key": os.environ["NEBUL_API_KEY"],
    "anthropic-version": "2023-06-01",
    "content-type": "application/json",
}

payload = {
    "model": "zai-org/GLM-5.3",
    "max_tokens": 1024,
    "stream": True,
    "messages": [
        {
            "role": "user",
            "content": "Tell me a short story."
        }
    ]
}

response = requests.post(url, headers=headers, json=payload, stream=True)

for line in response.iter_lines():
    if line:
        line_text = line.decode('utf-8')
        if line_text.startswith('data: '):
            data = line_text[6:]
            try:
                event = json.loads(data)
                if event.get('type') == 'content_block_delta':
                    delta = event.get('delta', {})
                    if delta.get('type') == 'text_delta':
                        print(delta.get('text', ''), end='', flush=True)
                    elif delta.get('type') == 'thinking_delta':
                        pass  # Skip thinking output
            except json.JSONDecodeError:
                pass

Count Tokens Endpoint

The count tokens endpoint lets you determine the number of tokens in a message before sending it, helping you manage context windows, costs, and prompt optimization.

Endpoint

POST https://api.inference.nebul.io/v1/messages/count_tokens

Headers

Same as the Messages endpoint.

Parameters

Parameter	Type	Required	Description
`model`	String	Yes	The model ID to use for token counting.
`messages`	Array	Yes	Array of message objects (same format as the Messages endpoint).
`system`	String or Array	No	System prompt to include in the count.
`tools`	Array	No	Tool definitions to include in the count.
`tool_choice`	Object	No	Tool choice configuration to include in the count.

Code Examples

import os

import requests

url = "https://api.inference.nebul.io/v1/messages/count_tokens"
headers = {
    "x-api-key": os.environ["NEBUL_API_KEY"],
    "anthropic-version": "2023-06-01",
    "content-type": "application/json",
}

payload = {
    "model": "zai-org/GLM-5.3",
    "system": "You are a helpful assistant.",
    "messages": [
        {
            "role": "user",
            "content": "Hello, how are you?"
        }
    ]
}

response = requests.post(url, headers=headers, json=payload)
print(response.json())

Response Format

{
  "input_tokens": 11,
  "context_management": {
    "original_input_tokens": 11
  }
}

Field	Type	Description
`input_tokens`	Integer	Estimated number of tokens in the input messages, system prompt, and tools.
`context_management`	Object	Context management metadata.
`context_management.original_input_tokens`	Integer	Token count before any context window adjustments.

Token counts are estimates

The actual number of tokens used when creating a message may differ by a small amount.

Count Tokens with Tools

Token counting supports tool definitions, so you can measure the full prompt size before making a request:

import os

import requests

url = "https://api.inference.nebul.io/v1/messages/count_tokens"
headers = {
    "x-api-key": os.environ["NEBUL_API_KEY"],
    "anthropic-version": "2023-06-01",
    "content-type": "application/json",
}

payload = {
    "model": "zai-org/GLM-5.3",
    "tools": [
        {
            "name": "get_weather",
            "description": "Get the current weather in a given location",
            "input_schema": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA"
                    }
                },
                "required": ["location"]
            }
        }
    ],
    "messages": [
        {
            "role": "user",
            "content": "What is the weather in San Francisco?"
        }
    ]
}

response = requests.post(url, headers=headers, json=payload)
print(response.json())

Using with the Anthropic SDK

You can use the official Anthropic Python or TypeScript SDKs by pointing base_url to the Nebul Inference API:

import os

import anthropic

client = anthropic.Anthropic(
    api_key=os.environ["NEBUL_API_KEY"],
    base_url="https://api.inference.nebul.io",
)

message = client.messages.create(
    model="zai-org/GLM-5.3",
    max_tokens=1024,
    messages=[
        {"role": "user", "content": "What is the capital of France?"}
    ],
)

# Reasoning models return thinking blocks first; take the text blocks.
for block in message.content:
    if block.type == "text":
        print(block.text)

Count Tokens with the SDK

import os

import anthropic

client = anthropic.Anthropic(
    api_key=os.environ["NEBUL_API_KEY"],
    base_url="https://api.inference.nebul.io",
)

token_count = client.messages.count_tokens(
    model="zai-org/GLM-5.3",
    messages=[
        {"role": "user", "content": "Hello, how are you?"}
    ],
)

print(token_count)

Using with Claude Code

Claude Code can be configured to use the Nebul Inference API as its backend by setting environment variables:

ANTHROPIC_BASE_URL=https://api.inference.nebul.io \
ANTHROPIC_API_KEY=$NEBUL_API_KEY \
ANTHROPIC_AUTH_TOKEN=$NEBUL_API_KEY \
ANTHROPIC_DEFAULT_OPUS_MODEL=zai-org/GLM-5.3 \
ANTHROPIC_DEFAULT_SONNET_MODEL=zai-org/GLM-5.3 \
ANTHROPIC_DEFAULT_HAIKU_MODEL=zai-org/GLM-5.3 \
claude

For detailed setup instructions, see the Claude Code integration guide.

Model Specifications

The Messages API supports the same LLM models as the Chat Completions API; see the Model Catalog for the full list.

Key Differences from Chat Completions

Aspect	Chat Completions (`/v1/chat/completions`)	Messages (`/v1/messages`)
Compatibility	OpenAI API	Anthropic API
System prompt	`system` role in messages array	Separate `system` parameter
Required params	`model`, `messages`	`model`, `messages`, `max_tokens`
Stop sequences	`stop` parameter	`stop_sequences` parameter
Tool choice types	`auto`, `required`, `none`, specific tool	`auto`, `any`, `tool`, `none`
Authentication	`Authorization: Bearer`	`x-api-key` header
Token counting	Not available	`/v1/messages/count_tokens`
Thinking blocks	Not supported	`thinking` content blocks for reasoning models

Best Practices

Always set max_tokens: Unlike Chat Completions, max_tokens is required in the Messages API. Set an appropriate value for your use case.
Use system for instructions: Pass system instructions via the system parameter rather than as a message role.
Token counting: Use /v1/messages/count_tokens to preview prompt sizes and manage context windows before making requests.
Streaming: Use streaming for long responses to improve perceived latency.
Error Handling: Always check response status codes and handle errors appropriately.

On this page