Function Calling & Tools

Let models call external functions with structured arguments, and feed the results back into the conversation

Let models decide when to call external functions and with what arguments. Your code executes the function and feeds the result back.

Understanding "function calling"

LLMs do not execute functions. They analyze the conversation and emit a structured request describing which function to call and with what parameters. Your code parses that request, executes the function, and optionally returns the result so the model can continue. Models marked with the tools capability in the Model Catalog are fine-tuned for this and expose a dedicated tools parameter; prompting a non-tool model to emit function calls is unreliable by comparison.

For background, see Hugging Face's guide on function calling.

Supported models

Tool use is model-specific. See the Model Catalog for the current set of tool-capable models. The catalog is the source of truth and is kept up to date as models are added or retired.

Model-specific behavior

Function calling behavior varies between models (argument formatting, parallel tool calls, etc.). Consult the model's documentation on Hugging Face for specifics.

Defining a function

A function is described with a JSON schema containing:

Name: Unique identifier for the function
Description: What the function does (helps the model decide when to use it)
Parameters: Input schema with types and descriptions

from openai import OpenAI
import os

client = OpenAI(
    api_key=os.environ["NEBUL_API_KEY"],
    base_url="https://api.inference.nebul.io/v1"
)

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Get the current weather in a given location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA"
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "The unit of temperature to use"
                    }
                },
                "required": ["location"]
            }
        }
    }
]

response = client.chat.completions.create(
    model="zai-org/GLM-5.3",
    messages=[
        {"role": "user", "content": "What's the weather like in Boston?"}
    ],
    tools=tools,
    tool_choice="auto"  # Let the model decide when to use tools
)

print(response.choices[0].message.tool_calls)

Handling Function Calls

When the model decides to use a function, it returns a tool_calls object instead of a text response. The format depends on whether you are using the Python SDK or the raw API:

The Python SDK returns a list of objects (not a dictionary):

[
    ChatCompletionMessageToolCall(
        id='call_abc123',
        function=Function(
            arguments='{"location": "Boston, MA", "unit": "fahrenheit"}',
            name='get_current_weather'
        ),
        type='function'
    )
]

You then:

Execute the function with the provided arguments
Return the result to the model
Get the final response

Complete Flow Example

import json
from openai import OpenAI
import os

client = OpenAI(
    api_key=os.environ["NEBUL_API_KEY"],
    base_url="https://api.inference.nebul.io/v1"
)

# Define your function implementation
def get_current_weather(location: str, unit: str = "fahrenheit"):
    # In production, call a real weather API
    return {
        "location": location,
        "temperature": 72,
        "unit": unit,
        "conditions": "sunny"
    }

# Define the function schema
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Get the current weather in a given location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA"
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"]
                    }
                },
                "required": ["location"]
            }
        }
    }
]

messages = [
    {"role": "user", "content": "What's the weather in Boston and New York?"}
]

# First API call
response = client.chat.completions.create(
    model="zai-org/GLM-5.3",
    messages=messages,
    tools=tools,
    tool_choice="auto"
)

response_message = response.choices[0].message
messages.append(response_message)

# Check if the model wants to call functions
if response_message.tool_calls:
    for tool_call in response_message.tool_calls:
        function_name = tool_call.function.name
        function_args = json.loads(tool_call.function.arguments)

        # Call the function
        if function_name == "get_current_weather":
            function_response = get_current_weather(**function_args)

        # Add function response to messages
        messages.append({
            "tool_call_id": tool_call.id,
            "role": "tool",
            "name": function_name,
            "content": json.dumps(function_response)
        })

    # Get final response from model
    final_response = client.chat.completions.create(
        model="zai-org/GLM-5.3",
        messages=messages
    )

    print(final_response.choices[0].message.content)

Multiple Functions

You can provide multiple functions and the model will choose the appropriate one(s):

tools = [
    {
        "type": "function",
        "function": {
            "name": "search_products",
            "description": "Search for products in the database",
            "parameters": {
                "type": "object",
                "properties": {
                    "query": {"type": "string"},
                    "category": {"type": "string"},
                    "max_price": {"type": "number"}
                },
                "required": ["query"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "get_product_details",
            "description": "Get detailed information about a specific product",
            "parameters": {
                "type": "object",
                "properties": {
                    "product_id": {"type": "string"}
                },
                "required": ["product_id"]
            }
        }
    }
]

Tool Choice Options

Control when the model uses functions:

auto (default): Model decides whether to call functions
none: Forces model to only generate text responses

Parallel Tool Calls

Models can return multiple tool calls in one response. Ask for several facts at once and a capable model fans out:

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather for a city",
            "parameters": {
                "type": "object",
                "properties": {"city": {"type": "string"}},
                "required": ["city"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "get_time",
            "description": "Get current time in a city",
            "parameters": {
                "type": "object",
                "properties": {"city": {"type": "string"}},
                "required": ["city"],
            },
        },
    },
]

response = client.chat.completions.create(
    model="zai-org/GLM-5.3",
    messages=[
        {
            "role": "user",
            "content": "Get the weather and time for both Paris and Rome.",
        }
    ],
    tools=tools,
)

for call in response.choices[0].message.tool_calls:
    print(call.function.name, call.function.arguments)

This one request returns four calls in a single tool_calls array:

get_weather {"city": "Paris"}
get_weather {"city": "Rome"}
get_time {"city": "Paris"}
get_time {"city": "Rome"}

Execute all of them, append one tool result message per call (keep the tool_call_id matched), and send the conversation back in a single follow-up request. Your loop should assume any number of calls per turn, including zero: some models call tools sequentially instead of in parallel, so the model may ask for the next tool only after you return the previous result.

Streaming Tool Calls

stream: true works with tools. The reasoning or text deltas stream as usual, and the tool calls arrive in a delta chunk with finish_reason: "tool_calls":

stream = client.chat.completions.create(
    model="zai-org/GLM-5.3",
    stream=True,
    messages=[{"role": "user", "content": "What's the weather in Paris?"}],
    tools=tools,
)

for chunk in stream:
    delta = chunk.choices[0].delta
    if delta.tool_calls:
        for call in delta.tool_calls:
            print(call.function.name, call.function.arguments)

Depending on the model, the arguments arrive complete in one chunk rather than incrementally. Aggregate until finish_reason is "tool_calls", then execute.

Best Practices

Clear descriptions: Write detailed descriptions that help the model understand when to use each function. Say what the function does, what each parameter means, and when not to use it.
Use enums and structure: Constrain parameters with enum values and nested objects instead of free-form strings. toggle_light(on: bool, off: bool) invites invalid calls; a single state enum does not.
Don't make the model fill what you already know: If your application knows the order_id, don't ask the model to pass it back. Fewer parameters means fewer failure modes.
Keep the tool list small: Function definitions are injected into the prompt and billed as input tokens. A handful of well-described tools beats a large grab bag.
Validation: Validate function arguments before execution.
Error handling: Return meaningful error messages to the model so it can correct itself on the next turn.
Security: Never execute arbitrary code. Only call predefined, safe functions.
Timeouts: Implement timeouts for external API calls.

Function Calling & Tools

On this page