Skip to main content

Function Calling & Tools

Let models decide when to call external functions and with what arguments. Your code executes the function and feeds the result back.

Understanding "function calling"

LLMs do not execute functions. They analyze the conversation and emit a structured request describing which function to call and with what parameters. Your code parses that request, executes the function, and optionally returns the result so the model can continue. Models marked with the tools capability in the Model Catalog are fine-tuned for this and expose a dedicated tools parameter; prompting a non-tool model to emit function calls is unreliable by comparison.

For background, see Hugging Face's guide on function calling.

Supported models​

Tool use is model-specific. See the Model Catalog for the current set of tool-capable models. The catalog is the source of truth and is kept up to date as models are added or retired.

Model-specific behavior

Function calling behavior varies between models (argument formatting, parallel tool calls, etc.). Consult the model's documentation on Hugging Face for specifics.

Defining a function​

A function is described with a JSON schema containing:

  • Name: Unique identifier for the function
  • Description: What the function does (helps the model decide when to use it)
  • Parameters: Input schema with types and descriptions
python
123456789101112131415161718192021222324252627282930313233343536373839404142
from openai import OpenAI


client = OpenAI(
    api_key="sk-your-api-key-here",
    base_url="https://api.inference.nebul.io/v1"
)


tools = [
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Get the current weather in a given location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA"
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "The unit of temperature to use"
                    }
                },
                "required": ["location"]
            }
        }
    }
]


response = client.chat.completions.create(
    model="openai/gpt-oss-120b",
    messages=[
        {"role": "user", "content": "What's the weather like in Boston?"}
    ],
    tools=tools,
    tool_choice="auto"  # Let the model decide when to use tools
)


print(response.choices[0].message.tool_calls)

Handling Function Calls​

When the model decides to use a function, it returns a tool_calls object instead of a text response. The format depends on whether you are using the Python SDK or the raw API:

The Python SDK returns a list of objects (not a dictionary):

python
12345678910
[
    ChatCompletionMessageToolCall(
        id='call_abc123',
        function=Function(
            arguments='{"location": "Boston, MA", "unit": "fahrenheit"}',
            name='get_current_weather'
        ),
        type='function'
    )
]

You then:

  1. Execute the function with the provided arguments
  2. Return the result to the model
  3. Get the final response

Complete Flow Example​

python
1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071727374757677787980818283
import json
from openai import OpenAI


client = OpenAI(
    api_key="sk-your-api-key-here",
    base_url="https://api.inference.nebul.io/v1"
)


# Define your function implementation
def get_current_weather(location: str, unit: str = "fahrenheit"):
    # In production, call a real weather API
    return {
        "location": location,
        "temperature": 72,
        "unit": unit,
        "conditions": "sunny"
    }


# Define the function schema
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Get the current weather in a given location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA"
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"]
                    }
                },
                "required": ["location"]
            }
        }
    }
]


messages = [
    {"role": "user", "content": "What's the weather in Boston and New York?"}
]


# First API call
response = client.chat.completions.create(
    model="openai/gpt-oss-120b",
    messages=messages,
    tools=tools,
    tool_choice="auto"
)


response_message = response.choices[0].message
messages.append(response_message)


# Check if the model wants to call functions
if response_message.tool_calls:
    for tool_call in response_message.tool_calls:
        function_name = tool_call.function.name
        function_args = json.loads(tool_call.function.arguments)


        # Call the function
        if function_name == "get_current_weather":
            function_response = get_current_weather(**function_args)


        # Add function response to messages
        messages.append({
            "tool_call_id": tool_call.id,
            "role": "tool",
            "name": function_name,
            "content": json.dumps(function_response)
        })


    # Get final response from model
    final_response = client.chat.completions.create(
        model="openai/gpt-oss-120b",
        messages=messages
    )


    print(final_response.choices[0].message.content)

Multiple Functions​

You can provide multiple functions and the model will choose the appropriate one(s):

python
1234567891011121314151617181920212223242526272829303132
tools = [
    {
        "type": "function",
        "function": {
            "name": "search_products",
            "description": "Search for products in the database",
            "parameters": {
                "type": "object",
                "properties": {
                    "query": {"type": "string"},
                    "category": {"type": "string"},
                    "max_price": {"type": "number"}
                },
                "required": ["query"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "get_product_details",
            "description": "Get detailed information about a specific product",
            "parameters": {
                "type": "object",
                "properties": {
                    "product_id": {"type": "string"}
                },
                "required": ["product_id"]
            }
        }
    }
]

Tool Choice Options​

Control when the model uses functions:

  • auto (default): Model decides whether to call functions
  • none: Forces model to only generate text responses
  • required: Forces model to call at least one function
  • {"type": "function", "function": {"name": "function_name"}}: Forces a specific function

Best Practices​

  1. Clear descriptions: Write detailed descriptions that help the model understand when to use each function.
  2. Required parameters: Mark essential parameters as required.
  3. Validation: Validate function arguments before execution.
  4. Error handling: Return meaningful error messages to the model.
  5. Security: Never execute arbitrary code. Only call predefined, safe functions.
  6. Timeouts: Implement timeouts for external API calls.
Last updated on