Skip to main content

Function Calling & Tools

Enable AI models to call external functions and interact with APIs, databases, and other services. Function calling allows models to determine when to use specific tools and provide structured arguments.

Understanding "Function Calling"

The term "function calling" can be misleading—LLMs don't actually execute any functions. Instead, they act as the decision-maker: analyzing the conversation and determining which function should be called with what parameters. Your code then parses this output, executes the actual function, and optionally feeds the result back to the LLM for further reasoning. This loop is what makes systems "agentic".

While any LLM can technically be prompted to output structured function calls, modern models are specifically fine-tuned for this task and expose dedicated parameters (like tools) that significantly improve reliability. Those models are marked with the 'tools' capability in our UI.

For a deeper dive, see Huggingface's guide on function calling.

Overview​

Function calling enables models to:

  • Call external APIs and retrieve real-time data
  • Interact with databases and internal systems
  • Execute calculations and data transformations
  • Trigger workflows and automations
  • Use multiple tools in a single conversation
Model-specific behavior

Function calling behavior can vary between models (e.g., argument formatting, parallel tool calls). Consult the model's official documentation on Huggingface.

Defining a Function​

Functions are defined using a JSON schema that describes:

  • Name: Unique identifier for the function
  • Description: What the function does (helps the model decide when to use it)
  • Parameters: Input schema with types and descriptions
python
123456789101112131415161718192021222324252627282930313233343536373839404142
from openai import OpenAI


client = OpenAI(
    api_key="sk-your-api-key-here",
    base_url="https://api.inference.nebul.io/v1"
)


tools = [
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Get the current weather in a given location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA"
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "The unit of temperature to use"
                    }
                },
                "required": ["location"]
            }
        }
    }
]


response = client.chat.completions.create(
    model="Qwen/Qwen3-VL-235B-A22B-Thinking",
    messages=[
        {"role": "user", "content": "What's the weather like in Boston?"}
    ],
    tools=tools,
    tool_choice="auto"  # Let the model decide when to use tools
)


print(response.choices[0].message.tool_calls)

Handling Function Calls​

When the model decides to use a function, it returns a tool_calls object instead of a text response. The format depends on whether you are using the Python SDK or the raw API:

The Python SDK returns a list of objects (not a dictionary):

python
12345678910
[
    ChatCompletionMessageToolCall(
        id='call_abc123',
        function=Function(
            arguments='{"location": "Boston, MA", "unit": "fahrenheit"}',
            name='get_current_weather'
        ),
        type='function'
    )
]

You then:

  1. Execute the function with the provided arguments
  2. Return the result to the model
  3. Get the final response

Complete Flow Example​

python
1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071727374757677787980818283
import json
from openai import OpenAI


client = OpenAI(
    api_key="sk-your-api-key-here",
    base_url="https://api.inference.nebul.io/v1"
)


# Define your function implementation
def get_current_weather(location: str, unit: str = "fahrenheit"):
    # In production, call a real weather API
    return {
        "location": location,
        "temperature": 72,
        "unit": unit,
        "conditions": "sunny"
    }


# Define the function schema
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Get the current weather in a given location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA"
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"]
                    }
                },
                "required": ["location"]
            }
        }
    }
]


messages = [
    {"role": "user", "content": "What's the weather in Boston and New York?"}
]


# First API call
response = client.chat.completions.create(
    model="Qwen/Qwen3-VL-235B-A22B-Thinking",
    messages=messages,
    tools=tools,
    tool_choice="auto"
)


response_message = response.choices[0].message
messages.append(response_message)


# Check if the model wants to call functions
if response_message.tool_calls:
    for tool_call in response_message.tool_calls:
        function_name = tool_call.function.name
        function_args = json.loads(tool_call.function.arguments)


        # Call the function
        if function_name == "get_current_weather":
            function_response = get_current_weather(**function_args)


        # Add function response to messages
        messages.append({
            "tool_call_id": tool_call.id,
            "role": "tool",
            "name": function_name,
            "content": json.dumps(function_response)
        })


    # Get final response from model
    final_response = client.chat.completions.create(
        model="Qwen/Qwen3-VL-235B-A22B-Thinking",
        messages=messages
    )


    print(final_response.choices[0].message.content)

Multiple Functions​

You can provide multiple functions and the model will choose the appropriate one(s):

python
1234567891011121314151617181920212223242526272829303132
tools = [
    {
        "type": "function",
        "function": {
            "name": "search_products",
            "description": "Search for products in the database",
            "parameters": {
                "type": "object",
                "properties": {
                    "query": {"type": "string"},
                    "category": {"type": "string"},
                    "max_price": {"type": "number"}
                },
                "required": ["query"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "get_product_details",
            "description": "Get detailed information about a specific product",
            "parameters": {
                "type": "object",
                "properties": {
                    "product_id": {"type": "string"}
                },
                "required": ["product_id"]
            }
        }
    }
]

Tool Choice Options​

Control when the model uses functions:

  • auto (default): Model decides whether to call functions
  • none: Forces model to only generate text responses
  • required: Forces model to call at least one function
  • {"type": "function", "function": {"name": "function_name"}}: Forces a specific function

Best Practices​

  1. Clear descriptions: Write detailed descriptions that help the model understand when to use each function
  2. Required parameters: Mark essential parameters as required
  3. Validation: Validate function arguments before execution
  4. Error handling: Return meaningful error messages to the model
  5. Security: Never execute arbitrary code - only call predefined, safe functions
  6. Timeouts: Implement timeouts for external API calls

Supported Models​

Function calling is currently available on:

  • Qwen/Qwen3-VL-235B-A22B-Thinking
  • openai/gpt-oss-120b
  • meta-llama/Llama-4-Maverick-17B-128E-Instruct

Check the Models page for the latest compatibility information.

Last updated on