Build a Local AI Agent with Tool Calling: Ollama + qwen3

A chat model can only predict the next token. It cannot query your database, call an API, or check the weather. That is the whole gap between a chatbot and an agent. Tool calling, also called function calling, is what closes it. The model does not run anything itself. It returns the name of a function and the arguments it wants; your code executes that function and feeds the result back. Repeat until the model stops asking.

Here is the part I like: you can run this loop entirely locally with Ollama and a small open model like qwen3. No API key, no network, no per-call cost. This guide builds that loop from scratch in Python, then shows where to go when the loop is no longer enough.

Prerequisites

• Ollama installed (from https://ollama.com/download)
• A model that supports tool calling, pulled locally:

ollama pull qwen3

• Python 3.9+ and the ollama SDK: pip install -U ollama

Step 1: Start Ollama and check the model

ollama serve

Confirm qwen3 responds:

ollama run qwen3 "say hi"

Ollama exposes an HTTP API on localhost:11434. Every request is a normal POST to /api/chat with a list of messages plus an optional tools array.

Step 2: Define a tool

A tool is just a function plus its signature. With the Python SDK you write a normal function with a docstring and pass it directly. Ollama converts the docstring and type hints into a JSON Schema the model understands.

from ollama import chat

def get_temperature(city: str) -> str:
    """Get the current temperature for a city.

    Args:
        city: The name of the city

    Returns:
        The current temperature for the city
    """
    temps = {"New York": "22C", "London": "15C", "Tokyo": "18C"}
    return temps.get(city, "Unknown")

messages = [{"role": "user", "content": "What is the temperature in New York?"}]
response = chat(model="qwen3", messages=messages, tools=[get_temperature], think=True)

Two things happen here that are easy to miss. First, when the model wants to use a tool, it does not write prose. It returns a message with a tool_calls field containing the tool name and the parsed arguments. Nothing has actually run yet. Second, the vendor SDKs (OpenAI, Anthropic) and the local one (Ollama) use different tool schemas, but the loop shape is identical everywhere: declare, decide, execute, feed back.

Step 3: Execute the tool and feed the result back

messages.append(response.message)
if response.message.tool_calls:
    call = response.message.tool_calls[0]
    result = get_temperature(**call.function.arguments)
    messages.append({"role": "tool", "tool_name": call.function.name, "content": str(result)})

    final = chat(model="qwen3", messages=messages, tools=[get_temperature], think=True)
    print(final.message.content)

The tool result is appended as a message with role tool, keyed by tool_name, so the model can read its own tool output. Then you call the model again and it writes the final answer grounded in that result.

Step 4: The full agent loop

A single tool result is one round trip. A real agent loops: the model may call several tools, in sequence or in parallel, until it has enough to answer. The control flow is just a while loop.

from ollama import chat, ChatResponse

def add(a: float, b: float) -> float:
    """Add two numbers. Args: a (first), b (second)."""
    return a + b

def multiply(a: float, b: float) -> float:
    """Multiply two numbers. Args: a (first), b (second)."""
    return a * b

available = {"add": add, "multiply": multiply}
messages = [{"role": "user", "content": "What is (11434+12341)*412?"}]

while True:
    response: ChatResponse = chat(model="qwen3", messages=messages, tools=[add, multiply], think=True)
    messages.append(response.message)
    if not response.message.tool_calls:
        break
    for tc in response.message.tool_calls:
        if tc.function.name in available:
            result = available[tc.function.name](**tc.function.arguments)
            messages.append({"role": "tool", "tool_name": tc.function.name, "content": str(result)})

print(messages[-1].content)

qwen3 also supports parallel tool calls. Need weather for two cities? Register both a temperature tool and a conditions tool; the model returns several entries in tool_calls in a single turn, you execute each, append every result, and call the model again.

Gotchas

• Not every model supports tool calling. qwen3, qwen2.5, llama3.1+, mistral, command-r, and granite work; older llama3 and gemma2 do not. Check the model page before pulling. A model that cannot handle tools does not error, it just hallucinates a JSON blob.
• Keep tool descriptions concrete. Vague ones make the model call tools it does not need.
• On a small model, keep the tool count low. Five or six well-described tools beat twenty fuzzy ones.
• Ollama keys the tool role by tool_name, not name. Some clients use OpenAI's field. If the loop silently ignores a result, check which field your results are keyed on.

When the loop is not enough

The while loop covers a single agent that calls tools. The moment you need a persistent graph with branches, human approval gates, or memory that survives a restart, reach for LangGraph. It models the same loop as an explicit graph of nodes and edges, with a checkpointer for state, and your tools stay identical. Ollama sits underneath as the model provider; LangGraph is the orchestration layer. The LangGraph quickstart shows this pattern.

Conclusion

You now have a working local agent: Ollama serves the tools, qwen3 decides, your code executes, and results feed back until the model answers. Everything runs on your machine, no API key. Next steps: add tools that actually touch your systems, say a SQLite-backed notes store or an HTTP endpoint, then move the loop into LangGraph when branching and persistence arrive.

References

• Ollama tool calling docs: https://docs.ollama.com/capabilities/tool-calling
• Ollama download: https://ollama.com/download
• LangGraph quickstart: https://docs.langchain.com/oss/python/langgraph/quickstart