Function Calling: Teaching Tools to the Assistant

This post is part of the Voice Assistant on Raspberry Pi series.

In article #5, we injected weather data into every conversation, even for questions like “what’s your name?”. That wastes tokens. Function calling fixes this: the LLM decides when it needs a tool and only calls it when the question actually warrants it.

The complete code for this article is available on GitHub.

Before (article #5):
  Weather injected into EVERY message → wasted tokens

After (article #6):
  "Should I bring a jacket today?" → LLM calls get_weather() → answers with real data
  "What's your name?"             → LLM answers directly, no tools

We’re in agent territory now.

How it works

The complete cycle:

1. Send messages + list of available tools to the LLM
2. The LLM decides:
   a. No tool needed → responds directly
   b. Tool needed → returns a tool_use block with parameters
3. We execute the tool locally
4. We send the result back to the LLM (tool_result)
5. The LLM generates the final response

The LLM doesn’t do anything itself, it requests. We execute, we return results.

Step 1: The ITool interface

Create Services/Tools/ITool.cs:

using System.Text.Json.Nodes;

namespace AudioAssistant.Services.Tools;

public interface ITool
{
    string Name { get; }
    string Description { get; }
    JsonObject GetParametersSchema();
    Task<string> ExecuteAsync(string input, CancellationToken cancellationToken);
}

Name and Description are sent to the LLM. The Description is what the LLM reads to decide when to call the tool. Write it carefully.

Step 2: WeatherTool

Create Services/Tools/WeatherTool.cs:

using System.Text.Json.Nodes;

namespace AudioAssistant.Services.Tools;

public class WeatherTool : ITool
{
    private readonly IWeatherService _weather;

    public WeatherTool(IWeatherService weather)
    {
        _weather = weather;
    }

    public string Name => "get_weather";

    public string Description =>
        "Returns current weather in Blainville. Call this tool when the user asks about " +
        "the weather, what to wear, whether to bring an umbrella, or planning outdoor activities.";

    public JsonObject GetParametersSchema() => new JsonObject
    {
        ["type"] = "object",
        ["properties"] = new JsonObject(),
        ["required"] = new JsonArray()
    };

    public Task<string> ExecuteAsync(string input, CancellationToken cancellationToken)
        => _weather.GetCurrentWeatherAsync(cancellationToken);
}

The tool delegates to the IWeatherService we built in article #5. No duplication.

Step 3: ToolRegistry

ToolRegistry centralizes all tools. Create Services/Tools/ToolRegistry.cs:

namespace AudioAssistant.Services.Tools;

public class ToolRegistry
{
    private readonly Dictionary<string, ITool> _tools = new();

    public ToolRegistry(IEnumerable<ITool> tools)
    {
        foreach (var tool in tools)
            _tools[tool.Name] = tool;
    }

    public IEnumerable<ITool> GetAll() => _tools.Values;

    public ITool? GetByName(string name) =>
        _tools.TryGetValue(name, out var tool) ? tool : null;
}

Step 4: Update ClaudeService

This is the core piece: the tool_use, execution, tool_result cycle. Anthropic.SDK 5.10 uses Function and the CommonTool alias to avoid the ambiguity between Anthropic.SDK.Common.Tool and Anthropic.SDK.Messaging.Tool.

using Anthropic.SDK;
using Anthropic.SDK.Common;
using Anthropic.SDK.Messaging;
using Microsoft.Extensions.Options;
using AudioAssistant.Services.Tools;
using CommonTool = Anthropic.SDK.Common.Tool;

namespace AudioAssistant.Services;

public class ClaudeService : ILlmService
{
    private readonly AssistantOptions _options;
    private readonly ToolRegistry _toolRegistry;
    private readonly ILogger<ClaudeService> _logger;

    public ClaudeService(
        IOptions<AssistantOptions> options,
        ToolRegistry toolRegistry,
        ILogger<ClaudeService> logger)
    {
        _options = options.Value;
        _toolRegistry = toolRegistry;
        _logger = logger;
    }

    public async Task<string> ChatAsync(List<ConversationMessage> history, CancellationToken cancellationToken)
    {
        _logger.LogInformation("Sending to Claude ({Count} messages)...", history.Count);

        var client = new AnthropicClient(_options.ClaudeApiKey);

        var systemPrompt = history
            .FirstOrDefault(m => m.Role == "system")?.Content ?? "";

        var messages = history
            .Where(m => m.Role != "system")
            .Select(m => new Message
            {
                Role = m.Role == "user" ? RoleType.User : RoleType.Assistant,
                Content = new List<ContentBase> { new TextContent { Text = m.Content } }
            })
            .ToList();

        IList<CommonTool> tools = _toolRegistry.GetAll()
            .Select(t => new Function(t.Name, t.Description, t.GetParametersSchema()))
            .Select(f => (CommonTool)f)
            .ToList();

        var parameters = new MessageParameters
        {
            Model = _options.ClaudeModel,
            MaxTokens = 512,
            System = new List<SystemMessage> { new SystemMessage(systemPrompt) },
            Messages = messages,
            Tools = tools
        };

        var response = await client.Messages.GetClaudeMessageAsync(parameters, cancellationToken);

        if (response.StopReason == "tool_use")
        {
            var toolUse = response.Content.OfType<ToolUseContent>().FirstOrDefault();
            if (toolUse != null)
            {
                _logger.LogInformation("Claude calling tool: {Tool}", toolUse.Name);

                var tool = _toolRegistry.GetByName(toolUse.Name);
                if (tool != null)
                {
                    var toolResult = await tool.ExecuteAsync(
                        toolUse.Input?.ToString() ?? "", cancellationToken);

                    _logger.LogInformation("Tool result: {Result}", toolResult);

                    messages.Add(new Message
                    {
                        Role = RoleType.Assistant,
                        Content = response.Content
                    });

                    messages.Add(new Message
                    {
                        Role = RoleType.User,
                        Content = new List<ContentBase>
                        {
                            new ToolResultContent
                            {
                                ToolUseId = toolUse.Id,
                                Content = new List<ContentBase>
                                {
                                    new TextContent { Text = toolResult }
                                }
                            }
                        }
                    });

                    var finalResponse = await client.Messages.GetClaudeMessageAsync(
                        new MessageParameters
                        {
                            Model = _options.ClaudeModel,
                            MaxTokens = 512,
                            System = new List<SystemMessage> { new SystemMessage(systemPrompt) },
                            Messages = messages,
                            Tools = tools
                        }, cancellationToken);

                    var finalText = finalResponse.Content.OfType<TextContent>()
                        .FirstOrDefault()?.Text?.Trim() ?? "I don't have a response.";

                    _logger.LogInformation("Claude final response: \"{Text}\"", finalText);
                    return finalText;
                }
            }
        }

        var text = response.Content.OfType<TextContent>()
            .FirstOrDefault()?.Text?.Trim() ?? "I don't have a response.";

        _logger.LogInformation("Claude response: \"{Text}\"", text);
        return text;
    }
}

Step 5: Simplify ContextService

Weather is no longer injected at startup. InitializeAsync becomes a synchronous Initialize, and the constructor no longer needs IWeatherService:

using Microsoft.Extensions.Options;

namespace AudioAssistant.Services;

public interface IContextService
{
    void Initialize();
    List<ConversationMessage> AddUserMessage(string userInput);
    void AddAssistantMessage(string response);
    void Reset();
}

public class ContextService : IContextService
{
    private readonly AssistantOptions _options;
    private readonly List<ConversationMessage> _history = new();
    private readonly ILogger<ContextService> _logger;

    public ContextService(IOptions<AssistantOptions> options, ILogger<ContextService> logger)
    {
        _options = options.Value;
        _logger = logger;
    }

    public void Initialize()
    {
        _history.Clear();
        _history.Add(new ConversationMessage("system", _options.SystemPrompt));
        _logger.LogInformation("Context initialized.");
    }

    public List<ConversationMessage> AddUserMessage(string userInput)
    {
        _history.Add(new ConversationMessage("user", userInput));
        _logger.LogInformation("History: {Count} messages", _history.Count);
        return _history;
    }

    public void AddAssistantMessage(string response)
    {
        _history.Add(new ConversationMessage("assistant", response));
    }

    public void Reset()
    {
        _history.Clear();
        _logger.LogInformation("History reset.");
    }
}

Step 6: Update Worker.cs

await _context.InitializeAsync() becomes _context.Initialize():

protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
    _logger.LogInformation("Assistant started. Initializing context...");
    _context.Initialize();
    _logger.LogInformation("Press the button to speak.");

    while (!stoppingToken.IsCancellationRequested)
    {
        _gpio.WaitForButtonPress(stoppingToken);
        if (stoppingToken.IsCancellationRequested) break;

        try
        {
            var audioFile = await _recorder.RecordAsync(stoppingToken);
            var text = await _transcription.TranscribeAsync(audioFile, stoppingToken);

            if (string.IsNullOrWhiteSpace(text))
            {
                await _speech.SpeakAsync("Je n'ai pas bien entendu. Peux-tu répéter?", stoppingToken);
            }
            else
            {
                var history = _context.AddUserMessage(text);
                var response = await _llm.ChatAsync(history, stoppingToken);
                _context.AddAssistantMessage(response);
                await _speech.SpeakAsync(response, stoppingToken);
            }

            if (File.Exists(audioFile))
                File.Delete(audioFile);
        }
        catch (Exception ex) when (!stoppingToken.IsCancellationRequested)
        {
            _logger.LogError(ex, "Error in pipeline");
            await _speech.SpeakAsync("Une erreur s'est produite.", stoppingToken);
        }
    }
}

Step 7: Update Program.cs

using AudioAssistant;
using AudioAssistant.Services;
using AudioAssistant.Services.Tools;
using Microsoft.Extensions.Options;

var builder = Host.CreateApplicationBuilder(args);

builder.Services.Configure<AssistantOptions>(
    builder.Configuration.GetSection("Assistant"));

builder.Services.AddSingleton<IGpioService, GpioService>();
builder.Services.AddSingleton<IAudioRecorderService, AudioRecorderService>();
builder.Services.AddSingleton<ITranscriptionService, WhisperTranscriptionService>();
builder.Services.AddSingleton<ISpeechService, PiperSpeechService>();
builder.Services.AddSingleton<IContextService, ContextService>();

builder.Services.AddHttpClient<IWeatherService, WeatherService>();

builder.Services.AddSingleton<ITool, WeatherTool>();
builder.Services.AddSingleton<ToolRegistry>();

builder.Services.AddHttpClient<OllamaService>(client =>
{
    client.Timeout = TimeSpan.FromSeconds(120);
});
builder.Services.AddTransient<ClaudeService>();

builder.Services.AddTransient<ILlmService>(sp =>
{
    var options = sp.GetRequiredService<IOptions<AssistantOptions>>().Value;
    return options.LlmProvider.Equals("claude", StringComparison.OrdinalIgnoreCase)
        ? (ILlmService)sp.GetRequiredService<ClaudeService>()
        : sp.GetRequiredService<OllamaService>();
});

builder.Services.AddHostedService<Worker>();

var host = builder.Build();
host.Run();

Testing

Run dotnet run and compare two kinds of questions:

You : "Should I take the kids outside today?"
Logs: Claude calling tool: get_weather
Logs: Tool result: Partly cloudy, 12.3°C...
Alex: "Yes, it's a nice day to get outside!"

You : "What's your name?"
Logs: (no tool calls)
Alex: "My name is Alex!"

The LLM is the orchestrator. You don’t tell it when to call what, it figures that out.

Adding more tools

The strength of this pattern: a new tool is one file and one line in Program.cs. Nothing else changes.

Create Services/Tools/TimeTool.cs:

using System.Text.Json.Nodes;

namespace AudioAssistant.Services.Tools;

public class TimeTool : ITool
{
    public string Name => "get_current_time";

    public string Description =>
        "Returns the current date and time. Call this tool when the user asks for " +
        "the time, date, day of the week, or month.";

    public JsonObject GetParametersSchema() => new JsonObject
    {
        ["type"] = "object",
        ["properties"] = new JsonObject(),
        ["required"] = new JsonArray()
    };

    public Task<string> ExecuteAsync(string input, CancellationToken cancellationToken)
    {
        var now = DateTime.Now;
        return Task.FromResult($"{now:dddd, MMMM d yyyy}, {now:HH:mm}");
    }
}

builder.Services.AddSingleton<ITool, WeatherTool>();
builder.Services.AddSingleton<ITool, TimeTool>();
builder.Services.AddSingleton<ToolRegistry>();

Done. The assistant now knows the time without any changes to ClaudeService or Worker.

What about Ollama?

OllamaService hasn’t been updated to support tools. llama3.2:1b technically supports function calling, but reliability is inconsistent. For now, Ollama stays the “free local mode without tools” and Claude handles “production quality with tools.” If you want to experiment with a local model, mistral:7b is more reliable for tool use, but it pushes the limits of a 4 GB Pi.