Instrument AI Agents

With Sentry AI Agent Monitoring, you can monitor and debug your AI systems with full-stack context. You'll be able to track key insights like token usage, latency, tool usage, and error rates. AI Agent Monitoring data will be fully connected to your other Sentry data like logs, errors, and traces.

As a prerequisite to setting up AI Agent Monitoring with .NET, you'll need to first set up tracing. Once this is done, you can use the Sentry.Extensions.AI package to automatically instrument AI agents created with Microsoft.Extensions.AI.

Installation

Install the Sentry.Extensions.AI package:

dotnet add package Sentry.Extensions.AI

The Sentry.Extensions.AI integration depends on the Microsoft.Extensions.AI.Abstractions package (version 9.7.0 or higher).

Automatic Instrumentation

The Sentry.Extensions.AI package provides automatic instrumentation for AI agents built with Microsoft.Extensions.AI. This works with any AI provider that implements the IChatClient interface, including:

• Microsoft.Extensions.AI.OpenAI
• Microsoft.Extensions.AI.AzureAIInference
• Anthropic.SDK

Basic Setup

To instrument your AI agent, simply call IChatClient.AddSentry() before building your chat client. If your AI agent uses tools (function calling), you can instrument them using the AddSentryToolInstrumentation() extension method on ChatOptions:

// Wrap your IChatClient with Sentry instrumentation
var openAiClient = new OpenAI.Chat.ChatClient("gpt-4o-mini", apiKey)
    .AsIChatClient()
    .AddSentry(options =>
    {
        options.Experimental.RecordInputs = true;
        options.Experimental.RecordOutputs = true;
        options.Experimental.AgentName = "MyAgent";
    });

// Wrap your client with FunctionInvokingChatClient
var chatClient = new ChatClientBuilder(openAiClient)
    .UseFunctionInvocation()
    .Build();

// Create chat options with tools and add Sentry instrumentation
var options = new ChatOptions
{
    ModelId = "gpt-4o-mini",
    MaxOutputTokens = 1024,
    Tools =
    [
        // Sample Tool
        AIFunctionFactory.Create(async (string location) =>
        {
            await Task.Delay(500);
            return $"The weather in {location} is sunny";
        }, "GetWeather", "Gets the current weather for a location")
    ]
}.AddSentryToolInstrumentation();

var response = await chatClient.GetResponseAsync(
    "What's the weather in New York?",
    options);

Configuration Options

The AddSentry() method accepts an optional configuration delegate to customize the instrumentation: