TypeSafe for Nitro
Adds instrumentation for TypeSafe Jev evaluations. (default)
Requires Sentry SDK version 11.7.0 or higher.
Import name: Sentry.typesafeIntegration
This integration is enabled by default when tracing is enabled. To customize it, see Modifying Default Integrations.
The typesafeIntegration adds instrumentation for @typesafe-ai/sdk to capture agent tracing gen_ai.decide spans, recording model, token usage, latency, and (when enabled) decision inputs and answers.
If you call Jev through Vercel AI SDK's experimental_decide(), see the Vercel AI integration instead. If you call Jev through Cloudflare Workers AI, see Workers AI. If you call Jev through a Mastra Classifier, see Mastra Classifier Evaluations.
If you call Jev through LangChain's TypeSafeClassifier, including from LangGraph nodes and the @langchain/typesafe middleware, see the LangChain integration.
Sentry.init({
dsn: "___PUBLIC_DSN___",
// Tracing must be enabled for agent tracing to work
tracesSampleRate: 1.0,
integrations: [Sentry.typesafeIntegration()],
});
Sentry.init({
dsn: "___PUBLIC_DSN___",
// Tracing must be enabled for agent tracing to work
tracesSampleRate: 1.0,
integrations: [Sentry.typesafeIntegration()],
});
Tracing support is added to TypeSafeClient.systemOne() calls to evaluate state against a set of questions using Jev. Each call creates a gen_ai.decide span named decide <model>. Recorded answers include probabilities and confidence values when returned by TypeSafe.
@typesafe-ai/sdk:>=0.5.0 <1
Evaluation state and answers can contain sensitive data. Input and output recording are enabled by default. To disable them for TypeSafe, configure the integration explicitly:
Sentry.init({
dsn: "___PUBLIC_DSN___",
tracesSampleRate: 1.0,
integrations: [
Sentry.typesafeIntegration({
recordInputs: false,
recordOutputs: false,
}),
],
});
Sentry.init({
dsn: "___PUBLIC_DSN___",
tracesSampleRate: 1.0,
integrations: [
Sentry.typesafeIntegration({
recordInputs: false,
recordOutputs: false,
}),
],
});
Type: boolean (optional)
Records evaluation state and questions. Defaults to dataCollection.genAI.inputs, which is true by default.
Type: boolean (optional)
Records evaluation answers. Defaults to dataCollection.genAI.outputs, which is true by default.
Import name: Sentry.instrumentTypeSafeClient
If your application does not use automatic instrumentation, wrap your TypeSafe client after setting up Sentry. Use the returned client for your systemOne() calls.
import * as Sentry from "___SDK_PACKAGE___";
import * as Sentry from "___SDK_PACKAGE___";
import { TypeSafeClient, choice } from "@typesafe-ai/sdk";
const typesafe = new TypeSafeClient({ apiKey: "your-typesafe-api-key" });
const client = Sentry.instrumentTypeSafeClient(typesafe, {
recordInputs: false,
recordOutputs: false,
});
const result = await client.systemOne({
state: "I was charged twice. Please refund the extra charge.",
questions: {
department: choice("Which team should handle this?", {
billing: "Charges and refunds",
technical: "Bugs and outages",
}),
},
});
import { TypeSafeClient, choice } from "@typesafe-ai/sdk";
const typesafe = new TypeSafeClient({ apiKey: "your-typesafe-api-key" });
const client = Sentry.instrumentTypeSafeClient(typesafe, {
recordInputs: false,
recordOutputs: false,
});
const result = await client.systemOne({
state: "I was charged twice. Please refund the extra charge.",
questions: {
department: choice("Which team should handle this?", {
billing: "Charges and refunds",
technical: "Bugs and outages",
}),
},
});
Use one instrumentation approach for each client. If automatic instrumentation is already active, you don't need to wrap the client.
Our documentation is open source and available on GitHub. Your contributions are welcome, whether fixing a typo (drat!) or suggesting an update ("yeah, this would be better").