Vercel AI SDK

Learn how to integrate a Vercel AI SDK project with Galileo using OpenTelemetry

Galileo supports logging traces from Vercel AI SDK applications using OpenTelemetry.

Set up OpenTelemetry

To log Vercel AI SDK applications using Galileo, the first step is to set up OpenTelemetry.

1. Add the OpenTelemetry packages to your application

   Add the OpenTelemetry NPM packages to your Vercel AI application project.

   Using GalileoSpanProcessor

   Terminal

   npm install galileo \
         @opentelemetry/sdk-node \
         @opentelemetry/sdk-trace-node
   

   Manual OTLP configuration

   Terminal

   npm i @opentelemetry/sdk-node \
         @opentelemetry/exporter-trace-otlp-http
   
2. Create environment variables for your Galileo settings

   Set environment variables for your Galileo settings, for example in a .env file:

   Using GalileoSpanProcessor

   .env

   # Your Galileo API key
   GALILEO_API_KEY="your-galileo-api-key"
   
   # Your Galileo project name
   GALILEO_PROJECT="your-galileo-project-name"
   
   # The name of the Log stream you want to use for logging
   GALILEO_LOG_STREAM="your-galileo-log-stream "
   
   # Provide the console url below if you are using a
   # custom deployment, and not using the free tier, or app.galileo.ai.
   # This will look something like “console.galileo.yourcompany.com”.
   # GALILEO_CONSOLE_URL="your-galileo-console-url"
   

   Manual OTLP configuration

   .env

   # Your Galileo API key
   GALILEO_API_KEY="your-galileo-api-key"
   
   # Your Galileo project name
   GALILEO_PROJECT="your-galileo-project-name"
   
   # The name of the Log stream you want to use for logging
   GALILEO_LOG_STREAM="your-galileo-log-stream "
   
3. Get your endpoint

   The OTel endpoint is different from Galileo’s regular API endpoint and is specifically designed to receive telemetry data in the OTLP format.

   If you are using:

   • Galileo Cloud at app.galileo.ai, then you don’t need to provide a custom OTel endpoint. The default endpoint https://api.galileo.ai/otel/traces will be used automatically.

   • A self-hosted Galileo deployment, replace the https://api.galileo.ai/otel/traces endpoint with your deployment URL. The format of this URL is based on your console URL, replacing console with api and appending /otel/traces.

   For example:

   • if your console URL is https://console.galileo.example.com, the OTel endpoint would be https://api.galileo.example.com/otel/traces
   • if your console URL is https://console-galileo.apps.mycompany.com, the OTel endpoint would be https://api-galileo.apps.mycompany.com/otel/traces
4. Start the OTel processor

   Using GalileoSpanProcessor

   The GalileoSpanProcessor from the galileo package handles OTLP configuration automatically using your environment variables. No manual endpoint or header setup is required.

   TypeScript

   import { NodeSDK } from '@opentelemetry/sdk-node';
   import { AlwaysOnSampler } from '@opentelemetry/sdk-trace-node';
   import { GalileoSpanProcessor } from 'galileo';
   
   const sdk = new NodeSDK({
     spanProcessors: [new GalileoSpanProcessor()],
     sampler: new AlwaysOnSampler(),
   });
   
   sdk.start();
   

   Manual OTLP configuration

   Update the value of galileoEndpoint to reflect your endpoint if you are using a custom Galileo deployment.

   TypeScript

   import { NodeSDK } from '@opentelemetry/sdk-node';
   import { 
     AlwaysOnSampler,
     SimpleSpanProcessor
   } from '@opentelemetry/sdk-trace-node';
   import { OTLPHttpProtoTraceExporter } from '@vercel/otel';
   import { env } from 'process';
   
   const galileoEndpoint = "https://api.galileo.ai/otel/traces";
   const galileoHeaders = {
     "Galileo-API-Key": env.GALILEO_API_KEY,
     "project": env.GALILEO_PROJECT,
     "logstream": env.GALILEO_LOG_STREAM,
   };
   
   const sdk = new NodeSDK({
     spanProcessors: [
       new SimpleSpanProcessor(new OTLPHttpProtoTraceExporter({
           url: galileoEndpoint,
           headers: galileoHeaders,
       }))
     ],
     sampler: new AlwaysOnSampler(),
   });
   
   sdk.start();
   
5. Run your application

   Your application is now configured to send telemetry to Galileo using OTel. Run your application to see traces in your Log stream.

Full example

Here is a full example based off the Vercel AI SDK Agents example. You can find this project in the Galileo SDK examples repo.

To run this example, create a .env file with the following values set, or set them as environment variables:

.env

OPENAI_API_KEY=your-openai-api-key
GALILEO_API_KEY=your-galileo-api-key
GALILEO_PROJECT=your-galileo-project
GALILEO_LOG_STREAM=your-log-stream

Remember to update these to match your Galileo API key, OpenAI API key, project name, and Log stream name.

Using GalileoSpanProcessor

TypeScript

import dotenv from "dotenv";
import { NodeSDK } from "@opentelemetry/sdk-node";
import { AlwaysOnSampler } from "@opentelemetry/sdk-trace-node";
import { GalileoSpanProcessor } from "galileo";
import { openai } from "@ai-sdk/openai";
import { Experimental_Agent as Agent, stepCountIs, tool } from "ai";
import { z } from "zod";

dotenv.config();

// Set up Galileo tracing (reads env vars automatically)
const sdk = new NodeSDK({
  spanProcessors: [new GalileoSpanProcessor()],
  sampler: new AlwaysOnSampler(),
});
sdk.start();

const weatherAgent = new Agent({
  model: openai("gpt-4-turbo"),
  tools: {
    weather: tool({
      description: "Get the weather in a location (in Fahrenheit)",
      inputSchema: z.object({
        location: z.string().describe("The location to get the weather for"),
      }),
      execute: async ({ location }) => ({
        location,
        temperature: 72,
      }),
    }),
    convertFahrenheitToCelsius: tool({
      description: "Convert temperature from Fahrenheit to Celsius",
      inputSchema: z.object({
        temperature: z.number().describe("Temperature in Fahrenheit"),
      }),
      execute: async ({ temperature }) => {
        const celsius = Math.round((temperature - 32) * (5 / 9));
        return { celsius };
      },
    }),
  },
  stopWhen: stepCountIs(20),
  experimental_telemetry: { isEnabled: true },
});

const result = await weatherAgent.generate({
  prompt: "What is the weather in San Francisco in celsius?",
});

console.log(result.text);

await sdk.shutdown();

Manual OTLP configuration

TypeScript

import { openai } from '@ai-sdk/openai';
import { Experimental_Agent as Agent, stepCountIs, tool } from 'ai';
import { z } from 'zod';
import { NodeSDK } from '@opentelemetry/sdk-node';
import { 
  AlwaysOnSampler,
  SimpleSpanProcessor
} from '@opentelemetry/sdk-trace-node';
import { OTLPHttpProtoTraceExporter } from '@vercel/otel';
import { env } from 'process';
import dotenv from "dotenv";

// Load the environment variables
dotenv.config();

// Get the OTel endpoint
const galileoEndpoint = env.GALILEO_API_ENDPOINT || 
  "https://api.galileo.ai/otel/traces";

// Get the Galileo API key, project, and log stream from environment
// variables
const galileoHeaders = {
  "Galileo-API-Key": env.GALILEO_API_KEY ,
  "project": env.GALILEO_PROJECT,
  "logstream": env.GALILEO_LOG_STREAM,
};

// Create and start the OpenTelemetry SDK
const sdk = new NodeSDK({
  spanProcessors: [new SimpleSpanProcessor(new OTLPHttpProtoTraceExporter({
    url: galileoEndpoint,
    headers: galileoHeaders,
  }))],
  sampler: new AlwaysOnSampler(),
});

sdk.start();

/**
 * The weather agent is a simple agent that can get the weather in a
 * location and convert the temperature from Fahrenheit to Celsius
 * using tools.
 */
const weatherAgent = new Agent({
  model: openai('gpt-4-turbo'),
  tools: {
    weather: tool({
      description: 'Get the weather in a location (in Fahrenheit)',
      inputSchema: z.object({
        location: z.string().describe('The location to get the weather for'),
      }),
      execute: async ({ location }) => ({
        location,
        temperature: 72,
      }),
    }),
    convertFahrenheitToCelsius: tool({
      description: 'Convert temperature from Fahrenheit to Celsius',
      inputSchema: z.object({
        temperature: z.number().describe('Temperature in Fahrenheit'),
      }),
      execute: async ({ temperature }) => {
        const celsius = Math.round((temperature - 32) * (5 / 9));
        return { celsius };
      },
    }),
  },
  stopWhen: stepCountIs(20),
  // enable telemetry for the agent
  experimental_telemetry: { isEnabled: true },

});

// Run the weather agent to get the weather in San Francisco in Celsius
const result = await weatherAgent.generate({
  prompt: 'What is the weather in San Francisco in celsius?',
});

// Log the result
console.log(result.text);