Prompts

Learn how to create and use prompt templates in experiments

Prompts in Galileo allow you to create, store, and reuse LLM prompts across your experiments. They provide a structured way to manage your LLM interactions.

Prompts in the Galileo console

The Prompts page of the Galileo console shows you the existing prompts associated with your selected project. Click on All prompts to view all prompts in your organization. Click the Create Prompt button to create a new prompt.

Prompts in code

Create prompts

Python

from galileo import Message, MessageRole
from galileo.prompts import create_prompt

# Create a prompt with system and user messages
prompt = create_prompt(
    name="storyteller-prompt",
    template=[
        Message(role=MessageRole.system,
                content="You are a great storyteller."),
        Message(role=MessageRole.user,
                content="""
                Please write a short story about
                the following topic: {{topic}}
                """)
    ]
)

from galileo import Message, MessageRole, Prompt

# Create a prompt with system and user messages
prompt = Prompt(
    name="storyteller-prompt",
    messages=[
        Message(role=MessageRole.system,
                content="You are a great storyteller."),
        Message(role=MessageRole.user,
                content="""
                Please write a short story about
                the following topic: {{topic}}
                """)
    ],
)
prompt.create()

import { createPrompt} from "galileo";

const template = await createPrompt({
  template: [
    {
      role: "system",
      content: "You are a great storyteller."
    },
    {
      role: "user",
      content: `
        Please write a short story about
        the following topic: {topic}
        `
    },
  ],
  name: "storyteller-prompt",
});

Connect prompt templates to dataset inputs

When you use datasets in Galileo, the attributes stored in the input in your dataset are made available to your prompt templates using mustache templating. This allows you to create dynamic prompts that adapt to the data in each row.

Suppose you have the following dataset:

Python

test_data = [
    { "input": { "city": "Rome, Italy", "days": "5" } },
    { "input": { "city": "Paris, France", "days": "3" } },
]

const testData = [
    { input: { "city": "Rome, Italy", "days": "5" } },
    { input: { "city": "Paris, France", "days": "3" } },
];

To reference fields from your dataset in your prompt, use double curly braces:

Python

from galileo import Message, MessageRole

message = Message(role=MessageRole.user,
        content="""
        Plan a {{ days }}-day travel itinerary
        for a trip to {{ city }}.
        """)

{
  role: "user",
  content: `
    Plan a {{ days }}-day travel itinerary
    for a trip to {{ city }}.
    `
}

• {{ city }} will be replaced with the value of the city field inside the input dictionary.
• {{ days }} will be replaced with the value of the days field inside the input dictionary.

Get prompts

Once prompts have been created in Galileo, they can be retrieved by name. For project level prompts, pass in either the project name or Id.

Python

from galileo.prompts import get_prompt

# Get an existing prompt
prompt = get_prompt(
    name="storyteller-prompt"
)

from galileo import Prompt

# Get an existing prompt
prompt = Prompt.get(name="storyteller-prompt")

import { getPrompt } from "galileo";

const prompt = await getPrompt({
  name: "storyteller-prompt",
});

List prompts

To list all prompt templates in a project or organization:

Python

from galileo.prompts import get_prompts

# List all prompt templates in a project
templates = get_prompts()

# Print template names
for template in templates:
    print(f"Template: {template.name}")

from galileo import Prompt

# List all prompt templates
templates = Prompt.list()

# Print template names
for template in templates:
    print(f"Template: {template.name}")

import { getPrompts } from "galileo";

// Search list of prompts by name
const prompts = await getPrompts({
  name: "story",
});

// Log the prompt names
prompts.forEach((prompt) => console.log(prompt.name));

Delete prompts

To delete a prompt:

Python

from galileo.prompts import delete_prompt

# Delete a prompt
delete_prompt(
    name="storyteller-prompt"
)

from galileo import Prompt

# Get and delete a prompt
prompt = Prompt.get(name="storyteller-prompt")
if prompt is None:
    raise ValueError("Prompt 'storyteller-prompt' not found")
prompt.delete()

import { deletePrompt } from "galileo";

await deletePrompt({
  name: "storyteller-prompt",
});

Use prompts in experiments

Prompts can be used in experiments to evaluate different prompt templates:

Python

from galileo.datasets import get_dataset
from galileo.experiments import run_experiment
from galileo.prompts import get_prompt
from galileo import GalileoMetrics

# Get an existing dataset
dataset = get_dataset(
    name="countries"
)

# Get an existing prompt
prompt = get_prompt(
    name="geography-prompt"
)

# Run an experiment with the dataset and prompt
results = run_experiment(
    "geography-experiment",
    dataset=dataset,
    prompt_template=prompt,
    metrics=[GalileoMetrics.completeness],
    project="my-project",
)

from galileo import Dataset, Experiment, Metric, Prompt

# Get an existing dataset
dataset = Dataset.get(name="countries")
if dataset is None:
    raise ValueError("Dataset 'countries' not found")

# Get an existing prompt
prompt = Prompt.get(name="geography-prompt")
if prompt is None:
    raise ValueError("Prompt 'geography-prompt' not found")

# Create and run the experiment in one step
# (create() triggers the run automatically)
experiment = Experiment(
    name="geography-experiment",
    dataset=dataset,
    prompt=prompt,
    metrics=[Metric.metrics.completeness],
    project_name="my-project",
)
experiment.create()

import {
  GalileoMetrics, getDataset,
  getPrompt, runExperiment
} from "galileo";

// Get an existing dataset
const dataset = await getDataset({
    name: "countries"
});

// Get an existing prompt
const prompt = await getPrompt({
    name: "geography-prompt",
  });

// Run an experiment with the dataset and prompt
await runExperiment({
    name: "geography-experiment",
    dataset: dataset,
    promptTemplate: prompt,
    metrics: [GalileoMetrics.correctness],
    projectName: "my-project",
  });

Best practices

When working with prompts:

1. Use descriptive names that reflect the prompt’s purpose
2. Include clear system messages to set context
3. Document any required input variables
4. Version your prompt templates appropriately
5. Test prompt templates with various inputs before using in production

Related resources

Datasets

Learn about more datasets, the data driving your experiments.

Experiments

Learn how to use datasets and experiments to improve your application.