Sign up, configure your first integration, run a training, and register the resulting metric — end-to-end in about 15 minutes.
This walkthrough takes you from a fresh sign-up to a registered custom metric. By the end, you’ll have one project, one training run, and one metric available in the Galileo metrics store.
Enterprise tier and your own deployment. Luna Studio is part of the enterprise tier of Galileo and is deployed by Galileo into your own cluster or cloud. See Availability and deployment, or contact
us to get started.
You’ll need:
An instance of Luna Studio deployed for your org (Luna Studio is part of the enterprise tier — see Availability and deployment).
A small labelled dataset (CSV or JSONL with the columns your metric requires). Use at least 300 rows for a first run when possible.
Optional but recommended: a Galileo API key, so you can import datasets and register metrics into the Galileo platform.
Don’t have a labelled training dataset yet? Luna Studio can generate a training set from your test set — see the training-set step below. The test set still needs enough labelled rows for reliable
evaluation.
Open your org’s Luna Studio URL and create an account.
Create a Luna Studio account
Enter an email and a password (8+ characters), accept the terms, and click Create account.
2
Connect your first LLM provider
On first launch, Luna Studio drops you into the onboarding wizard. Pick a supported LLM provider, such as OpenAI or Anthropic, and click Add integration on the card.
Onboarding, step 1 — connect at least one LLM provider
A modal opens with the fields the provider needs (typically an API key). Paste it and click Save changes.Optionally add the Galileo integration in the section below — this unlocks the “Import from Galileo” dataset source later.
3
Create your first project
Click Continue and enter a project name (e.g. support-tone-classifier). Click Continue again — Luna Studio creates the project and routes you to its Training runs page.
Onboarding, step 2 — name your first project
4
Start a new training run
On the project page, click New run in the top right. The run creation flow opens at Step 1 — Metric.
5
Pick a metric (Step 1)
Pick a metric from the dropdown. The list includes Galileo presets, custom Galileo metrics, and saved custom prompts that are trainable in Luna Studio.
New run, Step 1 — pick the metric you want to fine-tune
Or click the dropdown’s Use custom prompt option to write your own LLM-as-judge prompt. See Step 1: Metric for the full reference.Click Next step.
6
Add a test set (Step 2)
Pick an existing test set from the dropdown, or click Add new test set to upload one.
New run, Step 2 — pick or upload a test set to evaluate against
Test sets need the columns required by your metric, plus a label column. Luna Studio validates the file as soon as you upload it — wait for the Validated status before continuing.Click Next step.
7
Add a training set (Step 3)
Choose a training source:
Generate from test set (recommended for a first run) — Luna Studio uses 20% of your test set as seed examples and generates 2,000 labelled training examples using your LLM-as-judge prompt from Step 1.
Add training logs — upload or import your own production logs.
Use existing training set — reuse a training dataset that already exists in your workspace.
New run, Step 3 — generate, add, or reuse a training set
If you pick Generate, a drawer opens. Pick a model from your configured providers and click Generate sample dataset. Review the sample rows, select any that should steer regeneration, and click Generate final dataset.Click Next step.
8
Confirm and launch (Step 4)
Review the run summary. The base model is selected from the models configured for your organization.
New run, Step 4 — pick a base model and launch fine-tuning
Click Confirm and launch. The run enters the Queued state and you’re routed back to the project page.
9
Wait for training to complete
Training runs progress through Queued → Training → Fine-tuned. Click the run row to open its details page and see live status.Once the run reaches Fine-tuned, the run details page shows a metrics grid (F1 score, AUC-ROC, etc.) versus a baseline.
Run details for a fine-tuned run, ready to register
10
Register your metric
Happy with the result? Click Register metric in the run details footer. Enter a metric name and click Register.Your metric now appears in the Galileo metrics store and can be used across Galileo for evaluation, observability, and guardrails. The run’s status flips to Registered.
Congratulations! You’ve now created your first custom Luna metric.
