Annotations allow users to provide human feedback on LLM inputs and outputs through the Galileo Console UI and the API.Galileo supports annotations on Sessions, Traces, and Spans.
The Messages page allows you to submit annotations, and highlights available annotations with a dot indicator.
The Logs page allows you to view and export annotations as columns.
For each project, users with the necessary permissions can configure one or more of these annotation types:
Categories: Enable annotators to select one or more designated categories
Score: Enable annotators to select a number between 0 and a max score
Star: Enable annotators to rate from 1 to 5 stars
Text: Enable annotators to provide freeform text
Thumbs Up & Down: Enable annotators to rate a like or dislike
The following roles have permissions to configure annotations: organization admins, project owners, or project editors.The minimum permissions for creating annotations are: organization users, and project annotators. Read-only or viewer roles cannot submit annotations.Galileo’s Annotations feature is being actively improved. Please bookmark this page or contact support@galileo.ai to send requests and suggestions.
