Skip to main content

User Intent Change checks if users are using your agent system for different intents across multi-turn conversations.
User Intent Change is a binary evaluation metric, and is defined as a significant shift in the user’s primary conversational goal or workflow during a session, relative to their initial stated intent. This is a boolean metric, returning a confidence score that the user intent has changed significantly. The score ranges from 0% (no confidence the user intent has changed) to 100% (complete confidence that the user intent has changed).

​
User Intent Change at a glance

PropertyDescription
NameUser Intent Change
CategoryAgentic AI
Can be applied toSession
LLM-as-a-judge Support✅
Luna Support❌
Protect Runtime Protection❌
Value TypeBoolean shown as a percentage confidence score

​
When to use this metric

When to Use This Metric

User Intent Change is useful when working with multi-step chat bots or other AI systems with multiple user interactions
Check if user is using your agent across different topics in one conversation

​
Score interpretation

Expected Score: 80%-100%.
060%100%
Poor
Fair
Excellent