When to use
Classifier
The Polaris classifier labels every span automatically, using a taxonomy trained over 30+ domains. Each span gets one of five annotation types: Action, Exception, Misstep, Warning or Behavior. Concepts defines each type with examples.How it works
Labels run on every span, not a sample. Flows uses the label to name each step in an execution path, so a path missing from the labels would be missing from the Sankey. “Label” always means classifier output. Eval output is a “score”. On an Agent workspace, Kubit labels new conversations once an hour, not as spans arrive. Until then, the Labels column in Activity readsNot analysed yet. On a Sandbox or warehouse workspace, a labelling run starts from the Evaluator page.
Example Prompts
“Which tool calls had a Misstep label yesterday?”
“Show the share of spans with a Warning label by model version over the last 30 days”
Evaluator Questions
The Evaluator page is where Polaris answers questions about each conversation. It asks four system questions, and you can add your own.How it works
To ask your own question, click Add custom question. A custom question can be yes or no, a choice from a list you define, or a score. When it’s asked depends on the workspace:
Evals run on Polaris and are included in Agent Observability at no extra cost.
Example Prompts
“How many conversations last week failed Task resolved?”
“Compare Right tools failures between the two newest model versions”
Fine-tuning
Polaris fine-tuning retrains custom adapters on the corrections your team makes in the workspace, for the classifier or for an eval question. Span text is used during tuning and isn’t kept after. The corrections and the adapter weights stay in Kubit.Where Results Are Stored
On a warehouse workspace, Polaris writes its labels and scores to a schema you pick in that warehouse, next to the traces, in tables that start withKUBIT_. You pick the location under Evaluator results on the Evaluator page. Where Polaris results are stored lists the tables and the warehouses that support it.
Related
- Concepts: annotation types and how labels differ from scores.
- Flows: how labels name the steps of each execution path.
- Find Missteps that return OK: use labels to find silent failures.
- Add your own eval questions: the steps to score every conversation against your own questions.