Skip to main content
Polaris is Kubit’s in-house small language model. It labels and scores every span of your agent traces, in under a second per span, with no additional LLM cost or sampling. This page covers the classifier, eval questions, fine-tuning and where the results go.

When to use

Classifier

The Polaris classifier labels every span automatically, using a taxonomy trained over 30+ domains. Each span gets one of five annotation types: Action, Exception, Misstep, Warning or Behavior. Concepts defines each type with examples.

How it works

Labels run on every span, not a sample. Flows uses the label to name each step in an execution path, so a path missing from the labels would be missing from the Sankey. “Label” always means classifier output. Eval output is a “score”. On an Agent workspace, Kubit labels new conversations once an hour, not as spans arrive. Until then, the Labels column in Activity reads Not analysed yet. On a Sandbox or warehouse workspace, a labelling run starts from the Evaluator page.

Example Prompts

“Which tool calls had a Misstep label yesterday?”
“Show the share of spans with a Warning label by model version over the last 30 days”

Evaluator Questions

The Evaluator page is where Polaris answers questions about each conversation. It asks four system questions, and you can add your own.

How it works

To ask your own question, click Add custom question. A custom question can be yes or no, a choice from a list you define, or a score. When it’s asked depends on the workspace: Evals run on Polaris and are included in Agent Observability at no extra cost.

Example Prompts

“How many conversations last week failed Task resolved?”
“Compare Right tools failures between the two newest model versions”

Fine-tuning

Polaris fine-tuning retrains custom adapters on the corrections your team makes in the workspace, for the classifier or for an eval question. Span text is used during tuning and isn’t kept after. The corrections and the adapter weights stay in Kubit.

Where Results Are Stored

On a warehouse workspace, Polaris writes its labels and scores to a schema you pick in that warehouse, next to the traces, in tables that start with KUBIT_. You pick the location under Evaluator results on the Evaluator page. Where Polaris results are stored lists the tables and the warehouses that support it.