> ## Documentation Index
> Fetch the complete documentation index at: https://learn.kubit.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Polaris

> Reference for Polaris, Kubit's in-house small language model: the classifier, your own eval questions, fine-tuning and where labels are stored.

Polaris is Kubit's in-house small language model. It labels and scores every span of your agent traces, in under a second per span, with no additional LLM cost or sampling. This page covers the classifier, eval questions, fine-tuning and where the results go.

## When to use

| **Use case** | **Example** |
| - | - |
| Find calls that returned OK but did the wrong thing | Spans labeled `wrong_tool` or `malformed_args` |
| Check every conversation against your own definition of a good answer | A yes or no question such as "Did the agent confirm the refund amount?" |
| Correct labels that don't fit your domain | Fine-tune Polaris on the corrections your team makes |
| Group runs by what each step did | Flows names each step with the label Polaris gave its span |

## Classifier

The Polaris classifier labels every span automatically, using a taxonomy trained over 30+ domains. Each span gets one of five annotation types: **Action**, **Exception**, **Misstep**, **Warning** or **Behavior**. [Concepts](https://learn.kubit.ai/agent-observability/concepts.md#annotation-types) defines each type with examples.

### How it works

Labels run on every span, not a sample. Flows uses the label to name each step in an execution path, so a path missing from the labels would be missing from the Sankey. "Label" always means classifier output. Eval output is a "score".

On an Agent workspace, Kubit labels new conversations once an hour, not as spans arrive. Until then, the **Labels** column in **Activity** reads `Not analysed yet`. On a Sandbox or warehouse workspace, a labelling run starts from the **Evaluator** page.

### Example Prompts

> "Which tool calls had a Misstep label yesterday?"

> "Show the share of spans with a Warning label by model version over the last 30 days"

## Evaluator Questions

The **Evaluator** page is where Polaris answers questions about each conversation. It asks four system questions, and you can add your own.

| **Question** | **What it asks** |
| - | - |
| **Answer relevant** | Does the assistant's reply directly address what the user asked? |
| **Task resolved** | By the end of this conversation, had the assistant completed the task the user asked for? |
| **Right tools** | Were the tools the agent invoked the right ones for the user's request? |
| **Grounded in tool results** | Is every identifier, amount, status and completed action in the assistant's final reply supported by the tool results shown? |

### How it works

To ask your own question, click **Add custom question**. A custom question can be yes or no, a choice from a list you define, or a score. When it's asked depends on the workspace:

| **Workspace** | **When custom questions are asked** |
| - | - |
| **Sandbox** | On each upload's labelling run. Click **Relabel** on an upload to ask every question again and replace its labels. |
| Warehouse (**Warehouse**, **MLflow Tracing**, **Cortex Agents**, **Google ADK**, **Langfuse**) | When you label a window of days from the **Evaluator** page. Nothing is scheduled, except the first week of an **MLflow Tracing**, **Google ADK**, **Langfuse** or **Cortex Agents** workspace. Needs Snowflake, Databricks, BigQuery or ClickHouse. |
| **Agent** | Every hour, for each new conversation. A new question isn't asked of earlier conversations. |

Evals run on Polaris and are included in Agent Observability at no extra cost.

| **Constraint** | **Value** |
| - | - |
| **Uploads labeled at a time** | One |
| **Score types for custom questions** | Yes or no, choice, score |
| **Custom questions turned on per workspace** | 10. An 11th is saved paused. |

### Example Prompts

> "How many conversations last week failed Task resolved?"

> "Compare Right tools failures between the two newest model versions"

## Fine-tuning

Polaris fine-tuning retrains custom adapters on the corrections your team makes in the workspace, for the classifier or for an eval question. Span text is used during tuning and isn't kept after. The corrections and the adapter weights stay in Kubit.

## Where Results Are Stored

On a warehouse workspace, Polaris writes its labels and scores to a schema you pick in that warehouse, next to the traces, in tables that start with `KUBIT_`. You pick the location under **Evaluator results** on the **Evaluator** page. [Where Polaris results are stored](https://learn.kubit.ai/warehouses/overview.md#where-polaris-results-are-stored) lists the tables and the warehouses that support it.

## Related

* [Concepts](https://learn.kubit.ai/agent-observability/concepts.md#polaris-annotations): annotation types and how labels differ from scores.
* [Flows](https://learn.kubit.ai/agent-observability/use/flows.md): how labels name the steps of each execution path.
* [Find Missteps that return OK](https://learn.kubit.ai/agent-observability/use/find-missteps.md): use labels to find silent failures.
* [Add your own eval questions](https://learn.kubit.ai/agent-observability/use/eval-questions.md): the steps to score every conversation against your own questions.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.