> ## Documentation Index
> Fetch the complete documentation index at: https://learn.kubit.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Send traces from Python

> Send OpenTelemetry spans from a Python agent to Kubit with kubit-otel, as the only exporter or next to an existing TracerProvider.

`kubit-otel` sends OpenTelemetry spans from a Python app to Kubit. It has two entry points: `configure()` when Kubit is your only exporter, and `attach()` when your app already creates a `TracerProvider`. This page covers both, plus the settings to check before production.

## Before you start

* [ ] Python 3.9 or later 
* [ ] A Kubit API key in `KUBIT_API_KEY`

Kubit authenticates spans with an ingestion API key, read from the `KUBIT_API_KEY` environment variable. Each key belongs to one workspace.

| Way to get a key | How |
| - | - |
| **Coding agent** | Run `/kubit-integrate` in Claude Code. It creates or selects a workspace, mints a key and writes `KUBIT_API_KEY` to `.env.local` or `.env`. See [Set up with your coding agent](https://learn.kubit.ai/coding-agent-setup.md). |
| **Kubit app** | |

> [!WARNING]
>
> Keep the key out of source control. Put it in `.env` or your secrets manager, and check that `.env` is in `.gitignore`.

## Prompt for your coding agent

```text theme={null}
Add Kubit tracing to this Python repo. Follow https://learn.kubit.ai/send-traces/python.md exactly.
If the repo already creates a TracerProvider, use attach(). Otherwise use configure().
Read KUBIT_API_KEY from .env. Show me the diff before writing any file.
```

## Install

```bash pip theme={null}
pip install kubit-otel
```

```bash uv theme={null}
uv add kubit-otel
```

```bash poetry theme={null}
poetry add kubit-otel
```

## Choose configure() or attach()

| Your app today | Use | Why |
| - | - | - |
| No OpenTelemetry setup | `configure()` | Creates a `TracerProvider` with the Kubit exporter and registers it as the global provider |
| Creates its own `TracerProvider` and calls `trace.set_tracer_provider()` | `attach()` | Adds `KubitSpanProcessor` to your provider. No second provider, and your existing exporters keep working. |
| Sends traces to Langfuse or Braintrust | `attach()` | Same as above. [Set up with your coding agent](https://learn.kubit.ai/coding-agent-setup.md) handles the sink-specific wiring. |

> [!WARNING]
>
> Don't call `configure()` in a process that already sets a global `TracerProvider`. Use `attach()`.

## You have no OpenTelemetry setup

Create `kubit_instrumentation.py` and import it first in your entrypoint.

```python kubit_instrumentation.py theme={null}
import os
from kubit_otel import configure

configure(
    api_key=os.environ["KUBIT_API_KEY"],
    service_name="support-agent",
    service_version="1.0.0",
    resource_attributes={"deployment.environment": "production"},
)
```

```python main.py theme={null}
import kubit_instrumentation  # first import, before any LLM client is created
```

### configure() parameters

| Parameter | Required? | Notes |
| - | - | - |
| **api\_key** | **Yes** | The workspace ingestion key. Read it from `KUBIT_API_KEY`. |
| **service\_name** | Optional | Becomes the OpenTelemetry `service.name`. Set it. Atlas uses it to tell agents apart. |
| **service\_version** | Optional | Becomes `service.version`. Set it to your release or git SHA so you can compare versions. |
| **resource\_attributes** | Optional | A dict merged into the OpenTelemetry resource, e.g. `{"deployment.environment": "production"}` |

## You already have a TracerProvider

Call `attach()` after `trace.set_tracer_provider()`, in the module that creates the provider:

```python tracing.py theme={null}
import os
from opentelemetry import trace
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from kubit_otel import attach

provider = TracerProvider(resource=Resource.create({"service.name": "support-agent"}))
# your existing span processors and exporters stay here
trace.set_tracer_provider(provider)

attach(api_key=os.environ["KUBIT_API_KEY"])
```

`attach()` finds the global provider and adds `KubitSpanProcessor` to it. Your provider's resource (`service.name`, `service.version`) applies to the spans Kubit receives.

### attach() parameters

| Parameter | Required? | Notes |
| - | - | - |
| **api\_key** | **Yes** | The workspace ingestion key |

**Availability:** `attach()` must run after `trace.set_tracer_provider()`. Called earlier, it finds no provider to attach to. 

## Trace LLM and tool calls

Kubit receives whatever spans your app produces. To get one span per LLM call, add the OpenTelemetry GenAI instrumentor for your client after `configure()` or `attach()`:

```python kubit_instrumentation.py theme={null}
import os
from kubit_otel import configure
from opentelemetry.instrumentation.openai_v2 import OpenAIInstrumentor

configure(api_key=os.environ["KUBIT_API_KEY"], service_name="support-agent")
OpenAIInstrumentor().instrument()
```

```bash theme={null}
pip install opentelemetry-instrumentation-openai-v2
export OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true
```

`OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true` puts prompt and response text on the spans. Polaris labels spans from that text.

For tool calls, wrap each tool in its own span so it shows up as a step in the execution path:

```python tools.py theme={null}
from opentelemetry import trace

tracer = trace.get_tracer("support-agent")

def search_orders(customer_id: str, query: str) -> list[dict]:
    with tracer.start_as_current_span("execute_tool search_orders") as span:
        span.set_attribute("gen_ai.operation.name", "execute_tool")
        span.set_attribute("gen_ai.tool.name", "search_orders")
        results = run_order_search(customer_id, query)  # your existing tool code
        span.set_attribute("search.result_count", len(results))
        return results
```

[Span attributes](https://learn.kubit.ai/send-traces/span-attributes.md) lists the keys Atlas reads, including user and session IDs.

## Before production

| Check | Why |
| - | - |
| `service_version` set to your release | Triggers compare metrics across versions |
| `deployment.environment` set | Keeps dev and production traces apart |
| Short-lived jobs call `trace.get_tracer_provider().force_flush()` before exit | Spans still in the batch are lost when the process exits |
| User and session IDs on the root span | Without them Atlas can't group traces into sessions or join them to users |
| Sensitive fields masked before export | Message content goes to Kubit for labeling |

## Verify

Send a test span:

```bash theme={null}
KUBIT_API_KEY=<your-key> python -c "
import kubit_instrumentation
from opentelemetry import trace
trace.get_tracer('kubit-sdk').start_span('hello-kubit').end()
import time; time.sleep(2)
"
```

Then ask your assistant with the Kubit MCP server connected:

> "Show me traces from the last 15 minutes for service support-agent"

The result includes a trace with a root span named `hello-kubit`.

## If it fails

| Symptom | Cause | Fix |
| - | - | - |
| `KeyError: 'KUBIT_API_KEY'` | Env var not set, or `.env` not loaded yet | Export it, or call `load_dotenv()` before `import kubit_instrumentation` |
| Spans reach your old exporter but not Kubit | `attach()` ran before `set_tracer_provider()` | Move `attach()` after `trace.set_tracer_provider(provider)` |
| Spans reach Kubit but not your old exporter | `configure()` replaced your global provider | Remove `configure()` and use `attach()` |
| LLM spans have no prompt or response text | Content capture is off | `export OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true` |
| No spans from a cron job or script | Process exits before the batch flushes | Call `trace.get_tracer_provider().force_flush()` at the end |

## Next steps

* [Span attributes](https://learn.kubit.ai/send-traces/span-attributes.md): the attribute keys Atlas reads.
* [Is my data flowing?](https://learn.kubit.ai/data-flowing-checklist.md): step-by-step checks.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.