Connect Databricks

Kubit connects directly to Databricks and queries your data in place. You create a SQL warehouse and a read-only service principal for Kubit. Nothing is copied out of your workspace.

Before you start

Workspace admin rights in Databricks, to create a SQL warehouse and a service principal
The catalog and schema that hold the views Kubit should query
If your workspace has IP access lists, Kubit’s IP ranges. See Allow Kubit’s IP ranges.

Steps

Create a SQL warehouse

Create a Serverless SQL warehouse for Kubit. Start with the smallest size and the settings in Warehouse configuration.

Create a service principal

Create a service principal for Kubit and give it three permissions:

Permission	Why
`SELECT` on the schema with the views Kubit queries	Kubit reads your data
Can use on the Kubit SQL warehouse	Kubit runs its analyses on that warehouse
Access to the service principal’s Query History	Kubit and you can troubleshoot slow queries

With Unity Catalog, the read grants look like this. Replace the names in angle brackets.

GRANT USE CATALOG ON CATALOG <catalog> TO `<service-principal-application-id>`;
GRANT USE SCHEMA ON SCHEMA <catalog>.<schema> TO `<service-principal-application-id>`;
GRANT SELECT ON SCHEMA <catalog>.<schema> TO `<service-principal-application-id>`;

Add the connection in Kubit

Split test and production (optional)

Create a second SQL warehouse and service principal for development. Keeping development queries off the production warehouse avoids any impact on live workloads.

Verify

Check the grants in a Databricks SQL editor. The output lists USE SCHEMA and SELECT for the Kubit service principal.

SHOW GRANTS ON SCHEMA <catalog>.<schema>;

Best practices

Partitioning

Partition fact tables by an event date column. Partitions make Kubit’s queries faster and cheaper, since most reports filter on a date range. If your tables are already partitioned by year, month and day, Kubit uses those columns and you don’t need a separate date column.

Warehouse configuration

Four warehouse settings affect Kubit’s cost and speed. Databricks’ SQL warehouse sizing, scaling and queuing reference has the details.

Setting	Recommendation
Cluster size	Start small. Size up only when Query History shows it’s needed. If reports are slow, open them in Query Profile and look for bytes spilled to disk above 1. Frequent spilling means the warehouse is too small.
Auto stop	The minimum: `5 minutes` in the UI, or `1 minute` through the SQL warehouses API. A longer idle time saves a few seconds of wake-up on some reports and costs more.
Scaling	Minimum and maximum of `1`. Raise the maximum only when you hit 10 concurrent queries several times a day.
Type	Serverless. Pro costs less than Serverless for a warehouse that runs 24 hours a day.

Usage monitoring

Databricks’ monitoring tools show what Kubit’s queries cost:

Monitor a SQL warehouse
Monitor usage in the account console
Cost attribution queries on system tables
The query history system table

A stopped Serverless warehouse can start without a query to run. Opening a JDBC or ODBC connection, or a dashboard tied to the warehouse, starts it. Databricks’ “Start a SQL warehouse” page lists every trigger.

Next steps

Bring your own warehouse: how Kubit queries your warehouse in place.
Allow Kubit’s IP ranges: let Kubit through an IP access list.

Start here

Agent Observability

Product Analytics

Bring your own warehouse

Connect Databricks

Before you start

Steps

Create a SQL warehouse

Create a service principal

Add the connection in Kubit

Split test and production (optional)

Verify

Best practices

Partitioning

Warehouse configuration

Usage monitoring

Next steps

​Before you start

​Steps

Create a SQL warehouse

Create a service principal

Add the connection in Kubit

Split test and production (optional)

​Verify

​Best practices

​Partitioning

​Warehouse configuration

​Usage monitoring

​Next steps

Before you start

Steps

Verify

Best practices

Partitioning

Warehouse configuration

Usage monitoring

Next steps