> ## Documentation Index
> Fetch the complete documentation index at: https://learn.kubit.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Connect Databricks

> Create a Serverless SQL warehouse and a read-only service principal so Kubit queries your Databricks data in place.

Kubit connects directly to Databricks and queries your data in place. You create a SQL warehouse and a read-only service principal for Kubit. Nothing is copied out of your workspace.

## Before you start

* [ ] Workspace admin rights in Databricks, to create a SQL warehouse and a service principal
* [ ] The catalog and schema that hold the views Kubit should query
* [ ] If your workspace has IP access lists, Kubit's IP ranges. See [Allow Kubit's IP ranges](https://learn.kubit.ai/warehouses/ip-allowlist.md).

## Steps

### 1. Create a SQL warehouse

Create a Serverless SQL warehouse for Kubit. Start with the smallest size and the settings in [Warehouse configuration](#warehouse-configuration).

### 2. Create a service principal

Create a service principal for Kubit and give it three permissions:

| Permission | Why |
| - | - |
| `SELECT` on the schema with the views Kubit queries | Kubit reads your data |
| **Can use** on the Kubit SQL warehouse | Kubit runs its analyses on that warehouse |
| Access to the service principal's Query History | Kubit and you can troubleshoot slow queries |

With Unity Catalog, the read grants look like this. Replace the names in angle brackets.

```sql theme={null}
GRANT USE CATALOG ON CATALOG <catalog> TO `<service-principal-application-id>`;
GRANT USE SCHEMA ON SCHEMA <catalog>.<schema> TO `<service-principal-application-id>`;
GRANT SELECT ON SCHEMA <catalog>.<schema> TO `<service-principal-application-id>`;
```

### 3. Add the connection in Kubit

![Databricks Connection setup form](https://mintcdn.com/kubit/9i8QHxqek50tMBGy/images/warehouse/databricks.png?fit=max\&auto=format\&n=9i8QHxqek50tMBGy\&q=85\&s=e4c6c9819330d64ae6415b3400bfcb47)

### 4. Split test and production (optional)

Create a second SQL warehouse and service principal for development. Keeping development queries off the production warehouse avoids any impact on live workloads.

## Verify

Check the grants in a Databricks SQL editor. The output lists `USE SCHEMA` and `SELECT` for the Kubit service principal.

```sql theme={null}
SHOW GRANTS ON SCHEMA <catalog>.<schema>;
```

## Best practices

### Partitioning

Partition fact tables by an event date column. Partitions make Kubit's queries faster and cheaper, since most reports filter on a date range. If your tables are already partitioned by year, month and day, Kubit uses those columns and you don't need a separate date column.

### Warehouse configuration

Four warehouse settings affect Kubit's cost and speed. Databricks' SQL warehouse sizing, scaling and queuing reference has the details.

| Setting | Recommendation |
| - | - |
| **Cluster size** | Start small. Size up only when Query History shows it's needed. If reports are slow, open them in Query Profile and look for bytes spilled to disk above 1. Frequent spilling means the warehouse is too small. |
| **Auto stop** | The minimum: `5 minutes` in the UI, or `1 minute` through the SQL warehouses API. A longer idle time saves a few seconds of wake-up on some reports and costs more. |
| **Scaling** | Minimum and maximum of `1`. Raise the maximum only when you hit 10 concurrent queries several times a day. |
| **Type** | Serverless. Pro costs less than Serverless for a warehouse that runs 24 hours a day. |

## Usage monitoring

Databricks' monitoring tools show what Kubit's queries cost:

* Monitor a SQL warehouse
* Monitor usage in the account console
* Cost attribution queries on system tables
* The query history system table

A stopped Serverless warehouse can start without a query to run. Opening a JDBC or ODBC connection, or a dashboard tied to the warehouse, starts it. Databricks' "Start a SQL warehouse" page lists every trigger.

## Next steps

* [Bring your own warehouse](https://learn.kubit.ai/warehouses/overview.md): how Kubit queries your warehouse in place.
* [Allow Kubit's IP ranges](https://learn.kubit.ai/warehouses/ip-allowlist.md): let Kubit through an IP access list.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.