# Databricks Reference

> All Databricks authentication options and data type mappings.

#  Databricks Reference

Authentication options and data type handling for [Databricks connections](../connect/databricks.md).

## Authentication

| Variable | Example | Description |
|---|---|---|
| `DATACONTRACT_DATABRICKS_SERVER_HOSTNAME` | `dbc-abcdefgh-1234.cloud.databricks.com` | Host of the SQL warehouse or compute cluster |
| `DATACONTRACT_DATABRICKS_HTTP_PATH` | `/sql/1.0/warehouses/b053a3ff...` | HTTP path to the SQL warehouse or compute cluster |
| `DATACONTRACT_DATABRICKS_TOKEN` | `dapia0000...` | A personal access token (PAT) |
| `DATACONTRACT_DATABRICKS_CLIENT_ID` | `00000000-...` | Service principal client ID for OAuth M2M auth |
| `DATACONTRACT_DATABRICKS_CLIENT_SECRET` | `dose0000...` | Service principal OAuth secret (used with the client ID) |
| `DATACONTRACT_DATABRICKS_PROFILE` | `my-profile` | A profile from `~/.databrickscfg` (Databricks SDK unified auth) |
| `DATACONTRACT_DATABRICKS_AUTH_TYPE` | `databricks-oauth` | Explicit connector auth type, e.g. for interactive U2M browser login |

The authentication method is selected from the variables you set, in this order: PAT → OAuth service principal (`CLIENT_ID` + `CLIENT_SECRET`) → config profile → explicit `AUTH_TYPE`. `catalog` and `schema` come from the contract's `servers` block.

`datacontract import unity` needs only the hostname plus a PAT or profile; `datacontract test` additionally needs `DATACONTRACT_DATABRICKS_HTTP_PATH`. When a `spark` session is passed programmatically (notebook/pipeline), no credentials are needed at all.

## Data types

### Importing

`datacontract import unity` keeps the full Unity Catalog type text as `physicalType` (e.g. `decimal(10,2)`, `struct<x:int>`, `map<string,int>`) — also duplicated in the `databricksType` custom property — and maps to `logicalType`:

| Databricks type | `logicalType` |
|---|---|
| `STRING`, `VARCHAR`, `CHAR` | `string` |
| `INT`, `BIGINT`, `SMALLINT`, `TINYINT` | `integer` |
| `FLOAT`, `DOUBLE`, `DECIMAL` | `number` |
| `BOOLEAN` | `boolean` |
| `DATE` | `date` |
| `TIMESTAMP` | `timestamp` |
| `BINARY` | `string` (format `binary`) |
| `ARRAY<...>` | `array` (nested `items` when pyspark is available) |
| `STRUCT<...>` | `object` (nested `properties` when pyspark is available) |
| `MAP<...>`, `INTERVAL`, `VARIANT` | *(unset)* — `physicalType` is still written |

`datacontract import spark` maps Spark session types the same way, with two caveats: `TimestampType` and `TimestampNTZType` are currently imported as `date`, and `BinaryType` as `array`.

### Testing

Databricks supports **native type introspection**: the declared `physicalType` is checked against the actual `information_schema` column type. Timezone variants of timestamps are interchangeable; parameters are only enforced when declared. A `physicalType` that isn't valid Databricks SQL falls back to the logical type category comparison. When testing a Spark DataFrame (`type: dataframe`), only the logical type category is checked.

{/* AUTOGENERATED TYPE MAPPING: do not edit by hand; regenerate with update_reference_types.py */}

### Logical type mapping

When no `physicalType` is declared, the CLI derives the native type from the `logicalType` — for example in `datacontract export sql` and the dbt exports. This table is generated from the converter in the CLI's code:

| `logicalType` | Databricks type |
|---|---|
| `string` | `STRING` |
| `integer` | `INT` |
| `number` | `DECIMAL(38,0)` |
| `boolean` | `BOOLEAN` |
| `date` | `DATE` |
| `timestamp` | `TIMESTAMP` |
| `time` | `STRING` |
| `object` | `STRUCT<>` |
| `array` | `ARRAY<STRING>` |

{/* END AUTOGENERATED TYPE MAPPING */}