# Data Contract CLI

> An open-source command-line tool for working with data contracts based on the Open Data Contract Standard (ODCS).

This file contains all documentation content in a single document following the llmstxt.org standard.

## What is Data Contract CLI?


  
    
  
    
  
    
  
    

The `datacontract` CLI is an open-source command-line tool for working with [data contracts](https://datacontract.com).

It natively supports the [Open Data Contract Standard (ODCS)](https://bitol-io.github.io/open-data-contract-standard/latest/) to:

- **Lint** data contracts and validate them against the ODCS JSON Schema.
- **Connect** to data sources such as Snowflake, BigQuery, Databricks, Postgres, Kafka, S3, and many more.
- **Test** that the actual data complies with the schema and quality expectations defined in the contract.
- **Export** a contract to 25+ formats (SQL DDL, dbt, Avro, JSON Schema, HTML, Protobuf, …).
- **Import** an existing schema (SQL, dbt, BigQuery, Glue, Excel, …) into a data contract.

The tool is written in Python. It can be used as a standalone CLI tool, in a CI/CD pipeline, or directly as a Python library.

Three commands take you from an existing warehouse table to a tested data contract:

```bash
uv tool install --python python3.11 --upgrade 'datacontract-cli[snowflake]'
datacontract import snowflake --source <account> --database ORDER_DB --schema PUBLIC --output datacontract.yaml
datacontract test datacontract.yaml
```

```
🟢 data contract is valid. Run 24 checks. Took 5.2 seconds.
```

The same flow works for [BigQuery](./testing/bigquery.md), [Databricks](./testing/databricks.md), [Postgres](./testing/postgres.md), and [15+ other sources](./testing/index.md) — or [try it on a local CSV file](./testing/local.md) without any credentials.

![Overview of the Data Contract CLI: schemas from SQL DDL, JSON Schema, Iceberg, Protobuf, BigQuery, Unity Catalog, AWS Glue, and Excel are imported into an ODCS data contract, which is linted, tested against S3, BigQuery, Azure, Databricks, Snowflake, and Kafka, and exported to SQL DDL, HTML, dbt, Entropy Data, Avro, SodaCL, Pydantic, and Excel.](/img/datacontractcli.webp)

## Why data contracts?

A data contract is a machine-readable, versioned agreement that describes the **structure**, **semantics**, **quality**, and **service levels** of a data set. Because the contract is a single file, it can be:

- stored in Git next to your code,
- reviewed in pull requests,
- linted and tested automatically in CI/CD, and
- used to generate downstream artifacts (DDL, dbt models, schemas, documentation).

A typical contract has a `servers` section with endpoint details, a `schema` describing the structure and semantics of the data, plus `service levels` and `quality` attributes that describe expectations such as freshness and number of rows. This is enough information to connect to the data source and check that the actual data product is compliant.

## How it works

When you run `datacontract test`, the CLI connects to a data source and runs schema and quality tests to verify that the actual data complies with the contract. It picks a connection engine based on the server `type` — see **[Test your Data](./testing/index.md#how-it-works)**.

## Next steps

- New here? Start with the **[Quickstart](./quickstart.md)**.
- Test your own warehouse in 5 minutes: **[Snowflake](./testing/snowflake.md)**, **[BigQuery](./testing/bigquery.md)**, **[Databricks](./testing/databricks.md)**, **[Amazon Redshift](./testing/redshift.md)**, or **[any other source](./testing/index.md)**.
- Learn the underlying format in **[Open Data Contract Standard](./open-data-contract-standard.md)**.
- Author contracts visually with the **[Data Contract Editor](./editor.md)**.
- Run checks against real data with **[Test your Data](./testing/index.md)**, then keep them running with **[Scheduling and CI/CD](./ci-cd.md)**.
- Roll it out across a team with **[Adopting Data Contracts](./best-practices.md)**.
- See every command in the **[Commands reference](./commands/index.md)**.
- Evaluating options? See **[Comparison with other Tools](./comparison.md)**.
- Coming from the older `datacontract.yaml` format? **[Migrate from DCS to ODCS](./migrate-dcs-to-odcs.md)**.
- Upgrading? See what changed in **[Release Notes](./release-notes.md)**.

## Frequently asked questions

### What is the Data Contract CLI?

The Data Contract CLI (`datacontract`) is a free and open-source command-line tool, written in Python and published under the MIT license, for linting data contracts, testing real data against them, and importing or exporting them to other schema formats.

### Which data contract format does the Data Contract CLI use?

It natively uses the [Open Data Contract Standard (ODCS)](./open-data-contract-standard.md), an open specification governed by Bitol under the Linux Foundation AI & Data. The legacy Data Contract Specification (DCS) is still supported for reading and as an [export target](./exports/dcs.md).

### How do I install the Data Contract CLI?

Run `pip install 'datacontract-cli[postgres]'` (or the extra matching your data source), or use `uv tool install`. `datacontract-cli[all]` installs every optional dependency. A Docker image is published as `datacontract/cli`. See [Installation](./installation.md) for all options.

### Which data sources can the Data Contract CLI test?

Snowflake, Databricks, Google BigQuery, Amazon Athena, Amazon Redshift, Amazon S3, Azure Blob Storage and ADLS, Google Cloud Storage, Postgres, MySQL, Microsoft SQL Server, Oracle, Trino, Apache Impala, Kafka, Spark DataFrames, JSON HTTP APIs, and local Parquet, JSON, CSV, or Delta files. See [Test your Data](./testing/index.md).

### Is the Data Contract CLI free to use?

Yes. It is open source under the MIT license and free for commercial use. [Entropy Data](./entropy-data.md) offers a commercial platform that the CLI can publish test results to, but the CLI itself does not require it.

## Related links

- Website: [datacontract.com](https://datacontract.com)
- Source code: [github.com/datacontract/datacontract-cli](https://github.com/datacontract/datacontract-cli)
- Community Slack: [datacontract.com/slack](https://datacontract.com/slack)
- GitHub Action: [datacontract/datacontract-action](https://github.com/datacontract/datacontract-action/)

---

## Quickstart


This guide gets you from zero to a tested data contract in a few minutes — first against a hosted demo database (60 seconds, nothing to set up), then against your own data warehouse (about 5 minutes).

## Install

The preferred way to install is [uv](https://docs.astral.sh/uv/). Add the extra for the data source you want to test — the demo below uses Postgres:

```bash
uv tool install --python python3.11 --upgrade 'datacontract-cli[postgres]'
```

Every source has its own extra (`snowflake`, `bigquery`, `databricks`, `s3`, `duckdb` for local files, …), so you only install what you need. `datacontract-cli[all]` pulls in every optional dependency, including Spark — convenient, but a much larger download. See [Installation options](#installation-options) below for `pip`, `pipx`, and Docker.

Verify the installation:

```bash
datacontract --version
```

## Test your first data contract (60 seconds)

Let's use the example contract published at
[`https://datacontract.com/orders-v1.odcs.yaml`](https://datacontract.com/orders-v1.odcs.yaml).
It contains a `servers` section pointing at a Postgres database, a `schema`, and `quality` attributes.

Provide credentials as environment variables and run `test`:

```bash
export DATACONTRACT_POSTGRES_USERNAME=datacontract_cli.egzhawjonpfweuutedfy
export DATACONTRACT_POSTGRES_PASSWORD=jio10JuQfDfl9JCCPdaCCpuZ1YO

datacontract test https://datacontract.com/orders-v1.odcs.yaml
```

```
Testing https://datacontract.com/orders-v1.odcs.yaml
Server: production (type=postgres, host=..., database=postgres, schema=dp_orders_v1)
╭────────┬──────────────────────────────────────────────────────────┬─────────────────────────┬─────────╮
│ Result │ Check                                                      │ Field                   │ Details │
├────────┼──────────────────────────────────────────────────────────┼─────────────────────────┼─────────┤
│ passed │ Check that field 'order_id' is present                     │ orders.order_id         │         │
│ passed │ Check that field order_id has type UUID                    │ orders.order_id         │         │
│ passed │ Check that unique field order_id has no duplicate values   │ orders.order_id         │         │
│  ...   │                                                            │                         │         │
╰────────┴──────────────────────────────────────────────────────────┴─────────────────────────┴─────────╯
🟢 data contract is valid. Run 25 checks. Took 3.938887 seconds.
```

The CLI verified that the YAML itself is valid, that all records comply with the schema, and that all quality attributes are met.

## Test your own data (5 minutes)

The real magic moment is when a contract catches drift in *your* data. Import a contract straight from an existing table — the import generates the schema and a ready-to-test `servers` block — then test the actual data against it:

```bash
datacontract import postgres --source localhost --database mydb --output datacontract.yaml
datacontract test datacontract.yaml
```

That example uses Postgres because it matches the extra installed above. For another source, install its extra first — `uv tool install --python python3.11 --upgrade 'datacontract-cli[snowflake]'` — and use the matching import, e.g. `datacontract import snowflake`.

Follow the copy-paste guide for your data source, including credentials setup and troubleshooting:

- **[Snowflake](./testing/snowflake.md)** — import from your tables, test in 5 minutes
- **[Google BigQuery](./testing/bigquery.md)** — import from your tables, test in 5 minutes
- **[Databricks](./testing/databricks.md)** — import from Unity Catalog, test in 5 minutes
- **[Amazon Redshift](./testing/redshift.md)** — import from your tables, test in 5 minutes
- **[Postgres](./testing/postgres.md)**, **[Amazon S3](./testing/s3.md)**, and [15+ other sources](./testing/index.md)
- **[Local files](./testing/local.md)** — no warehouse or credentials needed, works offline

Each guide ends with the same payoff: tighten an expectation, rerun `datacontract test`, and watch the contract catch the violation with exit code `1` — the exact behavior you'll later use [in CI/CD](./ci-cd.md).

## Export to another format

You can use the contract metadata to generate downstream artifacts. For example, a SQL DDL:

```bash
datacontract export sql https://datacontract.com/orders-v1.odcs.yaml
```

```sql
-- Data Contract: orders
-- SQL Dialect: postgres
CREATE TABLE orders (
  order_id UUID not null primary key,
  customer_id text not null,
  order_total integer not null,
  order_timestamp TIMESTAMPTZ,
  order_status text
);
```

Or an HTML page:

```bash
datacontract export html --output orders-v1.odcs.html https://datacontract.com/orders-v1.odcs.yaml
```

See **[Exports](./exports/index.md)** for all 25+ target formats.

## The typical workflow

```bash
# Create a new data contract from a template and write it to odcs.yaml
datacontract init odcs.yaml

# Edit the data contract in the Data Contract Editor (web UI)
datacontract edit odcs.yaml

# Lint the odcs.yaml and stop after the first validation error (default)
datacontract lint odcs.yaml

# Show a changelog between two data contracts
datacontract changelog v1.odcs.yaml v2.odcs.yaml

# Execute schema and quality checks (credentials via environment variables)
datacontract test odcs.yaml

# Generate dbt tests from a contract into your dbt project and run `dbt test`
datacontract dbt sync orders.odcs.yaml --project-dir ./warehouse

# Export to HTML (and many other formats)
datacontract export html odcs.yaml --output odcs.html

# Import from an existing SQL DDL
datacontract import sql --source my-ddl.sql --dialect postgres --output odcs.yaml
```

## Use it as a Python library

```python
from datacontract.data_contract import DataContract

data_contract = DataContract(data_contract_file="odcs.yaml")
run = data_contract.test()
if not run.has_passed():
    print("Data quality validation failed.")
    # Abort pipeline, alert, or take corrective actions...
```

## Next steps

- Keep the tests running automatically: **[Scheduling and CI/CD](./ci-cd.md)** — GitHub Actions, Azure DevOps, cron, and orchestrators.
- Roll data contracts out across a team: **[Adopting Data Contracts](./best-practices.md)**.
- Browse everything the CLI can do: **[Commands](./commands/index.md)**.

## Installation options

See the dedicated **[Installation](./installation.md)** page for all options (uv, uvx, pip, venv, pipx, Docker) and the full list of optional dependencies (extras).

---

## Open Data Contract Standard


The Data Contract CLI natively uses the **[Open Data Contract Standard (ODCS)](https://bitol-io.github.io/open-data-contract-standard/latest/)** as its data contract format. ODCS is an open, vendor-neutral standard maintained by the [Bitol](https://bitol.io/) project under the Linux Foundation's AI & Data umbrella.

A data contract written in ODCS is a single YAML file that describes a data set's structure, semantics, quality, ownership, and the servers where the data lives.

## A minimal contract

```yaml
apiVersion: v3.1.0
kind: DataContract
id: urn:datacontract:checkout:orders-latest
name: orders
version: 1.0.0
status: active
description:
  purpose: One record per order. Includes cancelled and deleted orders.
team:
  name: Checkout
  members:
    - username: dataeng@example.com
      role: Owner
servers:
  - server: production
    type: postgres
    host: localhost
    port: 5432
    database: orders
    schema: public
schema:
  - name: orders
    physicalName: orders
    properties:
      - name: order_id
        logicalType: string
        physicalType: uuid
        primaryKey: true
        required: true
      - name: order_total
        logicalType: integer
        physicalType: integer
        required: true
        quality:
          - type: sql
            description: 95% of order totals are between 10 and 499 EUR
            query: SELECT quantile_cont(order_total, 0.95) FROM orders
            mustBeBetween: [1000, 99900]
```

`v3.1.0` is the current version of the standard and the one [`datacontract init`](./commands/init.md) writes. The CLI also validates contracts declaring `v3.0.2`, `v3.0.1`, `v3.0.0`, and the v2.2.x line.

:::note
The CLI also accepts the older Data Contract Specification format (which uses `models`/`fields` instead of ODCS `schema`/`properties`), but new contracts should follow ODCS — all examples in this documentation use ODCS. To convert an existing one, see [Migrate from DCS to ODCS](./migrate-dcs-to-odcs.md).
:::

## Key sections

| Section | Purpose |
|---|---|
| `apiVersion` / `kind` | Identifies the document as an ODCS data contract and its version. |
| `id`, `name`, `version`, `status` | Identity and lifecycle of the contract. |
| `description` | Human-readable purpose, usage, and limitations. |
| `servers` | Where the data physically lives — the connection details used by [`test`](./commands/test.md). One contract can have several servers. |
| `schema` | The logical structure: schemas (tables/objects) and their properties (columns/fields), types, constraints, and semantics. |
| `quality` | Data quality rules, attached to the schema or to individual properties. See [Quality Rules](./quality-rules/index.md). |
| `slaProperties` | Service-level expectations such as freshness, retention, and frequency. |
| `team` / `roles` | Ownership and access information. `team` is an object with a `members` array. |
| `customProperties` | Extension point for backend-specific settings (for example `clickhouseType`, `trinoType`, `avroLogicalType`). |

## Logical vs. physical types

ODCS separates the **logical type** (`logicalType`, e.g. `string`, `integer`, `number`, `boolean`, `date`, `timestamp`) from the **physical type** (`physicalType`, e.g. `varchar`, `uuid`, `INT64`).

- The CLI uses the logical type as the portable, server-independent description.
- When you select a server (via `--server` or the server `type`), the CLI maps logical types to that backend's physical types for [exports](./exports/index.md) and [tests](./testing/index.md).
- You can always override the physical type per field, or pin a backend-specific type via `customProperties` / `config` (for example `clickhouseType`, `trinoType`).

## Working with ODCS in the CLI

- **Author** contracts with [`datacontract init`](./commands/init.md) and the [Data Contract Editor](./editor.md).
- **Validate** the structure against the ODCS JSON Schema with [`datacontract lint`](./commands/lint.md). Pass `--json-schema` to use a specific schema version.
- **Compare** two versions of a contract with [`datacontract changelog`](./commands/changelog.md).
- **Generate** a contract from an existing system using the [importers](./imports/index.md).

## Learn more

- ODCS specification: [bitol-io.github.io/open-data-contract-standard](https://bitol-io.github.io/open-data-contract-standard/latest/)
- Data contracts overview: [datacontract.com](https://datacontract.com)

---

## Migrate from DCS to ODCS


Contracts that start with `dataContractSpecification:` and describe their structure under
`models:`/`fields:` use the **Data Contract Specification (DCS)**, the format the CLI was
originally built around. The CLI now uses the
**[Open Data Contract Standard (ODCS)](./open-data-contract-standard.md)** natively.

:::note[Your DCS contracts keep working]
`lint`, `test`, and `export` detect a DCS file and convert it in memory before doing their work.
Migrating just makes the ODCS version the file you keep in Git.
:::

## Convert

```bash
datacontract export odcs datacontract.yaml --output datacontract.odcs.yaml
```

For a whole repository of contracts:

```bash
find . -name 'datacontract.yaml' -exec sh -c \
  'datacontract export odcs "$1" --output "$(dirname "$1")/datacontract.odcs.yaml"' _ {} \;
```

## Verify

```bash
# 1. The new file is valid ODCS.
datacontract lint datacontract.odcs.yaml

# 2. Nothing changed semantically — both files are compared as ODCS,
#    so an empty table means the conversion was lossless.
datacontract changelog datacontract.yaml datacontract.odcs.yaml

# 3. The new file still passes against real data.
datacontract test datacontract.odcs.yaml
```

Once all three pass, delete the DCS file and point your [CI/CD pipeline](./ci-cd.md) at the new
one.

## Three things to fix by hand

- **`status` is required in ODCS.** Without `info.status` in the DCS file, `lint` reports
  `data must contain ['status'] properties`. Add a top-level `status: active`.
- **The deprecated `primary: true` is not carried over** — only `primaryKey: true` is. Search
  your contracts for `primary:` and set `primaryKey: true` in the ODCS file.
- **`physicalType` holds the DCS `type`**, so a column reads `physicalType: string` instead of
  its native type. Replace it with the real type (`VARCHAR`, `NUMBER(12,2)`, …) or drop it and
  keep only `logicalType`.

Everything else is mapped automatically: `models` → `schema`, `fields` → `properties`,
`servicelevels` → `slaProperties`, `terms` → `description`, `field.enum` → a
[quality rule](./quality-rules/index.md), `$ref` definitions inlined. Anything without an ODCS
equivalent (contact details, Kafka `topic`, `pii`, `precision`) is preserved under
`customProperties`.

## Going back

[`datacontract export dcs`](./exports/dcs.md) converts the other way, if a downstream tool still
expects DCS:

```bash
datacontract export dcs datacontract.odcs.yaml --output datacontract.yaml
```

---

## Installation


Python 3.10, 3.11, and 3.12 are supported. We recommend Python 3.11.

The `[all]` extra installs every optional data-source dependency. To keep the install small, replace it with just the [extras](#optional-dependencies-extras) you need.

## uv (recommended)

The preferred way to install is [uv](https://docs.astral.sh/uv/):

```bash
uv tool install --python python3.11 --upgrade 'datacontract-cli[all]'
datacontract --version
```

## uvx (run without installing)

If you have [uv](https://docs.astral.sh/uv/) installed, you can run the CLI directly without installing it:

```bash
uv run --with 'datacontract-cli[all]' datacontract --version
```

## pip

```bash
python3 -m pip install 'datacontract-cli[all]'
datacontract --version
```

## pip with venv

Typically it's better to install into a virtual environment for your project:

```bash
cd my-project
python3.11 -m venv venv
source venv/bin/activate
pip install 'datacontract-cli[all]'
datacontract --version
```

## pipx

`pipx` installs into an isolated environment:

```bash
pipx install 'datacontract-cli[all]'
datacontract --version
```

## Docker

Use the Docker image to run the CLI without a local Python install — convenient for CI/CD:

```bash
docker pull datacontract/cli
docker run --rm -v "${PWD}:/home/datacontract" datacontract/cli
```

Create an alias to make it easier to use:

```bash
alias datacontract='docker run --rm -v "${PWD}:/home/datacontract" datacontract/cli:latest'
```

:::note
The output of Docker command line messages is limited to 80 columns and may include line breaks. Don't pipe Docker output to files if you want to export code — use the `--output` option instead.
:::

## Optional dependencies (extras)

The CLI defines several optional dependencies (extras) for specific server types. With `all`, every server dependency is included.

```bash
uv tool install --python python3.11 --upgrade 'datacontract-cli[all]'
```

Available extras:

| Dependency | Installation command |
|---|---|
| Amazon Athena | `pip install datacontract-cli[athena]` |
| Avro support | `pip install datacontract-cli[avro]` |
| Azure integration | `pip install datacontract-cli[azure]` |
| Google BigQuery | `pip install datacontract-cli[bigquery]` |
| CSV | `pip install datacontract-cli[csv]` |
| Databricks integration | `pip install datacontract-cli[databricks]` |
| DBML | `pip install datacontract-cli[dbml]` |
| DuckDB (local/S3/GCS/Azure file testing) | `pip install datacontract-cli[duckdb]` |
| Excel | `pip install datacontract-cli[excel]` |
| GCS integration | `pip install datacontract-cli[gcs]` |
| Iceberg | `pip install datacontract-cli[iceberg]` |
| Impala | `pip install datacontract-cli[impala]` |
| Kafka integration | `pip install datacontract-cli[kafka]` |
| MySQL integration | `pip install datacontract-cli[mysql]` |
| Oracle | `pip install datacontract-cli[oracle]` |
| Parquet | `pip install datacontract-cli[parquet]` |
| PostgreSQL integration | `pip install datacontract-cli[postgres]` |
| protobuf | `pip install datacontract-cli[protobuf]` |
| RDF | `pip install datacontract-cli[rdf]` |
| Amazon Redshift | `pip install datacontract-cli[redshift]` |
| S3 integration | `pip install datacontract-cli[s3]` |
| Snowflake integration | `pip install datacontract-cli[snowflake]` |
| Microsoft SQL Server | `pip install datacontract-cli[sqlserver]` |
| Trino | `pip install datacontract-cli[trino]` |
| API (run as web server) | `pip install datacontract-cli[api]` |

Each [data source](./testing/index.md) lists the extra it needs.

---

## Edit your contract


The [Data Contract Editor](https://github.com/datacontract/datacontract-editor) is a web-based visual editor for ODCS data contracts. It is hosted at [editor.datacontract.com](https://editor.datacontract.com), and the CLI can launch it locally against a file on your machine.

## Edit a local file

```bash
datacontract edit odcs.yaml
```

This starts a local web server and opens the Data Contract Editor for the given file in your browser. The editor is bundled with the CLI, so **no internet access is required**. Saving in the editor writes directly back to the local file.

Key behaviors:

- If the file does not exist, you are asked whether to initialize a new data contract.
- If a **URL** is given, you are asked whether to download a local copy, which is then edited.
- The server also acts as the editor's **test runner**: clicking "Run test" in the editor executes the data contract tests locally against the servers defined in the contract. Credentials for the data sources must be provided as environment variables — see [Test your Data](./testing/index.md#configuring-the-connection).

## Requirements

The `edit` command requires the `api` extra:

```bash
pip install 'datacontract-cli[api]'
```

The editor assets (JS/CSS) are bundled with the CLI and work offline by default. You can change where they are loaded from:

- `--editor-version` — load a specific version of the `datacontract-editor` npm package from the CDN, e.g. `0.1.9` or `latest`.
- `--editor-assets-url` — load assets from a self-hosted editor build. Takes precedence over `--editor-version`.

## Options

| Option | Default | Description |
|---|---|---|
| `--port` | `4243` | Bind the local server to this port. |
| `--host` | `127.0.0.1` | Bind to this host. For Docker, set it to `0.0.0.0`. |
| `--editor-version` | bundled | Version of the `datacontract-editor` npm package to load from the CDN. |
| `--editor-assets-url` | — | Base URL to load editor assets from (e.g. a self-hosted build). |
| `--open` / `--no-open` | `--open` | Open the editor in the default browser. |
| `--debug` / `--no-debug` | `--no-debug` | Enable debug logging. |

## Example

```bash
datacontract edit datacontract.yaml
```

See the [`edit` command reference](./commands/edit.md) for the full signature.

---

## Test your Data


`datacontract test` connects to the data source defined in the contract's `servers` block and runs **schema** and **quality** tests to verify that the actual data complies with the data contract.

```bash
datacontract test --server production datacontract.yaml
```

## Supported connections

  
    
    SnowflakeImport a contract from your tables and test in 5 minutes
  
  
    
    Google BigQueryImport a contract from your tables and test in 5 minutes
  
  
    
    DatabricksImport a contract from Unity Catalog and test in 5 minutes
  
  
    
    Amazon RedshiftImport a contract from your tables and test in 5 minutes
  
  
    
    PostgresPostgres and Postgres-compatible (e.g. RisingWave)
  
  
    
    Amazon S3CSV, JSON, Delta, Parquet on S3 / S3-compatible storage
  
  
    
    Local filesTry it in 60 seconds — no credentials needed
  
  
    
    Amazon AthenaAthena over data in S3
  
  
    
    Apache ImpalaImpala
  
  
    
    Azure Blob / ADLSFiles on Azure Blob storage or ADLS Gen2
  
  
    
    Google Cloud StorageFiles on GCS via S3 interoperability
  
  
    
    HTTP APIJSON HTTP APIs (GET only)
  
  
    
    KafkaKafka topics (experimental)
  
  
    
    Microsoft SQL ServerSQL Server, Azure SQL, Synapse, Fabric
  
  
    
    MySQLMySQL / MariaDB
  
  
    
    OracleOracle Database
  
  
    
    Spark DataFrameIn-memory Spark DataFrames (programmatic)
  
  
    
    TrinoTrino (basic, JWT, OAuth2)
  

:::tip
Missing a source? [Open an issue on GitHub](https://github.com/datacontract/datacontract-cli/issues).
:::

Each connection requires the matching [optional dependency (extra)](../installation.md#optional-dependencies-extras), or install everything with `datacontract-cli[all]`.

## How it works

The CLI uses different engines based on the server `type`. Internally it connects with **DuckDB**, **Spark**, or a native connection, executes most checks with [_ibis_](https://ibis-project.org/) (compiling dialect-specific SQL per backend), and validates JSON with [_fastjsonschema_](https://pypi.org/project/fastjsonschema/).

Checks fall into categories you can select with `--checks`:

- `schema` — fields are present and have the expected type and nullability.
- `quality` — the [quality rules](../quality-rules/index.md) defined in the contract.
- `servicelevel` — service-level expectations (`slaProperties`).
- `custom` — custom checks.

Omit `--checks` to run all of them.

## Configuring the connection

The connection details (host, catalog, location, …) live in the contract's `servers` block; **credentials are provided as environment variables**.

```yaml
servers:
  - server: production
    type: postgres   # selects the connection engine
    host: localhost
    port: 5432
    database: postgres
    schema: public
```

Environment variables are also loaded from a `.env` file in the current working directory (or the nearest parent directory containing one). Already-set environment variables take precedence over values from `.env`.

```bash
# .env
DATACONTRACT_POSTGRES_USERNAME=postgres
DATACONTRACT_POSTGRES_PASSWORD=postgres
```

The page for each source above lists its `servers` fields and the environment variables it expects.

## Options

`--server`, `--schema-name`, and `--checks` narrow down what runs. `--output` with `--output-format` writes the results to a file as `json` or `junit`, `--publish` sends them to a URL, and `--include-failed-samples` collects a small sample of the offending rows. See the full [`test` command reference](../commands/test.md).

For CI/CD pipelines, use the [`ci`](../commands/ci.md) command, which wraps `test` with annotations, summaries, and exit-code control.

## Next steps

- Run the tests from Python instead of the CLI: **[Python Library](../python-library.md)**.
- Keep the tests running automatically: **[Scheduling and CI/CD](../ci-cd.md)**.
- Roll data contracts out across a team: **[Adopting Data Contracts](../best-practices.md)**.

---

## HTTP API


Test APIs that return data in JSON format. Currently, only GET requests are supported.

## 1. Install

No extra is required for API connections:

```bash
uv tool install --python python3.11 --upgrade datacontract-cli
```

See [Installation](../installation.md) for pip, pipx, and Docker.

## 2. Set credentials

If the API requires authentication, set the value for the `authorization` header:

```bash
# .env
DATACONTRACT_API_HEADER_AUTHORIZATION="Bearer <token>"
```

For public endpoints, skip this step.

## 3. Create a contract from a response

Fetch one response and import its schema, then point the generated `servers` block at the endpoint:

```bash
curl -o response.json https://api.example.com/orders
datacontract import json --source response.json --output datacontract.yaml
```

The import generates a `servers` entry of `type: local`. Replace it with the API endpoint:

```yaml
servers:
  - server: api
    type: api
    location: "https://api.example.com/orders"
    delimiter: none # new_line, array, or none (default)
```

## 4. Test the actual data

```bash
datacontract test datacontract.yaml
```

```
Testing datacontract.yaml
Server: production (type=api, format=json, location=https://api.example.com/orders)
╭────────┬─────────────────────────────────────────────────┬─────────────────┬─────────╮
│ Result │ Check                                           │ Field           │ Details │
├────────┼─────────────────────────────────────────────────┼─────────────────┼─────────┤
│ passed │ Check that field 'order_id' is present          │ orders.order_id │         │
│ passed │ Check that field order_id has no missing values │ orders.order_id │         │
│  ...   │                                                 │                 │         │
╰────────┴─────────────────────────────────────────────────┴─────────────────┴─────────╯
🟢 data contract is valid. Run 24 checks. Took 1.2 seconds.
```

## 5. Let it catch a violation

The contract becomes valuable when it detects drift. Tighten an expectation — for example, mark a field as `required: true`, restrict a status field to its allowed values, or add a quality rule. Run `datacontract test datacontract.yaml` again: every violation is listed as an error, and the command exits with code `1` — ready for [CI/CD scheduling](../ci-cd.md) so you catch drift before your consumers do.

## Reference

Authentication options and data type handling: **[HTTP API Reference](../reference/api.md)**.

## Troubleshooting

- **`401` / `403`** — set `DATACONTRACT_API_HEADER_AUTHORIZATION` including the scheme (e.g. `Bearer eyJ...`), not just the raw token.
- **Schema checks fail on a wrapped response** — if the API returns `{"data": [...]}` instead of a plain array, model the wrapper object in the schema, or set `delimiter` accordingly (`array` for a JSON array of records, `none` for a single object).

---

## Amazon Athena


Go from an existing Athena table to a tested data contract in about five minutes: import the schema straight from the catalog, then test the actual data against it. Athena reads data in S3, so this covers formats such as Iceberg, Parquet, JSON, and CSV.

## 1. Install

```bash
uv tool install --python python3.11 --upgrade 'datacontract-cli[athena]'
```

See [Installation](../installation.md) for pip, pipx, and Docker.

## 2. Authenticate

The easiest way is to sign in to AWS once — the CLI picks the session up, so no key is stored anywhere and nothing else has to be configured:

```bash
aws sso login   # or any other way of getting an AWS session
```

Prefer static keys? Set them directly and they are used instead:

```bash
# .env
DATACONTRACT_S3_REGION=eu-central-1
DATACONTRACT_S3_ACCESS_KEY_ID=AKIAXV5Q5QABCDEFGH
DATACONTRACT_S3_SECRET_ACCESS_KEY=93S7LRrJcqLaaaa/XXXXXXXXXXXXX
```

Either way, your AWS identity needs `glue:GetTables` to import, and `athena:StartQueryExecution` plus read access to the data and write access to the staging directory to test. `import` and `test` use the same setup.

## 3. Create a contract from your tables

Import the table metadata directly from the catalog. This also generates a ready-to-test `servers` block:

```bash
datacontract import athena \
  --schema my_database \
  --staging-dir s3://my-bucket/athena-results/ \
  --region eu-central-1 \
  --table orders \
  --output datacontract.yaml
```

`--schema` is the Athena database. Repeat `--table` for multiple tables, or omit it to import every table in the database. `--catalog` defaults to `awsdatacatalog`.

`--staging-dir` is where Athena writes query results; `datacontract test` needs it, so the import asks for it up front and writes it into the contract.

## 4. Test the actual data

```bash
datacontract test datacontract.yaml
```

```
Testing datacontract.yaml
Server: athena (type=athena, catalog=awsdatacatalog, schema=my_database, regionName=eu-central-1)
╭────────┬─────────────────────────────────────────────────┬─────────────────┬─────────╮
│ Result │ Check                                           │ Field           │ Details │
├────────┼─────────────────────────────────────────────────┼─────────────────┼─────────┤
│ passed │ Check that field 'order_id' is present          │ orders.order_id │         │
│ passed │ Check that field order_id has no missing values │ orders.order_id │         │
│  ...   │                                                 │                 │         │
╰────────┴─────────────────────────────────────────────────┴─────────────────┴─────────╯
🟢 data contract is valid. Run 24 checks. Took 7.8 seconds.
```

## 5. Let it catch a violation

The contract becomes valuable when it detects drift. Tighten an expectation — for example, add a quality rule to a schema in `datacontract.yaml`:

```yaml
schema:
  - name: orders
    # ...
    quality:
      - type: sql
        description: No order has a negative total
        query: SELECT COUNT(*) FROM orders WHERE order_total < 0
        mustBe: 0
```

Run `datacontract test datacontract.yaml` again: every violation is listed as an error, and the command exits with code `1` — ready for [CI/CD scheduling](../ci-cd.md) so you catch drift before your consumers do.

## Reference

All authentication options and the data type mappings: **[Athena Reference](../reference/athena.md)**.

## Troubleshooting

- **`Access Denied` on query start** — the credentials need `athena:StartQueryExecution` plus write access to the `stagingDir` bucket.
- **`Table not found`** — `schema` in the `servers` block must be the Athena *database* name (as shown in the Glue Data Catalog), and `regionName` must match where the catalog lives.
- **`Your session has expired`** — the AWS session has lapsed; run `aws sso login` again. No `DATACONTRACT_S3_*` variable is needed when a session is present.

---

## Azure Blob / ADLS


Test data stored in Azure Blob storage or Azure Data Lake Storage Gen2 (ADLS) in various formats.

## 1. Install

```bash
uv tool install --python python3.11 --upgrade 'datacontract-cli[azure,duckdb]'
```

See [Installation](../installation.md) for pip, pipx, and Docker.

## 2. Set credentials

Authentication uses an Azure Service Principal (App Registration) with a secret. Create a `.env` file in your working directory (or export the variables):

```bash
# .env
DATACONTRACT_AZURE_TENANT_ID=79f5b80f-10ff-40b9-9d1f-774b42d605fc
DATACONTRACT_AZURE_CLIENT_ID=3cf7ce49-e2e9-4cbc-a922-4328d4a58622
DATACONTRACT_AZURE_CLIENT_SECRET=yZK8Q~GWO1MMXXXXXXXXXXXXX
```

## 3. Create a contract from your files

Import the schema straight from the container. This also generates a ready-to-test `servers` block:

```bash
datacontract import adls \
  --source abfss://my-container/orders/*.json \
  --output datacontract.yaml
```

The format is taken from the file suffix; pass `--format` for Delta tables, which have none.

## 4. Test the actual data

```bash
datacontract test datacontract.yaml
```

```
Testing datacontract.yaml
Server: production (type=azure, format=parquet, location=abfss://my-container/orders/*.parquet)
╭────────┬─────────────────────────────────────────────────┬─────────────────┬─────────╮
│ Result │ Check                                           │ Field           │ Details │
├────────┼─────────────────────────────────────────────────┼─────────────────┼─────────┤
│ passed │ Check that field 'order_id' is present          │ orders.order_id │         │
│ passed │ Check that field order_id has no missing values │ orders.order_id │         │
│  ...   │                                                 │                 │         │
╰────────┴─────────────────────────────────────────────────┴─────────────────┴─────────╯
🟢 data contract is valid. Run 24 checks. Took 4.5 seconds.
```

## 5. Let it catch a violation

The contract becomes valuable when it detects drift. Tighten an expectation — for example, mark a field as `required: true` that occasionally arrives empty, or add a quality rule:

```yaml
schema:
  - name: inventory_events
    # ...
    quality:
      - type: sql
        description: No event has a negative quantity
        query: SELECT COUNT(*) FROM inventory_events WHERE quantity < 0
        mustBe: 0
```

Run `datacontract test datacontract.yaml` again: every violation is listed as an error, and the command exits with code `1` — ready for [CI/CD scheduling](../ci-cd.md) so you catch drift before your consumers do.

## Reference

All authentication options (service principal, connection string, account key), supported location URL formats, and the data type handling per file format: **[Azure Reference](../reference/azure.md)**.

## Troubleshooting

- **`AuthorizationPermissionMismatch` / `403`** — the service principal needs the **Storage Blob Data Reader** role on the container or storage account (IAM role assignment, not just API permissions).
- **`No files found that match the pattern`** — check the `location` URL format (see below) and the glob; it matches blob names under the prefix.

## Metadata checks

Instead of reading the file contents, you can validate the metadata of the blobs themselves
(size, content type, last modified, file count, …). Each ODCS schema object with
`physicalType: file` represents a **unique folder/prefix** in Azure Blob storage or ADLS Gen2.
Its `properties` map directly to `BlobProperties` attributes from the Azure SDK: for every
declared property the engine extracts the corresponding attribute from each blob and validates
it against the quality constraints declared on that property.

| Property name | `BlobProperties` attribute |
|---|---|
| `name` | `blob.name` |
| `size` | `blob.size` |
| `lastModified` | `blob.last_modified` (UTC datetime) |
| `creationTime` | `blob.creation_time` (UTC datetime) |
| `lastAccessedOn` | `blob.last_accessed_on` (UTC datetime) |
| `contentType` | `blob.content_settings.content_type` |
| `contentEncoding` | `blob.content_settings.content_encoding` |
| `contentLanguage` | `blob.content_settings.content_language` |
| `contentDisposition` | `blob.content_settings.content_disposition` |
| `cacheControl` | `blob.content_settings.cache_control` |
| `contentMd5` | `blob.content_settings.content_md5` |
| `etag` | `blob.etag` |
| `blobType` | `blob.blob_type.value` (e.g. `BlockBlob`) |
| `blobTier` | `blob.blob_tier.value` (e.g. `Hot`) |
| `archiveStatus` | `blob.archive_status` |
| `serverEncrypted` | `blob.server_encrypted` |
| `deleted` | `blob.deleted` |
| `snapshotId` | `blob.snapshot` |
| `versionId` | `blob.version_id` |
| `tagCount` | `blob.tag_count` |

Schema-level quality checks (`schema.quality`) on the `rowCount` metric are evaluated as
**file-count** thresholds against the total number of blobs found under the prefix.

`contentType` is normalised before comparison: MIME parameters are stripped, so
`application/json; charset=utf-8` matches `application/json`.

Supported `location` URL formats (on the server block):

- `https://<account>.blob.core.windows.net/<container>/<prefix>`
- `abfss://<container>@<account>.dfs.core.windows.net/<prefix>`
- `azure://<container>@<account>.blob.core.windows.net/<prefix>`
- `wasbs://<container>@<account>.blob.core.windows.net/<prefix>`

The metadata-check path also accepts `DATACONTRACT_AZURE_CONNECTION_STRING` or `DATACONTRACT_AZURE_STORAGE_ACCOUNT_KEY` instead of the service principal variables.

---

## Google BigQuery


Go from an existing BigQuery table to a tested data contract in about five minutes: import the schema straight from BigQuery, then test the actual data against it.

## 1. Install

```bash
uv tool install --python python3.11 --upgrade 'datacontract-cli[bigquery]'
```

See [Installation](../installation.md) for pip, pipx, and Docker.

## 2. Authenticate

The easiest way is Application Default Credentials (ADC) — both `import` and `test` pick them up automatically:

```bash
gcloud auth application-default login
```

Your account (or service account) needs the **BigQuery Job User** and **BigQuery Data Viewer** roles. For service-account key files and CI/CD, see the [BigQuery Reference](../reference/bigquery.md).

## 3. Create a contract from your tables

Import the table metadata directly from the BigQuery API. This also generates a ready-to-test `servers` block:

```bash
datacontract import bigquery \
  --project my-project \
  --dataset my_dataset \
  --table orders \
  --output datacontract.yaml
```

Repeat `--table` for multiple tables, or omit it to import every table in the dataset.

## 4. Test the actual data

```bash
datacontract test datacontract.yaml
```

```
Testing datacontract.yaml
Server: bigquery (type=bigquery, project=my-project, dataset=my_dataset)
╭────────┬─────────────────────────────────────────────────┬─────────────────┬─────────╮
│ Result │ Check                                           │ Field           │ Details │
├────────┼─────────────────────────────────────────────────┼─────────────────┼─────────┤
│ passed │ Check that field 'order_id' is present          │ orders.order_id │         │
│ passed │ Check that field order_id has no missing values │ orders.order_id │         │
│  ...   │                                                 │                 │         │
╰────────┴─────────────────────────────────────────────────┴─────────────────┴─────────╯
🟢 data contract is valid. Run 24 checks. Took 6.1 seconds.
```

## 5. Let it catch a violation

The contract becomes valuable when it detects drift. Tighten an expectation — for example, add a quality rule to a schema in `datacontract.yaml`:

```yaml
schema:
  - name: orders
    # ...
    quality:
      - type: sql
        description: No order has a negative total
        query: SELECT COUNT(*) FROM orders WHERE order_total < 0
        mustBe: 0
```

Run `datacontract test datacontract.yaml` again: every violation is listed as an error, and the command exits with code `1` — ready for [CI/CD scheduling](../ci-cd.md) so you catch drift before your consumers do.

## Reference

All authentication options (service-account keys, WIF, billing project) and the data type mappings: **[BigQuery Reference](../reference/bigquery.md)**.

## Troubleshooting

- **`Your default credentials were not found`** — run `gcloud auth application-default login`, or set `GOOGLE_APPLICATION_CREDENTIALS` / `DATACONTRACT_BIGQUERY_ACCOUNT_INFO_JSON_PATH` to a service-account key file.
- **`403 Access Denied`** — the account is missing **BigQuery Job User** (to run query jobs) or **BigQuery Data Viewer** (to read the tables).
- **Queries billed to the wrong project** — set `DATACONTRACT_BIGQUERY_BILLING_PROJECT`.

---

## Databricks


Go from an existing Unity Catalog table to a tested data contract in about five minutes: import the schema from Unity Catalog, then test the actual data on a SQL warehouse. Works with Unity Catalog and the Hive metastore; testing needs a running SQL warehouse or compute cluster.

## 1. Install

```bash
uv tool install --python python3.11 --upgrade 'datacontract-cli[databricks]'
```

See [Installation](../installation.md) for pip, pipx, and Docker.

## 2. Set credentials

Create a `.env` file in your working directory (or export the variables):

```bash
# .env
DATACONTRACT_DATABRICKS_SERVER_HOSTNAME=dbc-abcdefgh-1234.cloud.databricks.com
DATACONTRACT_DATABRICKS_HTTP_PATH=/sql/1.0/warehouses/b053a3ff69ee87a8
DATACONTRACT_DATABRICKS_TOKEN=<your-personal-access-token>
```

Find the hostname and HTTP path under your SQL warehouse's **Connection details** tab. For OAuth service principals and profile-based auth, see the [Databricks Reference](../reference/databricks.md).

## 3. Create a contract from your tables

Import the table metadata directly from Unity Catalog. This also generates a ready-to-test `servers` block:

```bash
datacontract import databricks \
  --table my_catalog.my_schema.orders \
  --output datacontract.yaml
```

Repeat `--table` for multiple tables.

## 4. Test the actual data

```bash
datacontract test datacontract.yaml
```

```
Testing datacontract.yaml
Server: databricks (type=databricks, catalog=my_catalog, schema=my_schema)
╭────────┬─────────────────────────────────────────────────┬─────────────────┬─────────╮
│ Result │ Check                                           │ Field           │ Details │
├────────┼─────────────────────────────────────────────────┼─────────────────┼─────────┤
│ passed │ Check that field 'order_id' is present          │ orders.order_id │         │
│ passed │ Check that field order_id has no missing values │ orders.order_id │         │
│  ...   │                                                 │                 │         │
╰────────┴─────────────────────────────────────────────────┴─────────────────┴─────────╯
🟢 data contract is valid. Run 24 checks. Took 8.4 seconds.
```

## 5. Let it catch a violation

The contract becomes valuable when it detects drift. Tighten an expectation — for example, add a quality rule to a schema in `datacontract.yaml`:

```yaml
schema:
  - name: orders
    # ...
    quality:
      - type: sql
        description: No order has a negative total
        query: SELECT COUNT(*) FROM orders WHERE order_total < 0
        mustBe: 0
```

Run `datacontract test datacontract.yaml` again: every violation is listed as an error, and the command exits with code `1` — ready for [CI/CD scheduling](../ci-cd.md) so you catch drift before your consumers do.

## Reference

All authentication options (PAT, OAuth M2M, profiles, precedence order) and the data type mappings: **[Databricks Reference](../reference/databricks.md)**.

## Programmatic (in a notebook or pipeline)

When running in a notebook or pipeline, pass the existing `spark` session — no extra authentication is required (requires Databricks Runtime with Python ≥ 3.10).

```python
from datacontract.data_contract import DataContract

data_contract = DataContract(
  data_contract_file="/Volumes/acme_catalog_prod/orders_latest/datacontract/datacontract.yaml",
  spark=spark)
run = data_contract.test()
run.result
```

:::tip[Installing on Databricks compute]
On Databricks LTS ML runtimes (15.4, 16.4), installing via `%pip install` in notebooks can cause issues. Instead, add `datacontract-cli[databricks]` as a **PyPI library** on the cluster (Compute → your cluster → Libraries → Install new → PyPI), then restart the cluster.
:::

## Troubleshooting

- **`Invalid HTTP path`, or the test hangs** — `DATACONTRACT_DATABRICKS_HTTP_PATH` must point at a running SQL warehouse or cluster (check the **Connection details** tab). `import databricks` works without it, but `test` requires it.
- **`403 Invalid access token`** — the PAT is expired or belongs to a different workspace than `DATACONTRACT_DATABRICKS_SERVER_HOSTNAME`.
- **`TABLE_OR_VIEW_NOT_FOUND`** — the token's principal lacks `USE CATALOG`/`USE SCHEMA`/`SELECT` privileges on the table.

---

## Spark DataFrame


Test Spark DataFrames in a pipeline before writing them to a data source. DataFrames are registered as named temporary views; multiple views are supported if the contract has multiple schemas. This connection is used programmatically from Python — no credentials are needed, the existing Spark session is reused.

## 1. Install

No extra is required (the Spark session comes from your environment):

```bash
uv tool install --python python3.11 --upgrade datacontract-cli
```

In a notebook or pipeline environment, install `datacontract-cli` as a library instead. See [Installation](../installation.md) for pip, pipx, and Docker.

## 2. Create a contract from your DataFrames

Inside an active Spark session, import the schema of registered tables or views:

```bash
datacontract import spark --tables my_table --output datacontract.yaml
```

The generated contract includes a `servers` entry of type `dataframe`:

```yaml
servers:
  - server: production
    type: dataframe
```

## 3. Test the DataFrame

Register the DataFrame as a temporary view named like the schema, then run the test with the Spark session:

```python
from datacontract.data_contract import DataContract

df.createOrReplaceTempView("my_table")

data_contract = DataContract(
  data_contract_file="datacontract.yaml",
  spark=spark,
)
run = data_contract.test()
assert run.result == "passed"
```

## 4. Let it catch a violation

The contract becomes valuable when it detects drift. Tighten an expectation — for example, mark a field as `required: true` or add a quality rule — and run the test again: `run.result` becomes `"failed"` and `run.checks` lists each violation, so the assert stops your pipeline before bad data is written.

## Reference

The Spark data type mappings: **[Spark DataFrame Reference](../reference/dataframe.md)**.

---

## Google Cloud Storage

#  Google Cloud Storage (GCS)

The [Amazon S3](./s3.md) integration also works with files on Google Cloud Storage through its [interoperability](https://cloud.google.com/storage/docs/interoperability). Use `https://storage.googleapis.com` as the endpoint URL and the `s3://` scheme for the location.

## 1. Install

```bash
uv tool install --python python3.11 --upgrade 'datacontract-cli[gcs]'
```

See [Installation](../installation.md) for pip, pipx, and Docker.

## 2. Set credentials

Create an [HMAC key](https://cloud.google.com/storage/docs/authentication/hmackeys) for your user or service account, then create a `.env` file in your working directory (or export the variables):

```bash
# .env
DATACONTRACT_S3_ACCESS_KEY_ID=GOOG1EZZZXXXXXXXXXXXXX
DATACONTRACT_S3_SECRET_ACCESS_KEY=PDWWpbXXXXXXXXXXXXX
```

## 3. Create a contract from your files

Import the schema straight from the bucket. This also generates a ready-to-test `servers` block:

```bash
datacontract import gcs \
  --source s3://my-bucket/orders/*.json \
  --output datacontract.yaml
```

duckdb reads Google Cloud Storage through its S3-compatible endpoint, so the location uses the `s3://` scheme rather than `gs://`. The format is taken from the file suffix; pass `--format` for Delta tables, which have none.

## 4. Test the actual data

```bash
datacontract test datacontract.yaml
```

```
Testing datacontract.yaml
Server: production (type=gcs, format=json, location=s3://my-bucket/orders/*.json)
╭────────┬─────────────────────────────────────────────────┬─────────────────┬─────────╮
│ Result │ Check                                           │ Field           │ Details │
├────────┼─────────────────────────────────────────────────┼─────────────────┼─────────┤
│ passed │ Check that field 'order_id' is present          │ orders.order_id │         │
│ passed │ Check that field order_id has no missing values │ orders.order_id │         │
│  ...   │                                                 │                 │         │
╰────────┴─────────────────────────────────────────────────┴─────────────────┴─────────╯
🟢 data contract is valid. Run 24 checks. Took 3.1 seconds.
```

## 5. Let it catch a violation

The contract becomes valuable when it detects drift. Tighten an expectation — for example, mark a field as `required: true` that occasionally arrives empty, or add a quality rule:

```yaml
schema:
  - name: orders
    # ...
    quality:
      - type: sql
        description: No order has a negative total
        query: SELECT COUNT(*) FROM orders WHERE order_total < 0
        mustBe: 0
```

Run `datacontract test datacontract.yaml` again: every violation is listed as an error, and the command exits with code `1` — ready for [CI/CD scheduling](../ci-cd.md) so you catch drift before your consumers do.

## Reference

All authentication options and the data type handling per file format: **[GCS Reference](../reference/gcs.md)**.

## Troubleshooting

- **`403 Forbidden`** — the HMAC key's principal lacks `storage.objects.get`/`storage.objects.list` on the bucket, or the key was deactivated.
- **`No files found that match the pattern`** — the `location` must use the `s3://` scheme (not `gs://`), and `endpointUrl` must be `https://storage.googleapis.com`.

---

## Apache Impala


Run checks against an Apache Impala cluster.

## 1. Install

```bash
uv tool install --python python3.11 --upgrade 'datacontract-cli[impala]'
```

See [Installation](../installation.md) for pip, pipx, and Docker.

## 2. Set credentials

Create a `.env` file in your working directory (or export the variables):

```bash
# .env
DATACONTRACT_IMPALA_USERNAME=analytics_user
DATACONTRACT_IMPALA_PASSWORD=mysecretpassword
```

## 3. Create a contract from your tables

Get the DDL of a table (`SHOW CREATE TABLE orders;` in impala-shell or Hue), save it to a file, and import it. Impala DDL is Hive-compatible, so use the `spark` dialect:

```bash
datacontract import sql --source orders.sql --dialect spark --output datacontract.yaml
```

The SQL import can't know your connection details, so it writes a `servers` block with placeholder values. Open `datacontract.yaml` and fill in your cluster:

```yaml
servers:
  - server: impala
    type: impala
    host: my-impala-host
    port: 443
    database: my_database # optional default database
```

## 4. Test the actual data

```bash
datacontract test datacontract.yaml
```

```
Testing datacontract.yaml
Server: production (type=impala, host=my-impala-host, port=443, database=my_database)
╭────────┬─────────────────────────────────────────────────┬─────────────────┬─────────╮
│ Result │ Check                                           │ Field           │ Details │
├────────┼─────────────────────────────────────────────────┼─────────────────┼─────────┤
│ passed │ Check that field 'order_id' is present          │ orders.order_id │         │
│ passed │ Check that field order_id has no missing values │ orders.order_id │         │
│  ...   │                                                 │                 │         │
╰────────┴─────────────────────────────────────────────────┴─────────────────┴─────────╯
🟢 data contract is valid. Run 24 checks. Took 2.8 seconds.
```

## 5. Let it catch a violation

The contract becomes valuable when it detects drift. Tighten an expectation — for example, add a quality rule to a schema in `datacontract.yaml`:

```yaml
schema:
  - name: orders
    # ...
    quality:
      - type: sql
        description: No order has a negative total
        query: SELECT COUNT(*) FROM orders WHERE order_total < 0
        mustBe: 0
```

Run `datacontract test datacontract.yaml` again: every violation is listed as an error, and the command exits with code `1` — ready for [CI/CD scheduling](../ci-cd.md) so you catch drift before your consumers do.

## Reference

All authentication options (SSL, transport, auth mechanism) and the data type mappings: **[Impala Reference](../reference/impala.md)**.

## Troubleshooting

- **Connection errors** — the defaults assume LDAP auth over SSL and HTTP transport (port 443, e.g. behind a load balancer). For a plain binary-protocol cluster, set `DATACONTRACT_IMPALA_USE_HTTP_TRANSPORT=false` and use port `21050`.
- **`AuthorizationException`** — the user lacks `SELECT` on the table or the Ranger/Sentry policy doesn't cover the database in the `servers` block.

---

## Kafka


Test data in Kafka topics. Kafka support is currently considered **experimental**.

## 1. Install

```bash
uv tool install --python python3.11 --upgrade 'datacontract-cli[kafka]'
```

Kafka checks run on Spark, which requires a **Java runtime (JDK 17 or 21)** — make sure `java` is on the path or `JAVA_HOME` is set. See [Installation](../installation.md) for pip, pipx, and Docker.

## 2. Set credentials

Create a `.env` file in your working directory (or export the variables):

```bash
# .env
DATACONTRACT_KAFKA_SASL_USERNAME=mykey
DATACONTRACT_KAFKA_SASL_PASSWORD=mysecret
```

If no username/password is set, the CLI connects without authentication (e.g. a local broker).

## 3. Create a contract for your topic

If you have an Avro schema for the topic (e.g. from a schema registry), import it:

```bash
datacontract import avro --source orders.avsc --output datacontract.yaml
```

Then add a `servers` entry pointing at your broker and topic:

```yaml
servers:
  - server: production
    type: kafka
    host: abc-12345.eu-central-1.aws.confluent.cloud:9092
    topic: my-topic-name
    format: json # or avro
```

## 4. Test the actual data

```bash
datacontract test datacontract.yaml
```

```
Testing datacontract.yaml
Server: production (type=kafka, format=json, host=abc-12345.eu-central-1.aws.confluent.cloud:9092)
╭────────┬─────────────────────────────────────────────────┬─────────────────┬─────────╮
│ Result │ Check                                           │ Field           │ Details │
├────────┼─────────────────────────────────────────────────┼─────────────────┼─────────┤
│ passed │ Check that field 'order_id' is present          │ orders.order_id │         │
│ passed │ Check that field order_id has no missing values │ orders.order_id │         │
│  ...   │                                                 │                 │         │
╰────────┴─────────────────────────────────────────────────┴─────────────────┴─────────╯
🟢 data contract is valid. Run 24 checks. Took 8.4 seconds.
```

## 5. Let it catch a violation

The contract becomes valuable when it detects drift. Tighten an expectation — for example, mark a field as `required: true` or restrict a field to its allowed values. Run `datacontract test datacontract.yaml` again: every violation is listed as an error, and the command exits with code `1` — ready for [CI/CD scheduling](../ci-cd.md) so you catch drift before your consumers do.

## Reference

All authentication options (SASL mechanisms) and the Avro data type mappings: **[Kafka Reference](../reference/kafka.md)**.

## Troubleshooting

- **`JAVA_HOME is not set` / `Unable to locate a Java Runtime`** — install a JDK (17 or 21) and set `JAVA_HOME`; the Kafka checks run on Spark.
- **Authentication failures against Confluent Cloud** — use an API key/secret as `SASL_USERNAME`/`SASL_PASSWORD` with the default `PLAIN` mechanism.
- **The test reads no messages** — the check consumes the topic from the beginning; verify the topic name in the `servers` block and that the topic contains messages in the declared `format`.

---

## Local files


Test local files in Parquet, JSON, CSV, or Delta format. This is the fastest way to see the CLI in action — no warehouse, no credentials.

## 1. Install

```bash
uv tool install --python python3.11 --upgrade 'datacontract-cli[duckdb]'
```

See [Installation](../installation.md) for pip, pipx, and Docker.

## 2. Create a contract from a file

Take any CSV file — or use this one, saved as `orders.csv`:

```text
order_id,order_timestamp,customer_id,order_total,status
ORD-1001,2024-01-01T10:00:00Z,CUST-1,4999,delivered
ORD-1002,2024-01-02T11:30:00Z,CUST-2,2500,shipped
ORD-1003,2024-01-03T09:15:00Z,CUST-3,1299,pending
```

Import it. The CLI profiles the data and generates a contract with a schema, value ranges, and a ready-to-test `servers` block:

```bash
datacontract import csv --source orders.csv --output datacontract.yaml
```

## 3. Test the data against the contract

```bash
datacontract test datacontract.yaml
```

```
Testing datacontract.yaml
Server: production (type=local, format=csv, path=orders.csv)
╭────────┬──────────────────────────────────────────────────────┬──────────────┬─────────╮
│ Result │ Check                                                │ Field        │ Details │
├────────┼──────────────────────────────────────────────────────┼──────────────┼─────────┤
│ passed │ Check that field 'order_id' is present               │ order_id     │         │
│ passed │ Check that field order_id has no missing values      │ order_id     │         │
│ passed │ Check that field order_total has a minimum of 1299.0 │ order_total  │         │
│  ...   │                                                      │              │         │
╰────────┴──────────────────────────────────────────────────────┴──────────────┴─────────╯
🟢 data contract is valid. Run 17 checks. Took 1.2 seconds.
```

## 4. Let it catch a violation

Now break the data — append a row with a negative total and a duplicate customer:

```bash
echo 'ORD-1004,2024-01-04T12:00:00Z,CUST-1,-100,delivered' >> orders.csv
datacontract test datacontract.yaml
```

```
🔴 data contract is invalid, found the following errors:
1) customer_id Check that unique field customer_id has no duplicate values:
Actual duplicate_count(customer_id) was 1, expected = 0
2) order_total Check that field order_total has a minimum of 1299.0: Actual
invalid_count(order_total) was 1, expected = 0
```

The command exits with code `1`, so the same call works as a gate in [CI/CD pipelines](../ci-cd.md).

Ready for your real data? Do the same against [Snowflake](./snowflake.md), [BigQuery](./bigquery.md), or [Databricks](./databricks.md).

---

## MySQL


Test data in MySQL or MySQL-compatible databases (e.g. MariaDB).

## 1. Install

```bash
uv tool install --python python3.11 --upgrade 'datacontract-cli[mysql]'
```

See [Installation](../installation.md) for pip, pipx, and Docker.

## 2. Authenticate

Create a `.env` file in your working directory (or export the variables):

```bash
# .env
DATACONTRACT_MYSQL_USERNAME=root
DATACONTRACT_MYSQL_PASSWORD=mysecretpassword
```

## 3. Create a contract from your tables

Import the table metadata directly from the database. This also generates a ready-to-test `servers` block:

```bash
datacontract import mysql \
  --source localhost \
  --database mydb \
  --table orders \
  --output datacontract.yaml
```

`--source` is the host of your MySQL server. Add `--port` if it doesn't listen on the default `3306`, repeat `--table` for multiple tables, or omit it to import every table in the database.

Only have a DDL file? `datacontract import sql --source orders.sql --dialect mysql` works too, but writes a `servers` block with placeholder values that you have to fill in by hand.

## 4. Test the actual data

```bash
datacontract test datacontract.yaml
```

```
Testing datacontract.yaml
Server: mysql (type=mysql, host=localhost, port=3306, database=mydb)
╭────────┬─────────────────────────────────────────────────┬─────────────────┬─────────╮
│ Result │ Check                                           │ Field           │ Details │
├────────┼─────────────────────────────────────────────────┼─────────────────┼─────────┤
│ passed │ Check that field 'order_id' is present          │ orders.order_id │         │
│ passed │ Check that field order_id has no missing values │ orders.order_id │         │
│  ...   │                                                 │                 │         │
╰────────┴─────────────────────────────────────────────────┴─────────────────┴─────────╯
🟢 data contract is valid. Run 24 checks. Took 2.1 seconds.
```

## 5. Let it catch a violation

The contract becomes valuable when it detects drift. Tighten an expectation — for example, add a quality rule to a schema in `datacontract.yaml`:

```yaml
schema:
  - name: orders
    # ...
    quality:
      - type: sql
        description: No order has a negative total
        query: SELECT COUNT(*) FROM orders WHERE order_total < 0
        mustBe: 0
```

Run `datacontract test datacontract.yaml` again: every violation is listed as an error, and the command exits with code `1` — ready for [CI/CD scheduling](../ci-cd.md) so you catch drift before your consumers do.

## Reference

All authentication options and the data type mappings: **[MySQL Reference](../reference/mysql.md)**.

## Troubleshooting

- **`Access denied for user`** — check the two environment variables above and the user's host restrictions (`'user'@'%'` vs `'user'@'localhost'`).
- **`Can't connect to MySQL server`** — host/port in the `servers` block are wrong, or the server doesn't accept remote connections (`bind-address`).

---

## Oracle


Test data in Oracle Database.

## 1. Install

```bash
uv tool install --python python3.11 --upgrade 'datacontract-cli[oracle]'
```

See [Installation](../installation.md) for pip, pipx, and Docker.

## 2. Authenticate

Create a `.env` file in your working directory (or export the variables):

```bash
# .env
DATACONTRACT_ORACLE_USERNAME=system
DATACONTRACT_ORACLE_PASSWORD=mysecretpassword
```

## 3. Create a contract from your tables

Import the table metadata directly from the database. This also generates a ready-to-test `servers` block:

```bash
datacontract import oracle \
  --source localhost \
  --service-name ORCL \
  --schema ADMIN \
  --table ORDERS \
  --output datacontract.yaml
```

`--source` is the host and `--service-name` the service. Repeat `--table` for multiple tables, or omit it to import every table in the schema; Oracle upper-cases identifiers, so `--schema` is upper-cased for you.

Only have a DDL script? `datacontract import sql --source orders.sql --dialect oracle` works too, but writes a `servers` block with placeholder values that you have to fill in by hand.

## 4. Test the actual data

```bash
datacontract test datacontract.yaml
```

```
Testing datacontract.yaml
Server: oracle (type=oracle, host=localhost, port=1521, schema=ADMIN)
╭────────┬─────────────────────────────────────────────────┬─────────────────┬─────────╮
│ Result │ Check                                           │ Field           │ Details │
├────────┼─────────────────────────────────────────────────┼─────────────────┼─────────┤
│ passed │ Check that field 'order_id' is present          │ orders.order_id │         │
│ passed │ Check that field order_id has no missing values │ orders.order_id │         │
│  ...   │                                                 │                 │         │
╰────────┴─────────────────────────────────────────────────┴─────────────────┴─────────╯
🟢 data contract is valid. Run 24 checks. Took 3.4 seconds.
```

## 5. Let it catch a violation

The contract becomes valuable when it detects drift. Tighten an expectation — for example, add a quality rule to a schema in `datacontract.yaml`:

```yaml
schema:
  - name: orders
    # ...
    quality:
      - type: sql
        description: No order has a negative total
        query: SELECT COUNT(*) FROM orders WHERE order_total < 0
        mustBe: 0
```

Run `datacontract test datacontract.yaml` again: every violation is listed as an error, and the command exits with code `1` — ready for [CI/CD scheduling](../ci-cd.md) so you catch drift before your consumers do.

## Reference

All authentication options and the data type mappings: **[Oracle Reference](../reference/oracle.md)**.

## Troubleshooting

- **`ORA-12514: TNS:listener does not currently know of service`** — `service_name` in the `servers` block doesn't match the database service; check with `lsnrctl status` or your DBA.
- **`DPY-3010: connections to this database server version are not supported`** — older Oracle versions need thick mode: install the Oracle Instant Client and set `DATACONTRACT_ORACLE_CLIENT_DIR`.
- **`ORA-00942: table or view does not exist`** — the `schema` in the `servers` block must be the owning schema (in uppercase), and the user needs `SELECT` on the table.

---

## Postgres


Go from an existing Postgres table to a tested data contract in about five minutes: import the schema straight from Postgres, then test the actual data against it. Works with Postgres and Postgres-compatible databases (e.g. RisingWave).

## 1. Install

```bash
uv tool install --python python3.11 --upgrade 'datacontract-cli[postgres]'
```

See [Installation](../installation.md) for pip, pipx, and Docker.

## 2. Authenticate

Create a `.env` file in your working directory (or export the variables):

```bash
# .env
DATACONTRACT_POSTGRES_USERNAME=postgres
DATACONTRACT_POSTGRES_PASSWORD=mysecretpassword
```

The database user needs `USAGE` on the schema and `SELECT` on the tables. `import` and `test` use the same two variables.

## 3. Create a contract from your tables

Import the table metadata directly from the Postgres catalog. This also generates a ready-to-test `servers` block:

```bash
datacontract import postgres \
  --source localhost \
  --database postgres \
  --schema public \
  --table orders \
  --output datacontract.yaml
```

`--source` is the host of your Postgres server. Add `--port` if it doesn't listen on the default `5432`, repeat `--table` for multiple tables, or omit it to import every table in the schema. `--schema` defaults to `public`.

Only have a DDL file? `datacontract import sql --source orders.sql --dialect postgres` works too, but writes a `servers` block with placeholder values that you have to fill in by hand.

## 4. Test the actual data

```bash
datacontract test datacontract.yaml
```

```
Testing datacontract.yaml
Server: postgres (type=postgres, host=localhost, port=5432, database=postgres, schema=public)
╭────────┬─────────────────────────────────────────────────┬─────────────────┬─────────╮
│ Result │ Check                                           │ Field           │ Details │
├────────┼─────────────────────────────────────────────────┼─────────────────┼─────────┤
│ passed │ Check that field 'order_id' is present          │ orders.order_id │         │
│ passed │ Check that field order_id has no missing values │ orders.order_id │         │
│  ...   │                                                 │                 │         │
╰────────┴─────────────────────────────────────────────────┴─────────────────┴─────────╯
🟢 data contract is valid. Run 24 checks. Took 2.3 seconds.
```

:::tip[No database at hand?]
The [Quickstart](../quickstart.md) tests a public demo contract against a hosted Postgres database — no setup required.
:::

## 5. Let it catch a violation

The contract becomes valuable when it detects drift. Tighten an expectation — for example, add a quality rule to a schema in `datacontract.yaml`:

```yaml
schema:
  - name: orders
    # ...
    quality:
      - type: sql
        description: No order has a negative total
        query: SELECT COUNT(*) FROM orders WHERE order_total < 0
        mustBe: 0
```

Run `datacontract test datacontract.yaml` again: every violation is listed as an error, and the command exits with code `1` — ready for [CI/CD scheduling](../ci-cd.md) so you catch drift before your consumers do.

## Reference

All authentication options and the data type mappings: **[Postgres Reference](../reference/postgres.md)**.

## Troubleshooting

- **`password authentication failed`** — check the two environment variables above; note that values from an already-set shell variable take precedence over `.env`.
- **`relation does not exist`** — the user lacks `USAGE` on the schema or `SELECT` on the table; a missing grant reads as a missing relation.

---

## Amazon Redshift


Go from an existing Redshift table to a tested data contract in about five minutes: import the schema straight from Redshift, then test the actual data against it. Works with provisioned clusters and Redshift Serverless — both are reached over the PostgreSQL wire protocol.

## 1. Install

```bash
uv tool install --python python3.11 --upgrade 'datacontract-cli[redshift]'
```

See [Installation](../installation.md) for pip, pipx, and Docker.

## 2. Authenticate

The easiest way is IAM — sign in to AWS once and the CLI requests temporary database credentials itself, so no database password is stored anywhere and nothing else has to be configured:

```bash
aws sso login   # or any other way of getting an AWS session
```

Your AWS identity needs `redshift-serverless:GetCredentials` (Serverless) or `redshift:GetClusterCredentialsWithIAM` (provisioned cluster), and the resulting database user needs `USAGE` on the schema plus `SELECT` on the tables.

Prefer a database user? Set the credentials directly and they are used instead:

```bash
# .env
DATACONTRACT_REDSHIFT_USERNAME=awsuser
DATACONTRACT_REDSHIFT_PASSWORD=mysecretpassword
```

The CLI picks the method from what you configured: a password means a database login, otherwise it uses your AWS session. Either way, `import` and `test` use the same setup. For custom domains, VPC endpoints, and the full list of options, see the [Redshift Reference](../reference/redshift.md).

## 3. Create a contract from your tables

Import the table metadata directly from the Redshift catalog. This also generates a ready-to-test `servers` block:

```bash
datacontract import redshift \
  --source my-workgroup.123456789012.us-east-1.redshift-serverless.amazonaws.com \
  --database dev \
  --schema analytics \
  --table orders \
  --output datacontract.yaml
```

`--source` is the endpoint host of your cluster or serverless workgroup (without the trailing `:5439/dev` that the console shows). Repeat `--table` for multiple tables, or omit it to import every table in the schema.

## 4. Test the actual data

```bash
datacontract test datacontract.yaml
```

```
Testing datacontract.yaml
Server: redshift (type=redshift, host=my-workgroup..., database=dev, schema=analytics)
╭────────┬─────────────────────────────────────────────────┬─────────────────┬─────────╮
│ Result │ Check                                           │ Field           │ Details │
├────────┼─────────────────────────────────────────────────┼─────────────────┼─────────┤
│ passed │ Check that field 'order_id' is present          │ orders.order_id │         │
│ passed │ Check that field order_id has no missing values │ orders.order_id │         │
│  ...   │                                                 │                 │         │
╰────────┴─────────────────────────────────────────────────┴─────────────────┴─────────╯
🟢 data contract is valid. Run 24 checks. Took 4.9 seconds.
```

## 5. Let it catch a violation

The contract becomes valuable when it detects drift. Tighten an expectation — for example, add a quality rule to a schema in `datacontract.yaml`:

```yaml
schema:
  - name: orders
    # ...
    quality:
      - type: sql
        description: No order has a negative total
        query: SELECT COUNT(*) FROM orders WHERE order_total < 0
        mustBe: 0
```

Run `datacontract test datacontract.yaml` again: every violation is listed as an error, and the command exits with code `1` — ready for [CI/CD scheduling](../ci-cd.md) so you catch drift before your consumers do.

## Reference

All authentication options and the data type mappings: **[Redshift Reference](../reference/redshift.md)**.

## Troubleshooting

- **Connection timeout** — the cluster/workgroup must be reachable from your machine: check **Publicly accessible**, the VPC security group's inbound rule for port `5439`, and VPN routing.
- **`password authentication failed`** — Redshift Serverless expects the namespace's admin credentials or a database user; an IAM user name is not a database user. Unset `DATACONTRACT_REDSHIFT_PASSWORD` to sign in with your AWS identity instead.
- **`Could not obtain temporary Redshift credentials`** — the AWS identity may call the wrong region (set `DATACONTRACT_REDSHIFT_REGION`) or lack `GetCredentials` / `GetClusterCredentialsWithIAM` permission.
- **`Using the login credential provider requires an additional dependency`** — your AWS profile uses the `aws login` flow, which needs `pip install "botocore[crt]"` in the same environment as the CLI.
- **`relation does not exist`** — check the `schema` in the `servers` block and the user's `USAGE` grant on it.
- **The import finds no tables** — `--schema` is case-sensitive and must match the schema as stored in the catalog; external schemas need the `SVV_EXTERNAL_*` permissions granted to your user.

---

## Amazon S3


Go from files in a bucket to a tested data contract in about five minutes. Works with S3 and any S3-compatible endpoint (e.g. MinIO), for data in CSV, JSON, Delta, or Parquet format.

## 1. Install

```bash
uv tool install --python python3.11 --upgrade 'datacontract-cli[s3,duckdb]'
```

See [Installation](../installation.md) for pip, pipx, and Docker.

## 2. Authenticate

The easiest way is to sign in to AWS once — the CLI picks the session up, so no key is stored anywhere:

```bash
aws sso login   # or any other way of getting an AWS session
```

Prefer static keys? Set them directly and they are used instead:

```bash
# .env
DATACONTRACT_S3_REGION=eu-central-1
DATACONTRACT_S3_ACCESS_KEY_ID=AKIAXV5Q5QABCDEFGH
DATACONTRACT_S3_SECRET_ACCESS_KEY=93S7LRrJcqLaaaa/XXXXXXXXXXXXX
```

Public buckets need neither: with nothing configured the objects are read unsigned.

## 3. Create a contract from your files

Import the schema straight from the bucket. This also generates a ready-to-test `servers` block:

```bash
datacontract import s3 \
  --source s3://my-bucket/orders/*.json \
  --output datacontract.yaml
```

The format is taken from the file suffix; pass `--format` for Delta tables, which have none. Add `--endpoint-url` for an S3-compatible store such as MinIO.

## 4. Test the actual data

```bash
datacontract test datacontract.yaml
```

```
Testing datacontract.yaml
Server: production (type=s3, format=json, location=s3://my-bucket/orders/*.json)
╭────────┬─────────────────────────────────────────────────┬─────────────────┬─────────╮
│ Result │ Check                                           │ Field           │ Details │
├────────┼─────────────────────────────────────────────────┼─────────────────┼─────────┤
│ passed │ Check that field 'order_id' is present          │ orders.order_id │         │
│ passed │ Check that field order_id has no missing values │ orders.order_id │         │
│  ...   │                                                 │                 │         │
╰────────┴─────────────────────────────────────────────────┴─────────────────┴─────────╯
🟢 data contract is valid. Run 17 checks. Took 3.6 seconds.
```

## 5. Let it catch a violation

The contract becomes valuable when it detects drift. Tighten an expectation — for example, mark a field as `required: true` that occasionally arrives empty, or add a quality rule:

```yaml
schema:
  - name: orders
    # ...
    quality:
      - type: sql
        description: No order has a negative total
        query: SELECT COUNT(*) FROM orders WHERE order_total < 0
        mustBe: 0
```

Run `datacontract test datacontract.yaml` again: every violation is listed as an error, and the command exits with code `1` — ready for [CI/CD scheduling](../ci-cd.md) so you catch drift before your consumers do.

## Reference

All authentication options and the data type handling per file format: **[S3 Reference](../reference/s3.md)**.

## Troubleshooting

- **`403 Forbidden` / `Access Denied`** — the key pair lacks `s3:GetObject`/`s3:ListBucket` on the location, or `DATACONTRACT_S3_REGION` doesn't match the bucket's region.
- **`No files found that match the pattern`** — check the `location` glob; it matches object keys, not directories.
- **MinIO / S3-compatible storage fails to connect** — set `endpointUrl` in the `servers` block; the CLI then switches to path-style addressing.

---

## Snowflake


Go from an existing Snowflake table to a tested data contract in about five minutes: import the schema straight from Snowflake, then test the actual data against it.

## 1. Install

```bash
uv tool install --python python3.11 --upgrade 'datacontract-cli[snowflake]'
```

See [Installation](../installation.md) for pip, pipx, and Docker.

## 2. Set credentials

Create a `.env` file in your working directory (or export the variables):

```bash
# .env
DATACONTRACT_SNOWFLAKE_USERNAME=<your-username>
DATACONTRACT_SNOWFLAKE_PASSWORD=<your-password>
DATACONTRACT_SNOWFLAKE_WAREHOUSE=COMPUTE_WH
DATACONTRACT_SNOWFLAKE_ROLE=<your-role>
```

If `DATACONTRACT_SNOWFLAKE_PASSWORD` is not set, the import falls back to browser-based SSO. For key-pair authentication and other modes, see the [Snowflake Reference](../reference/snowflake.md).

## 3. Create a contract from your tables

Import the table metadata directly from Snowflake. This also generates a ready-to-test `servers` block:

```bash
datacontract import snowflake \
  --source <orgname>-<accountname> \
  --database ORDER_DB \
  --schema PUBLIC \
  --output datacontract.yaml
```

`--source` is your [account identifier](https://docs.snowflake.com/en/user-guide/admin-account-identifier) in the `<orgname>-<accountname>` format.

## 4. Test the actual data

```bash
datacontract test datacontract.yaml
```

```
Testing datacontract.yaml
Server: workspace (type=snowflake, account=..., database=ORDER_DB, schema=PUBLIC)
╭────────┬─────────────────────────────────────────────────┬─────────────────┬─────────╮
│ Result │ Check                                           │ Field           │ Details │
├────────┼─────────────────────────────────────────────────┼─────────────────┼─────────┤
│ passed │ Check that field 'order_id' is present          │ orders.order_id │         │
│ passed │ Check that field order_id has no missing values │ orders.order_id │         │
│  ...   │                                                 │                 │         │
╰────────┴─────────────────────────────────────────────────┴─────────────────┴─────────╯
🟢 data contract is valid. Run 24 checks. Took 5.2 seconds.
```

## 5. Let it catch a violation

The contract becomes valuable when it detects drift. Tighten an expectation — for example, add a quality rule to a schema in `datacontract.yaml`:

```yaml
schema:
  - name: orders
    # ...
    quality:
      - type: sql
        description: No order has a negative total
        query: SELECT COUNT(*) FROM orders WHERE order_total < 0
        mustBe: 0
```

Run `datacontract test datacontract.yaml` again: every violation is listed as an error, and the command exits with code `1` — ready for [CI/CD scheduling](../ci-cd.md) so you catch drift before your consumers do.

## Reference

All authentication options (key-pair, SSO, connections.toml) and the data type mappings: **[Snowflake Reference](../reference/snowflake.md)**.

## Troubleshooting

- **`250001: Could not connect to Snowflake backend`** — the account identifier is wrong. Use the `<orgname>-<accountname>` format, without `.snowflakecomputing.com`.
- **`Object does not exist or not authorized`** — the role in `DATACONTRACT_SNOWFLAKE_ROLE` lacks `USAGE` on the database/schema or `SELECT` on the table.
- **Test hangs waiting for MFA** — interactive MFA pushes don't work well for repeated runs; use key-pair auth or an appropriate `DATACONTRACT_SNOWFLAKE_AUTHENTICATOR`.

---

## Microsoft SQL Server


Test data in MS SQL Server, including Azure SQL, Synapse Analytics SQL Pool, and Microsoft Fabric.

## 1. Install

```bash
uv tool install --python python3.11 --upgrade 'datacontract-cli[sqlserver]'
```

The connection also requires the [Microsoft ODBC Driver 18 for SQL Server](https://learn.microsoft.com/en-us/sql/connect/odbc/download-odbc-driver-for-sql-server) on your machine. See [Installation](../installation.md) for pip, pipx, and Docker.

## 2. Authenticate

Create a `.env` file in your working directory (or export the variables):

```bash
# .env
DATACONTRACT_SQLSERVER_USERNAME=sa
DATACONTRACT_SQLSERVER_PASSWORD=mysecretpassword
DATACONTRACT_SQLSERVER_TRUST_SERVER_CERTIFICATE=True # for local/dev servers
```

For Entra ID (Active Directory) authentication modes, see the [SQL Server Reference](../reference/sqlserver.md).

## 3. Create a contract from your tables

Import the table metadata directly from the database. This also generates a ready-to-test `servers` block:

```bash
datacontract import sqlserver \
  --source localhost \
  --database mydb \
  --schema dbo \
  --table orders \
  --output datacontract.yaml
```

`--source` is the host of your SQL Server instance. Add `--port` if it doesn't listen on the default `1433`, repeat `--table` for multiple tables, or omit it to import every table. `--schema` defaults to `dbo`.

Only have a DDL script? `datacontract import sql --source orders.sql --dialect sqlserver` works too, but writes a `servers` block with placeholder values that you have to fill in by hand.

## 4. Test the actual data

```bash
datacontract test datacontract.yaml
```

```
Testing datacontract.yaml
Server: production (type=sqlserver, host=localhost, port=1433, database=mydb, schema=dbo)
╭────────┬─────────────────────────────────────────────────┬─────────────────┬─────────╮
│ Result │ Check                                           │ Field           │ Details │
├────────┼─────────────────────────────────────────────────┼─────────────────┼─────────┤
│ passed │ Check that field 'order_id' is present          │ orders.order_id │         │
│ passed │ Check that field order_id has no missing values │ orders.order_id │         │
│  ...   │                                                 │                 │         │
╰────────┴─────────────────────────────────────────────────┴─────────────────┴─────────╯
🟢 data contract is valid. Run 24 checks. Took 3.7 seconds.
```

## 5. Let it catch a violation

The contract becomes valuable when it detects drift. Tighten an expectation — for example, add a quality rule to a schema in `datacontract.yaml`:

```yaml
schema:
  - name: orders
    # ...
    quality:
      - type: sql
        description: No order has a negative total
        query: SELECT COUNT(*) FROM orders WHERE order_total < 0
        mustBe: 0
```

Run `datacontract test datacontract.yaml` again: every violation is listed as an error, and the command exits with code `1` — ready for [CI/CD scheduling](../ci-cd.md) so you catch drift before your consumers do.

## Reference

All authentication options (SQL logins, Entra ID modes, `az login`) and the data type mappings: **[SQL Server Reference](../reference/sqlserver.md)**.

## Troubleshooting

- **`Can't open lib 'ODBC Driver 18 for SQL Server'`** — the ODBC driver isn't installed, or its name doesn't match `driver` in the `servers` block / `DATACONTRACT_SQLSERVER_DRIVER`.
- **`SSL Provider: certificate verify failed`** — for servers with self-signed certificates (local Docker, dev), set `DATACONTRACT_SQLSERVER_TRUST_SERVER_CERTIFICATE=True`.
- **`Login failed for user`** — check the authentication mode: SQL logins need `DATACONTRACT_SQLSERVER_AUTHENTICATION=sql` (the default); Entra ID users need one of the `ActiveDirectory*` modes or `cli`.

---

## Trino


Test data in Trino.

## 1. Install

```bash
uv tool install --python python3.11 --upgrade 'datacontract-cli[trino]'
```

See [Installation](../installation.md) for pip, pipx, and Docker.

## 2. Authenticate

Create a `.env` file in your working directory (or export the variables):

```bash
# .env
DATACONTRACT_TRINO_USERNAME=trino
DATACONTRACT_TRINO_PASSWORD=mysecretpassword
```

The default is `basic` auth; JWT and OAuth2 are also supported — see the [Trino Reference](../reference/trino.md).

## 3. Create a contract from your tables

Import the table metadata directly from the catalog. This also generates a ready-to-test `servers` block:

```bash
datacontract import trino \
  --source localhost \
  --catalog my_catalog \
  --schema my_schema \
  --table orders \
  --output datacontract.yaml
```

Repeat `--table` for multiple tables, or omit it to import every table in the schema.

Only have a DDL script? `datacontract import sql --source orders.sql --dialect postgres` works too — Trino's ANSI-style DDL generally parses with that dialect — but writes a `servers` block with placeholder values that you have to fill in by hand.

## 4. Test the actual data

```bash
datacontract test datacontract.yaml
```

```
Testing datacontract.yaml
Server: trino (type=trino, host=localhost, port=8080, catalog=my_catalog, schema=my_schema)
╭────────┬─────────────────────────────────────────────────┬─────────────────┬─────────╮
│ Result │ Check                                           │ Field           │ Details │
├────────┼─────────────────────────────────────────────────┼─────────────────┼─────────┤
│ passed │ Check that field 'order_id' is present          │ orders.order_id │         │
│ passed │ Check that field order_id has no missing values │ orders.order_id │         │
│  ...   │                                                 │                 │         │
╰────────┴─────────────────────────────────────────────────┴─────────────────┴─────────╯
🟢 data contract is valid. Run 24 checks. Took 1.9 seconds.
```

## 5. Let it catch a violation

The contract becomes valuable when it detects drift. Tighten an expectation — for example, add a quality rule to a schema in `datacontract.yaml`:

```yaml
schema:
  - name: orders
    # ...
    quality:
      - type: sql
        description: No order has a negative total
        query: SELECT COUNT(*) FROM orders WHERE order_total < 0
        mustBe: 0
```

Run `datacontract test datacontract.yaml` again: every violation is listed as an error, and the command exits with code `1` — ready for [CI/CD scheduling](../ci-cd.md) so you catch drift before your consumers do.

## Reference

All authentication options (basic, JWT, OAuth2) and the data type handling: **[Trino Reference](../reference/trino.md)**.

## Troubleshooting

- **`401 Unauthorized`** — check the auth mode: password-protected clusters need `basic` credentials over HTTPS; token-based setups need `DATACONTRACT_TRINO_AUTHENTICATION=jwt` and `DATACONTRACT_TRINO_JWT_TOKEN`.
- **`Catalog ... does not exist` / `Schema ... does not exist`** — `catalog` and `schema` in the `servers` block must match `SHOW CATALOGS` / `SHOW SCHEMAS FROM <catalog>`.

---

## Define your Quality Rules


Beyond schema checks (presence, type, nullability), a data contract can declare **quality rules**. When you run [`datacontract test`](../testing/index.md), executable rules run against the data source and are reported alongside the schema checks.

Quality rules use the ODCS `quality` attribute, which can be attached either to a **schema** (table/object level) or to an individual **property** (column/field level):

```yaml
schema:
  - name: orders
    quality:                       # schema-level rule
      - type: library
        metric: rowCount
        mustBeGreaterThan: 0
    properties:
      - name: order_total
        quality:                   # property-level rule
          - type: library
            metric: nullValues
            mustBe: 0
```

## Quality rule types

ODCS defines four `type`s of quality rule. Each has its own page:

  
    
    SQLRun a custom SQL query and compare the result to a threshold.
  
  
    
    LibraryPredefined, portable checks such as rowCount, nullValues, and duplicateValues.
  
  
    
    TextA human-readable expectation, for documentation only.
  
  
    
    CustomEngine-specific checks (e.g. DQX, SodaCL, Great Expectations).
  

## Running only quality checks

Use `--checks` to restrict a run to quality rules:

```bash
datacontract test --checks quality datacontract.yaml
```

## Where quality rules are used

Executable rules (`sql`, `library`) run during `test`. All rule types are also translated when you export to a quality engine:

- **[dbt](../dbt.md)** — rules become native dbt tests, with `dbt_utils` tests and singular SQL tests for rules that cannot be expressed natively.
- **[SodaCL](../exports/sodacl.md)** — rules become Soda checks.
- **[Great Expectations](../exports/great-expectations.md)** — rules become expectations in a suite.
- **[DQX](../exports/dqx.md)** — rules become Databricks DQX checks.

## Inspecting failing rows

When a quality (or schema) check fails, add `--include-failed-samples` to `test` to collect a small sample of the offending rows (identifier + offending columns; sensitive columns are omitted). This is off by default.

```bash
datacontract test --include-failed-samples datacontract.yaml
```

## Learn more

- The full quality syntax is part of the [Open Data Contract Standard](https://bitol-io.github.io/open-data-contract-standard/latest/).

---

## Custom Quality Rules


A `type: custom` rule carries a check written in the **native syntax of a specific quality engine**. You name the engine with `engine` and provide the engine-specific definition under `implementation`. Custom rules let you use capabilities of an engine that the portable [library](./library.md) and [SQL](./sql.md) rules don't cover.

Custom rules are consumed by the matching engine — typically via the corresponding [exporter](../exports/index.md) — rather than being compiled to portable SQL.

## Databricks DQX

```yaml
schema:
  - name: events
    properties:
      - name: event_type
        logicalType: string
        quality:
          - type: custom
            engine: dqx
            implementation:
              criticality: error
              check:
                function: is_in_list
                arguments:
                  allowed:
                    - click
                    - view
                    - purchase
```

Export these to a DQX check file with [`datacontract export dqx`](../exports/dqx.md).

## SodaCL

```yaml
schema:
  - name: orders
    quality:
      - type: custom
        engine: soda
        implementation: |
          checks for orders:
            - row_count > 10
```

Export to a SodaCL file with [`datacontract export sodacl`](../exports/sodacl.md).

:::warning
Raw SodaCL custom checks (`type: custom`, `engine: soda`) are **no longer executed** by `datacontract test` since `soda-core` was removed; such a rule is reported as a warning. Migrate it to a [SQL rule](./sql.md) (`type: sql`) to keep it executable, or keep it as a custom rule purely for the SodaCL export.
:::

## Other engines

The same pattern applies to other engines (for example Great Expectations). Provide `engine` and an `implementation` understood by that engine, and use the matching [exporter](../exports/index.md) to generate its native artifact.

---

## Library Quality Rules


A `type: library` rule is a **predefined, portable check** selected with a `metric` name. Unlike [SQL rules](./sql.md), you don't write a query — the CLI compiles the metric into dialect-specific SQL for the selected server, so the same rule works across all backends.

```yaml
quality:
  - type: library
    metric: nullValues
    mustBe: 0
```

Each rule combines a **metric**, optional **arguments**, and exactly one **comparator** (`mustBe`, `mustBeBetween`, …).

## Supported metrics

| Metric | Level | Measures | Arguments |
|---|---|---|---|
| `rowCount` | schema | Number of rows in the table | — |
| `duplicateValues` | schema or property | Number of duplicate values | schema-level: `properties` (list of columns to check together) |
| `nullValues` | property | Number of `NULL` values | — |
| `missingValues` | property | Number of missing values (`NULL` plus the values you treat as missing) | `missingValues` (list of values counted as missing, e.g. `['', 'N/A']`) |
| `invalidValues` | property | Number of values **not** in an allow-list | `validValues` (list of permitted values) |

:::note
`nullValues`, `missingValues`, and `invalidValues` are only supported at the **property** level. `rowCount` is **schema** level. `duplicateValues` works at either level. Any other metric is reported as "not yet supported".
:::

## Examples

**Row count** (schema level):

```yaml
schema:
  - name: orders
    quality:
      - type: library
        metric: rowCount
        mustBeGreaterThan: 0
```

**No nulls** in a property:

```yaml
quality:
  - type: library
    metric: nullValues
    mustBe: 0
```

**Missing values** — treat empty string and `N/A` as missing, in addition to `NULL`:

```yaml
quality:
  - type: library
    metric: missingValues
    arguments:
      missingValues: [null, '', 'N/A']
    mustBe: 0
```

**Invalid values** — the value must be one of an allow-list:

```yaml
quality:
  - type: library
    metric: invalidValues
    arguments:
      validValues: ['CX-263-DU', 'IK-894-MN', 'ER-399-JY']
    mustBe: 0
```

**Duplicate values** across a set of columns (schema level):

```yaml
schema:
  - name: orders
    quality:
      - type: library
        metric: duplicateValues
        arguments:
          properties: [order_id, line_number]
        mustBe: 0
```

## Comparators

Use exactly one comparator to define the threshold:

| Comparator | Passes when the metric… |
|---|---|
| `mustBe` | equals the value |
| `mustNotBe` | does not equal the value |
| `mustBeGreaterThan` / `mustBeGreaterOrEqualTo` | is above a lower bound |
| `mustBeLessThan` / `mustBeLessOrEqualTo` | is below an upper bound |
| `mustBeBetween` / `mustNotBeBetween` | is inside / outside a `[min, max]` range |

## Percentage thresholds

For the count-of-bad-rows metrics (`nullValues`, `missingValues`, `invalidValues`), set `unit: percent` to compare against a percentage of rows instead of an absolute count:

```yaml
quality:
  - type: library
    metric: nullValues
    unit: percent
    mustBeLessThan: 1   # fewer than 1% of rows are null
```

`unit: percent` is ignored (with a warning) for metrics that are not count-of-bad-rows.

---

## SQL Quality Rules


A `type: sql` rule runs a custom SQL query against the server and compares the single returned value to a threshold. It is the most flexible rule type — use it whenever a check can be expressed as a query. The query runs in the dialect of the selected server.

## Property-level example

```yaml
schema:
  - name: orders
    properties:
      - name: order_total
        logicalType: integer
        physicalType: bigint
        required: true
        quality:
          - type: sql
            description: 95% of all order total values are expected to be between 10 and 499 EUR.
            query: |
              SELECT quantile_cont(order_total, 0.95) AS percentile_95
              FROM orders
            mustBeBetween:
              - 1000
              - 99900
```

## Schema-level example

```yaml
schema:
  - name: orders
    quality:
      - type: sql
        description: The maximum duration between two orders should be less than 3600 seconds.
        query: |
          SELECT MAX(duration) AS max_duration
          FROM (
            SELECT EXTRACT(EPOCH FROM (order_timestamp - LAG(order_timestamp)
                   OVER (ORDER BY order_timestamp))) AS duration
            FROM orders
          ) subquery
        mustBeLessThan: 3600
```

## Comparators

The query must return a single value, which is compared using exactly one of:

| Comparator | Passes when the result… |
|---|---|
| `mustBe` | equals the value |
| `mustNotBe` | does not equal the value |
| `mustBeGreaterThan` / `mustBeGreaterOrEqualTo` | is above a lower bound |
| `mustBeLessThan` / `mustBeLessOrEqualTo` | is below an upper bound |
| `mustBeBetween` / `mustNotBeBetween` | is inside / outside a `[min, max]` range |

## Notes

- **`dialect`** — optionally pin the SQL dialect the query is written in (e.g. `dialect: postgres`). By default the query runs in the selected server's dialect.
- **Referencing the data** — reference the schema/table by its name in the `FROM` clause (e.g. `FROM orders`).
- **`severity`** — set `severity: warning` to report a failing rule without failing the run (see [`--fail-on`](../commands/ci.md)).

---

## Text Quality Rules


A `type: text` rule is a **human-readable expectation** written in plain language. It is **not executed** by `datacontract test` — it documents an agreement or an expectation that cannot (yet) be automated, so it stays visible in the contract and in generated documentation.

```yaml
schema:
  - name: orders
    properties:
      - name: order_status
        logicalType: string
        quality:
          - type: text
            description: >
              order_status transitions follow the lifecycle
              placed → shipped → delivered, and never moves backwards.
```

## When to use it

- The expectation is understood and agreed, but you don't (yet) have a query or metric for it.
- The rule is enforced by an external process and you want to record it in the contract.
- You want to communicate intent to consumers alongside the executable rules.

When you are ready to enforce it automatically, promote it to a [SQL](./sql.md) or [library](./library.md) rule.

---

## Imports


`datacontract import` creates a data contract from an existing source format. This is the fastest way to bootstrap a contract from a system you already have.

```bash
# Import from a SQL DDL file
datacontract import sql --source my_ddl.sql --dialect postgres

# Save the result to a file
datacontract import sql --source my_ddl.sql --dialect postgres --output datacontract.yaml
```

The [Snowflake](./snowflake.md), [BigQuery](./bigquery.md), [Amazon Redshift](./redshift.md), [Postgres](./postgres.md), [MySQL](./mysql.md), [SQL Server](./sqlserver.md), [Oracle](./oracle.md), [Trino](./trino.md), [Amazon Athena](./athena.md), [Amazon S3](./s3.md), [Google Cloud Storage](./gcs.md), [Azure Blob / ADLS](./adls.md), [Databricks](./databricks.md), and [AWS Glue](./glue.md) importers can connect directly to the live system and introspect your tables — no export files needed. Snowflake, BigQuery, Redshift, Postgres, MySQL, SQL Server, Oracle, Trino, Athena, S3, GCS, ADLS, and Databricks also generate a ready-to-test `servers` block, so `datacontract test` works right after the import.

Run `datacontract import <format> --help` to see the format-specific options (e.g. `datacontract import sql --help`). If a format you need is missing, [open an issue on GitHub](https://github.com/datacontract/datacontract-cli/issues).

## Example sources

Each import page shows a runnable example: a small source file under [`examples/imports/`](https://github.com/datacontract/datacontract-cli/tree/main/examples/imports) and the data contract the CLI generates from it. Download a source and run the command on the page to reproduce the output.

## Available importers

  
    
    adlsFiles in Azure Blob Storage.
  
  
    
    athenaAn Amazon Athena database.
  
  
    
    avroAn Avro schema file.
  
  
    
    bigqueryGoogle BigQuery (file or API).
  
  
    
    csvA CSV file.
  
  
    
    databricksDatabricks Unity Catalog.
  
  
    
    dbmlA DBML file.
  
  
    
    dbtA dbt manifest file.
  
  
    
    excelAn ODCS Excel template.
  
  
    
    gcsFiles in Google Cloud Storage.
  
  
    
    glueAWS Glue Data Catalog.
  
  
    
    icebergAn Iceberg schema.
  
  
    
    jsonA JSON data file.
  
  
    
    jsonschemaA JSON Schema file.
  
  
    
    mysqlA MySQL database.
  
  
    
    odcsAn ODCS data contract file.
  
  
    
    oracleAn Oracle database.
  
  
    
    parquetA Parquet file.
  
  
    
    postgresA Postgres schema.
  
  
    
    powerbiA Power BI semantic model (.pbit, .bim, or .json).
  
  
    
    protobufA Protobuf schema file.
  
  
    
    redshiftAn Amazon Redshift schema.
  
  
    
    s3Files in an Amazon S3 bucket.
  
  
    
    snowflakeA Snowflake workspace.
  
  
    
    sparkA Spark schema / DataFrame.
  
  
    
    sqlA SQL DDL file.
  
  
    
    sqlserverA SQL Server database.
  
  
    
    trinoA Trino catalog.
  

See the [`import` command reference](../commands/import.md) for the common signature.

---

## Import: Azure Blob / ADLS


Creates a data contract from files in Azure Blob Storage, in CSV, JSON, Parquet, or Delta format.

```bash
datacontract import adls \
  --source abfss://my-container/orders/*.json \
  --output datacontract.yaml
```

The format is taken from the file suffix; pass `--format` for Delta tables, which have none, or to override the guess. `--delimiter` pins how a JSON file is laid out (`new_line`, `array`, or `none`) instead of letting it be detected.

The objects are read with duckdb through the same connection the test path uses, so the import authenticates identically and infers the column types from exactly the reader that later verifies them.

The generated contract includes a ready-to-test `servers` block, so you can run `datacontract test datacontract.yaml` immediately afterwards — see the **[Azure Blob / ADLS connection guide](../testing/azure.md)** for the full 5-minute walkthrough and troubleshooting.

Credentials are the same ones `datacontract test` uses — see the [Azure Blob Storage Reference](../reference/azure.md).

---

## Import: Amazon Athena


Creates a data contract from an Amazon Athena database, including column types, partition keys, and comments.

```bash
datacontract import athena \
  --schema my_database \
  --staging-dir s3://my-bucket/athena-results/ \
  --region eu-central-1 \
  --output datacontract.yaml
```

`--schema` is the Athena database. Repeat `--table` to import only specific tables (by default every table in the database is imported), and set `--catalog` if you don't use `awsdatacatalog`.

`--staging-dir` is the S3 location Athena writes query results to. `datacontract test` needs it, so the import writes it into the `servers` block.

Athena keeps its table metadata in the AWS Glue Data Catalog, so the import reads the catalog rather than querying Athena — it needs only `glue:GetTables`, costs nothing to run, and requires no query results to be staged. Glue stores the Hive spelling of two types, which the import rewrites to what Athena reports back (`string` → `varchar`, `binary` → `varbinary`) so the physical type checks pass on the first test run.

The generated contract includes a ready-to-test `servers` block, so you can run `datacontract test datacontract.yaml` immediately afterwards — see the **[Athena connection guide](../testing/athena.md)** for the full 5-minute walkthrough and troubleshooting.

Credentials are provided as environment variables and are the same ones `datacontract test` uses: `DATACONTRACT_S3_REGION`, `DATACONTRACT_S3_ACCESS_KEY_ID`, and `DATACONTRACT_S3_SECRET_ACCESS_KEY` — see the [Athena Reference](../reference/athena.md).

Want the Glue Data Catalog itself rather than Athena? Use [`datacontract import glue`](./glue.md).

---

## Import: Avro


Creates a data contract from an Avro schema (`.avsc`) file.

{/* AUTOGENERATED EXAMPLE: do not edit by hand; regenerate with the update script. */}

Given this [`orders.avsc`](https://github.com/datacontract/datacontract-cli/blob/main/examples/imports/avro/orders.avsc):

```json
{
  "type": "record",
  "name": "orders",
  "namespace": "com.example.checkout",
  "fields": [
    { "name": "order_id", "type": "string", "doc": "Unique identifier of the order." },
    { "name": "order_timestamp", "type": { "type": "long", "logicalType": "timestamp-millis" } },
    { "name": "customer_id", "type": "string" },
    { "name": "order_total", "type": "long" },
    { "name": "status", "type": "string" }
  ]
}
```

Run:

```bash
datacontract import avro --source orders.avsc
```

to produce the data contract:

```yaml
version: 1.0.0
kind: DataContract
apiVersion: v3.1.0
id: my-data-contract
name: My Data Contract
status: draft
schema:
- name: orders
  physicalType: record
  customProperties:
  - property: namespace
    value: com.example.checkout
  logicalType: object
  physicalName: orders
  properties:
  - name: order_id
    physicalType: string
    description: Unique identifier of the order.
    logicalType: string
    required: true
  - name: order_timestamp
    physicalType: long
    logicalType: date
    required: true
  - name: customer_id
    physicalType: string
    logicalType: string
    required: true
  - name: order_total
    physicalType: long
    logicalType: integer
    required: true
  - name: status
    physicalType: string
# …
```

{/* END AUTOGENERATED EXAMPLE */}

---

## Import: BigQuery


Creates a data contract from BigQuery, either directly from the BigQuery API or from an exported table definition (JSON file).

## From the BigQuery API

```bash
datacontract import bigquery \
  --project my-project \
  --dataset my_dataset \
  --table orders \
  --output datacontract.yaml
```

Repeat `--table` for multiple tables, or omit it to import every table in the dataset. Authentication uses Application Default Credentials (`gcloud auth application-default login`, or `GOOGLE_APPLICATION_CREDENTIALS`).

The generated contract includes a ready-to-test `servers` block, so you can run `datacontract test datacontract.yaml` immediately afterwards — see the **[BigQuery connection guide](../testing/bigquery.md)** for the full 5-minute walkthrough and troubleshooting.

## From an exported JSON file

{/* AUTOGENERATED EXAMPLE: do not edit by hand; regenerate with the update script. */}

Given this [`orders.json`](https://github.com/datacontract/datacontract-cli/blob/main/examples/imports/bigquery/orders.json):

```json
{
  "kind": "bigquery#table",
  "tableReference": {
    "projectId": "my-gcp-project",
    "datasetId": "orders",
    "tableId": "orders"
  },
  "description": "One row per customer order.",
  "schema": {
    "fields": [
      { "name": "order_id", "type": "STRING", "mode": "REQUIRED", "description": "Unique identifier of the order." },
      { "name": "order_timestamp", "type": "TIMESTAMP", "mode": "REQUIRED" },
      { "name": "customer_id", "type": "STRING", "mode": "REQUIRED" },
      { "name": "order_total", "type": "NUMERIC", "mode": "REQUIRED" },
      { "name": "status", "type": "STRING", "mode": "REQUIRED" }
    ]
  }
}
```

Run:

```bash
datacontract import bigquery --source orders.json
```

to produce the data contract:

```yaml
version: 1.0.0
kind: DataContract
apiVersion: v3.1.0
id: my-data-contract
name: My Data Contract
status: draft
servers:
- server: bigquery
  type: bigquery
  dataset: orders
  project: my-gcp-project
schema:
- name: orders
  physicalType: table
  description: One row per customer order.
  logicalType: object
  physicalName: orders
  properties:
  - name: order_id
    physicalType: STRING
    description: Unique identifier of the order.
    logicalType: string
    required: true
  - name: order_timestamp
    physicalType: TIMESTAMP
    logicalType: timestamp
    required: true
  - name: customer_id
    physicalType: STRING
    logicalType: string
    required: true
  - name: order_total
    physicalType: NUMERIC
    logicalType: number
# …
```

{/* END AUTOGENERATED EXAMPLE */}

---

## Import: CSV


Creates a data contract by inferring the schema (column names and types) from a CSV file.

{/* AUTOGENERATED EXAMPLE: do not edit by hand; regenerate with the update script. */}

Given this [`orders.csv`](https://github.com/datacontract/datacontract-cli/blob/main/examples/imports/csv/orders.csv):

```text
order_id,order_timestamp,customer_id,order_total,status
ORD-1001,2024-01-01T10:00:00Z,CUST-1,4999,delivered
ORD-1002,2024-01-02T11:30:00Z,CUST-2,2500,shipped
ORD-1003,2024-01-03T09:15:00Z,CUST-3,1299,pending
```

Run:

```bash
datacontract import csv --source orders.csv
```

to produce the data contract:

```yaml
version: 1.0.0
kind: DataContract
apiVersion: v3.1.0
id: my-data-contract
name: My Data Contract
status: draft
servers:
- server: production
  type: local
  customProperties:
  - property: delimiter
    value: ','
  format: csv
  path: examples/imports/csv/orders.csv
schema:
- name: orders
  physicalType: table
  description: Generated model of examples/imports/csv/orders.csv
  logicalType: object
  physicalName: orders
  properties:
  - name: order_id
    physicalType: VARCHAR
    logicalType: string
    required: true
    unique: true
    examples:
    - ORD-1003
    - ORD-1001
    - ORD-1002
  - name: order_timestamp
    physicalType: VARCHAR
    logicalType: date
    required: true
# …
```

{/* END AUTOGENERATED EXAMPLE */}

---

## Import: Databricks


Creates a data contract from Databricks Unity Catalog, from an exported JSON file or via the HTTP endpoint.

```bash
# From the HTTP endpoint (repeat --table for multiple tables)
datacontract import databricks --table my_catalog.my_schema.orders

# From an exported Unity Catalog TableInfo JSON file
datacontract import databricks --source unity_table.json
```

For the HTTP endpoint, authenticate either with a personal access token (`DATACONTRACT_DATABRICKS_SERVER_HOSTNAME` + `DATACONTRACT_DATABRICKS_TOKEN`) or with a profile from `~/.databrickscfg` (`DATACONTRACT_DATABRICKS_PROFILE`).

The generated contract includes a ready-to-test `servers` block (`type: databricks` with catalog and schema). To run `datacontract test` afterwards, additionally set `DATACONTRACT_DATABRICKS_HTTP_PATH` to a running SQL warehouse — see the **[Databricks connection guide](../testing/databricks.md)** for the full 5-minute walkthrough and troubleshooting.

The importer was previously called `unity`. That name still works, so existing scripts keep running.

---

## Import: DBML


Creates a data contract from a [DBML](https://dbml.dbdiagram.io/) file. You can filter by schema and table name.

{/* AUTOGENERATED EXAMPLE: do not edit by hand; regenerate with the update script. */}

Given this [`orders.dbml`](https://github.com/datacontract/datacontract-cli/blob/main/examples/imports/dbml/orders.dbml):

```text
Table orders {
  order_id varchar [pk, note: 'Unique identifier of the order.']
  order_timestamp timestamp [not null]
  customer_id varchar [not null]
  order_total bigint [not null]
  status varchar [not null]
}
```

Run:

```bash
datacontract import dbml --source orders.dbml
```

to produce the data contract:

```yaml
version: 1.0.0
kind: DataContract
apiVersion: v3.1.0
id: my-data-contract
name: My Data Contract
status: draft
schema:
- name: orders
  physicalType: table
  customProperties:
  - property: namespace
    value: public
  logicalType: object
  physicalName: orders
  properties:
  - name: order_id
    physicalType: varchar
    description: Unique identifier of the order.
    primaryKey: true
    primaryKeyPosition: 1
    logicalType: string
  - name: order_timestamp
    physicalType: timestamp
    logicalType: timestamp
    required: true
  - name: customer_id
    physicalType: varchar
    logicalType: string
    required: true
  - name: order_total
    physicalType: bigint
    logicalType: integer
    required: true
  - name: status
# …
```

{/* END AUTOGENERATED EXAMPLE */}

---

## Import: dbt


Creates a data contract from a dbt `manifest.json`.

```bash
# Import specific tables
datacontract import dbt --source manifest.json --model orders --model line_items

# Import all tables in the database
datacontract import dbt --source manifest.json
```

See the [dbt Integration](../dbt.md) guide for the full dbt workflow.

---

## Import: Excel


Creates a data contract from an ODCS Excel template — the round-trip counterpart of [`export excel`](../exports/excel.md).

```bash
datacontract import excel --source odcs.xlsx --output datacontract.yaml
```

---

## Import: Google Cloud Storage


Creates a data contract from files in Google Cloud Storage, in CSV, JSON, Parquet, or Delta format.

```bash
datacontract import gcs \
  --source s3://my-bucket/orders/*.json \
  --output datacontract.yaml
```

The format is taken from the file suffix; pass `--format` for Delta tables, which have none, or to override the guess. `--delimiter` pins how a JSON file is laid out (`new_line`, `array`, or `none`) instead of letting it be detected.

duckdb reads Google Cloud Storage through its S3-compatible endpoint, so the location uses the `s3://` scheme rather than `gs://`.

The objects are read with duckdb through the same connection the test path uses, so the import authenticates identically and infers the column types from exactly the reader that later verifies them.

The generated contract includes a ready-to-test `servers` block, so you can run `datacontract test datacontract.yaml` immediately afterwards — see the **[Google Cloud Storage connection guide](../testing/gcs.md)** for the full 5-minute walkthrough and troubleshooting.

Credentials are the same ones `datacontract test` uses — see the [Google Cloud Storage Reference](../reference/gcs.md).

---

## Import: AWS Glue


Creates a data contract from the AWS Glue Data Catalog.

```bash
# Import specific tables from a Glue database
datacontract import glue --database my_database --table orders --table line_items

# Import all tables in the database
datacontract import glue --database my_database
```

AWS credentials are resolved from the standard AWS environment / configuration (e.g. `AWS_PROFILE`, `AWS_REGION`).

:::note
`datacontract test` cannot connect to Glue directly. To test the actual data, replace the generated `servers` entry with the underlying storage — typically [Amazon S3](../testing/s3.md) or [Amazon Athena](../testing/athena.md).
:::

---

## Import: Iceberg


Creates a data contract from an [Apache Iceberg](https://iceberg.apache.org/) table schema.

{/* AUTOGENERATED EXAMPLE: do not edit by hand; regenerate with the update script. */}

Given this [`orders.json`](https://github.com/datacontract/datacontract-cli/blob/main/examples/imports/iceberg/orders.json):

```json
{
  "type": "struct",
  "fields": [
    { "id": 1, "name": "order_id", "type": "string", "required": true },
    { "id": 2, "name": "order_timestamp", "type": "timestamptz", "required": true },
    { "id": 3, "name": "customer_id", "type": "string", "required": true },
    { "id": 4, "name": "order_total", "type": "long", "required": true },
    { "id": 5, "name": "status", "type": "string", "required": true }
  ],
  "schema-id": 0,
  "identifier-field-ids": [1]
}
```

Run:

```bash
datacontract import iceberg --source orders.json
```

to produce the data contract:

```yaml
version: 1.0.0
kind: DataContract
apiVersion: v3.1.0
id: my-data-contract
name: My Data Contract
status: draft
schema:
- name: iceberg_table
  physicalType: table
  logicalType: object
  physicalName: iceberg_table
  properties:
  - name: order_id
    physicalType: string
    customProperties:
    - property: icebergFieldId
      value: 1
    primaryKey: true
    primaryKeyPosition: 1
    logicalType: string
    required: true
  - name: order_timestamp
    physicalType: timestamptz
    customProperties:
    - property: icebergFieldId
      value: 2
    logicalType: date
    required: true
  - name: customer_id
    physicalType: string
    customProperties:
    - property: icebergFieldId
      value: 3
    logicalType: string
# …
```

{/* END AUTOGENERATED EXAMPLE */}

---

## Import: JSON


Creates a data contract by inferring the schema from a JSON data file.

{/* AUTOGENERATED EXAMPLE: do not edit by hand; regenerate with the update script. */}

Given this [`orders.json`](https://github.com/datacontract/datacontract-cli/blob/main/examples/imports/json/orders.json):

```json
{
  "order_id": "ORD-1001",
  "order_timestamp": "2024-01-01T10:00:00Z",
  "customer_id": "CUST-1",
  "order_total": 4999,
  "status": "delivered"
}
```

Run:

```bash
datacontract import json --source orders.json
```

to produce the data contract:

```yaml
version: 1.0.0
kind: DataContract
apiVersion: v3.1.0
id: my-data-contract
name: My Data Contract
status: draft
servers:
- server: production
  type: local
  format: json
  path: examples/imports/json/orders.json
schema:
- name: orders
  physicalType: object
  description: Generated from JSON object in examples/imports/json/orders.json
  logicalType: object
  physicalName: orders
  properties:
  - name: order_id
    physicalType: string
    logicalType: string
    examples:
    - ORD-1001
  - name: order_timestamp
    physicalType: string
    logicalType: string
    logicalTypeOptions:
      format: date-time
    examples:
    - '2024-01-01T10:00:00Z'
  - name: customer_id
    physicalType: string
    logicalType: string
    examples:
# …
```

{/* END AUTOGENERATED EXAMPLE */}

---

## Import: JSON Schema


Creates a data contract from a [JSON Schema](https://json-schema.org/) file.

{/* AUTOGENERATED EXAMPLE: do not edit by hand; regenerate with the update script. */}

Given this [`orders.schema.json`](https://github.com/datacontract/datacontract-cli/blob/main/examples/imports/jsonschema/orders.schema.json):

```json
{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "orders",
  "type": "object",
  "properties": {
    "order_id": { "type": "string", "description": "Unique identifier of the order." },
    "order_timestamp": { "type": "string", "format": "date-time" },
    "customer_id": { "type": "string" },
    "order_total": { "type": "integer" },
    "status": { "type": "string" }
  },
  "required": ["order_id", "order_timestamp", "customer_id", "order_total", "status"]
}
```

Run:

```bash
datacontract import jsonschema --source orders.schema.json
```

to produce the data contract:

```yaml
version: 1.0.0
kind: DataContract
apiVersion: v3.1.0
id: my-data-contract
name: orders
status: draft
schema:
- name: orders
  physicalType: object
  businessName: orders
  logicalType: object
  physicalName: orders
  properties:
  - name: order_id
    physicalType: string
    description: Unique identifier of the order.
    logicalType: string
    required: true
  - name: order_timestamp
    physicalType: string
    logicalType: string
    logicalTypeOptions:
      format: date-time
    required: true
  - name: customer_id
    physicalType: string
    logicalType: string
    required: true
  - name: order_total
    physicalType: integer
    logicalType: integer
    required: true
  - name: status
    physicalType: string
# …
```

{/* END AUTOGENERATED EXAMPLE */}

---

## Import: MySQL


Creates a data contract from a MySQL database by reading `information_schema` — including the full declared column types, nullability, primary keys, and the comments on tables and columns.

```bash
datacontract import mysql \
  --source localhost \
  --database mydb \
  --output datacontract.yaml
```

`--source` is the host of your MySQL server. Add `--port` if it doesn't listen on the default `3306`, and repeat `--table` to import only specific tables (by default every table and view in the database is imported).

The database is reached the way `datacontract test` reaches it: duckdb attaches it through its `mysql` extension, so no extra driver is needed and the import authenticates with the same variables.

The generated contract includes a ready-to-test `servers` block, so you can run `datacontract test datacontract.yaml` immediately afterwards — see the **[MySQL connection guide](../testing/mysql.md)** for the full 5-minute walkthrough and troubleshooting.

Credentials are provided as environment variables and are the same ones `datacontract test` uses: `DATACONTRACT_MYSQL_USERNAME` and `DATACONTRACT_MYSQL_PASSWORD` — see the [MySQL Reference](../reference/mysql.md).

Only have a DDL file? Use [`datacontract import sql --dialect mysql`](./sql.md).

---

## Import: ODCS


Imports an existing [ODCS](../open-data-contract-standard.md) data contract file (for normalization or conversion).

```bash
datacontract import odcs --source other.odcs.yaml --output datacontract.yaml
```

---

## Import: Oracle


Creates a data contract from an Oracle database by reading `ALL_TAB_COLUMNS` — including column types with length and precision, nullability, and primary keys.

```bash
datacontract import oracle \
  --source localhost \
  --service-name XEPDB1 \
  --schema ADMIN \
  --output datacontract.yaml
```

`--source` is the host of your Oracle database and `--service-name` its service. Add `--port` if it doesn't listen on the default `1521`, and repeat `--table` to import only specific tables. Oracle stores identifiers in upper case, so `--schema` is upper-cased for you.

Types are reconstructed exactly as the test path reads them back. Oracle reports a `DATA_LENGTH` for *every* column, but it is only part of the declared type for character and raw types — so a `DATE` stays `DATE` rather than becoming `DATE(7)`.

The generated contract includes a ready-to-test `servers` block, so you can run `datacontract test datacontract.yaml` immediately afterwards — see the **[Oracle connection guide](../testing/oracle.md)** for the full 5-minute walkthrough and troubleshooting.

Credentials are the same ones `datacontract test` uses: `DATACONTRACT_ORACLE_USERNAME` and `DATACONTRACT_ORACLE_PASSWORD` — see the [Oracle Reference](../reference/oracle.md).

Only have a DDL script? Use [`datacontract import sql --dialect oracle`](./sql.md).

---

## Import: Parquet


Creates a data contract from the schema of a Parquet file.

```bash
datacontract import parquet --source data.parquet --output datacontract.yaml
```

---

## Import: Postgres


Creates a data contract from a Postgres schema by reading table metadata from `information_schema` — including column types with length and precision, nullability, primary keys, and the comments stored in `pg_description`. Works with Postgres and Postgres-compatible databases (e.g. RisingWave).

```bash
datacontract import postgres \
  --source localhost \
  --database postgres \
  --schema public \
  --output datacontract.yaml
```

`--source` is the host of your Postgres server. Add `--port` if it doesn't listen on the default `5432`, and repeat `--table` to import only specific tables (by default every table and view in the schema is imported). `--schema` defaults to `public`.

The generated contract includes a ready-to-test `servers` block, so you can run `datacontract test datacontract.yaml` immediately afterwards — see the **[Postgres connection guide](../testing/postgres.md)** for the full 5-minute walkthrough and troubleshooting.

Credentials are provided as environment variables and are the same ones `datacontract test` uses: `DATACONTRACT_POSTGRES_USERNAME` and `DATACONTRACT_POSTGRES_PASSWORD` — see the [Postgres Reference](../reference/postgres.md).

Working from a DDL file instead of a live database? Use [`datacontract import sql --dialect postgres`](./sql.md).

---

## Import: Power BI


Creates a data contract from a Power BI semantic model (`.pbit`, `.bim`, or `.json`).

```bash
datacontract import powerbi --source model.pbit --output datacontract.yaml
```

---

## Import: Protobuf


Creates a data contract from a [Protobuf](https://protobuf.dev/) `.proto` schema file.

{/* AUTOGENERATED EXAMPLE: do not edit by hand; regenerate with the update script. */}

Given this [`orders.proto`](https://github.com/datacontract/datacontract-cli/blob/main/examples/imports/protobuf/orders.proto):

```protobuf
syntax = "proto3";

package checkout;

// One row per customer order.
message Orders {
  string order_id = 1;
  string order_timestamp = 2;
  string customer_id = 3;
  int64 order_total = 4;
  string status = 5;
}
```

Run:

```bash
datacontract import protobuf --source orders.proto
```

to produce the data contract:

```yaml
version: 1.0.0
kind: DataContract
apiVersion: v3.1.0
id: my-data-contract
name: My Data Contract
status: draft
schema:
- name: Orders
  physicalType: message
  description: Details of Orders.
  logicalType: object
  physicalName: Orders
  properties:
  - name: order_id
    physicalType: '9'
    description: Field order_id
    logicalType: string
    required: false
  - name: order_timestamp
    physicalType: '9'
    description: Field order_timestamp
    logicalType: string
    required: false
  - name: customer_id
    physicalType: '9'
    description: Field customer_id
    logicalType: string
    required: false
  - name: order_total
    physicalType: '3'
    description: Field order_total
    logicalType: integer
    required: false
  - name: status
# …
```

{/* END AUTOGENERATED EXAMPLE */}

---

## Import: Amazon Redshift


Creates a data contract from an Amazon Redshift schema by reading table metadata from the `SVV_TABLES` and `SVV_COLUMNS` catalog views — including column types with length and precision, nullability, primary keys, and comments. Works with provisioned clusters and Redshift Serverless, and covers local tables, views, and external (Spectrum) tables.

```bash
datacontract import redshift \
  --source my-workgroup.123456789012.us-east-1.redshift-serverless.amazonaws.com \
  --database dev \
  --schema analytics \
  --output datacontract.yaml
```

`--source` is the endpoint host of your cluster or serverless workgroup. Add `--port` if it doesn't listen on the default `5439`, and repeat `--table` to import only specific tables (by default every table in the schema is imported).

The generated contract includes a ready-to-test `servers` block, so you can run `datacontract test datacontract.yaml` immediately afterwards — see the **[Redshift connection guide](../testing/redshift.md)** for credentials, the full 5-minute walkthrough, and troubleshooting.

Credentials are provided as environment variables and are the same ones `datacontract test` uses: either IAM (`DATACONTRACT_REDSHIFT_AUTHENTICATION=iam`, which requests temporary credentials from your AWS session) or a database user (`DATACONTRACT_REDSHIFT_USERNAME`, `DATACONTRACT_REDSHIFT_PASSWORD`) — see the [Redshift Reference](../reference/redshift.md).

---

## Import: Amazon S3


Creates a data contract from files in S3, in CSV, JSON, Parquet, or Delta format. Works with S3 and any S3-compatible endpoint (e.g. MinIO).

```bash
datacontract import s3 \
  --source s3://my-bucket/orders/*.json \
  --output datacontract.yaml
```

The format is taken from the file suffix; pass `--format` for Delta tables, which have none, or to override the guess. Add `--endpoint-url` for an S3-compatible store, and `--delimiter` to pin how a JSON file is laid out (`new_line`, `array`, or `none`) instead of letting it be detected.

The objects are read with duckdb through the same connection the test path uses, so the import authenticates identically and infers the column types from exactly the reader that later verifies them.

The generated contract includes a ready-to-test `servers` block, so you can run `datacontract test datacontract.yaml` immediately afterwards — see the **[S3 connection guide](../testing/s3.md)** for the full 5-minute walkthrough and troubleshooting.

Credentials are the same ones `datacontract test` uses: an existing AWS session, or `DATACONTRACT_S3_ACCESS_KEY_ID` and `DATACONTRACT_S3_SECRET_ACCESS_KEY` — see the [S3 Reference](../reference/s3.md).

Working from a local file instead? Use [`datacontract import csv`](./csv.md), [`json`](./json.md), or [`parquet`](./parquet.md).

---

## Import: Snowflake


Creates a data contract from a Snowflake workspace by reading table metadata from `INFORMATION_SCHEMA` — including schemas, column types, primary keys, and comments.

```bash
datacontract import snowflake \
  --source <orgname>-<accountname> \
  --database ORDER_DB \
  --schema PUBLIC \
  --output datacontract.yaml
```

`--source` is your [account identifier](https://docs.snowflake.com/en/user-guide/admin-account-identifier).

The generated contract includes a ready-to-test `servers` block, so you can run `datacontract test datacontract.yaml` immediately afterwards — see the **[Snowflake connection guide](../testing/snowflake.md)** for credentials, the full 5-minute walkthrough, and troubleshooting.

Credentials are provided as environment variables (`DATACONTRACT_SNOWFLAKE_USERNAME`, `DATACONTRACT_SNOWFLAKE_PASSWORD`, `DATACONTRACT_SNOWFLAKE_ROLE`, `DATACONTRACT_SNOWFLAKE_WAREHOUSE`). If no password is set, the import falls back to browser-based SSO. Key-pair auth and `connections.toml` are also supported — see the [Snowflake Reference](../reference/snowflake.md).

---

## Import: Spark


Creates a data contract from a Spark schema. This importer is typically used programmatically from within a Spark context.

```python
# Import table(s) from the Spark context
datacontract import spark --source orders,line_items

# Import a single table
datacontract import spark --source orders

# Import a Spark dataframe (programmatic)
# data_contract = DataContract().import_from_source("spark", dataframe=df)
```

A table description can be supplied alongside the table or dataframe to enrich the generated contract.

---

## Import: SQL DDL

#  Import: SQL

Creates a data contract from a SQL DDL file. Pass the SQL dialect with `--dialect`.

{/* AUTOGENERATED EXAMPLE: do not edit by hand; regenerate with the update script. */}

Given this [`orders.sql`](https://github.com/datacontract/datacontract-cli/blob/main/examples/imports/sql/orders.sql):

```sql
CREATE TABLE orders (
    order_id TEXT PRIMARY KEY,
    order_timestamp TIMESTAMPTZ NOT NULL,
    customer_id TEXT NOT NULL,
    order_total NUMERIC NOT NULL,
    status TEXT NOT NULL
);
```

Run:

```bash
datacontract import sql --source orders.sql --dialect postgres
```

to produce the data contract:

```yaml
version: 1.0.0
kind: DataContract
apiVersion: v3.1.0
id: my-data-contract
name: My Data Contract
status: draft
servers:
- server: postgres
  type: postgres
  database: my_database
  host: my_host
  port: 5432
  schema: public
schema:
- name: orders
  physicalType: table
  logicalType: object
  physicalName: orders
  properties:
  - name: order_id
    physicalType: TEXT
    primaryKey: true
    primaryKeyPosition: 1
    logicalType: string
  - name: order_timestamp
    physicalType: TIMESTAMPTZ
    logicalType: timestamp
    required: true
  - name: customer_id
    physicalType: TEXT
    logicalType: string
    required: true
  - name: order_total
    physicalType: DECIMAL
# …
```

{/* END AUTOGENERATED EXAMPLE */}

---

## Import: SQL Server


Creates a data contract from a SQL Server database by reading `information_schema` — including column types with length and precision, nullability, and primary keys.

```bash
datacontract import sqlserver \
  --source localhost \
  --database mydb \
  --schema dbo \
  --output datacontract.yaml
```

`--source` is the host of your SQL Server instance. Add `--port` if it doesn't listen on the default `1433`, repeat `--table` to import only specific tables, and set `--schema` if your tables don't live in `dbo`.

The connection uses the same helper as `datacontract test`, so every authentication mode works for the import too: SQL logins, Windows integrated auth, the Entra ID modes, and `az login`.

Types are reconstructed exactly as the test path reads them back, so `varchar(36)`, `decimal(10,2)`, `datetime2` and `varchar(max)` all match on the first run.

The generated contract includes a ready-to-test `servers` block, so you can run `datacontract test datacontract.yaml` immediately afterwards — see the **[SQL Server connection guide](../testing/sqlserver.md)** for the full 5-minute walkthrough and troubleshooting.

Credentials are the same ones `datacontract test` uses: `DATACONTRACT_SQLSERVER_USERNAME` and `DATACONTRACT_SQLSERVER_PASSWORD` — see the [SQL Server Reference](../reference/sqlserver.md).

Only have a DDL script? Use [`datacontract import sql --dialect sqlserver`](./sql.md).

---

## Import: Trino


Creates a data contract from a Trino catalog by reading `information_schema`, including the declared column types and nullability.

```bash
datacontract import trino \
  --source localhost \
  --catalog my_catalog \
  --schema my_schema \
  --output datacontract.yaml
```

`--source` is the host of your Trino coordinator. Add `--port` if it doesn't listen on the default `8080`, and repeat `--table` to import only specific tables.

Trino reports a complete type string, so `varchar(36)`, `decimal(10,2)` and `array(varchar)` are taken verbatim and match what `datacontract test` reads back.

The generated contract includes a ready-to-test `servers` block, so you can run `datacontract test datacontract.yaml` immediately afterwards — see the **[Trino connection guide](../testing/trino.md)** for the full 5-minute walkthrough and troubleshooting.

Credentials are the same ones `datacontract test` uses — see the [Trino Reference](../reference/trino.md).

Only have a DDL script? Use [`datacontract import sql --dialect postgres`](./sql.md); Trino's ANSI-style DDL generally parses with that dialect.

---

## Import: Unity Catalog


This importer is now documented as **[Import: Databricks](./databricks.md)**.
The `unity` format name still works.

---

## Exports


`datacontract export` converts a data contract to a target format. Use it to generate DDL, schemas, models, documentation, and data-quality artifacts directly from the contract.

```bash
datacontract export html datacontract.yaml --output datacontract.html
```

Run `datacontract export <format> --help` to see the format-specific options (e.g. `datacontract export sql --help`). If a format you need is missing, [open an issue on GitHub](https://github.com/datacontract/datacontract-cli/issues).

For SQL dialects (`postgres`, `mysql`, `snowflake`, `databricks`, `sqlserver`, `trino`, `oracle`, `clickhouse`), use `datacontract export sql --dialect <dialect>`.

## Example contract

The examples on each export page are generated from the same data contract, so you can compare formats side by side:

- [`examples/orders/orders.odcs.yaml`](https://github.com/datacontract/datacontract-cli/blob/main/examples/orders/orders.odcs.yaml) — an `orders` and `line_items` schema with a Snowflake and a BigQuery server. Used by most pages.
- [`examples/user_interactions/user_interactions.odcs.yaml`](https://github.com/datacontract/datacontract-cli/blob/main/examples/user_interactions/user_interactions.odcs.yaml) — a Databricks contract with DQX quality rules. Used by the [`dqx`](./dqx.md) page.

Download a file and run the commands below against it to reproduce the output.

## Available exporters

  
    
    avroAvro schema.
  
  
    
    avro-idlAvro IDL.
  
  
    
    bigqueryBigQuery schema.
  
  
    
    customAny format, via a custom Jinja template.
  
  
    
    data-catererData Caterer data-generation task.
  
  
    
    dbmlDBML.
  
  
    
    dbt-modelsdbt model schema YAML.
  
  
    
    dbt-sourcesdbt sources YAML.
  
  
    
    dbt-staging-sqlA dbt staging SQL file.
  
  
    
    dcsDCS format.
  
  
    
    dqxDatabricks DQX checks.
  
  
    
    excelODCS Excel template.
  
  
    
    goGo structs.
  
  
    
    great-expectationsGreat Expectations suite.
  
  
    
    htmlA standalone HTML page.
  
  
    
    icebergIceberg schema.
  
  
    
    jsonschemaJSON Schema.
  
  
    
    markdownMarkdown documentation.
  
  
    
    mermaidA Mermaid diagram.
  
  
    
    odcsODCS format.
  
  
    
    protobufProtobuf schema.
  
  
    
    pydantic-modelA Pydantic model.
  
  
    
    rdfRDF representation.
  
  
    
    sodaclSodaCL checks.
  
  
    
    sparkSpark StructType schema.
  
  
    
    sqlSQL DDL (CREATE TABLE).
  
  
    
    sql-queryA SQL SELECT query.
  
  
    
    sqlalchemySQLAlchemy models.
  

See the [`export` command reference](../commands/export.md) for the common signature.

---

## Export: Avro IDL


Converts the data contract to [Avro IDL](https://avro.apache.org/docs/1.11.1/idl-language/).

{/* AUTOGENERATED EXAMPLE: do not edit by hand; regenerate with the update script. */}

```bash
datacontract export avro-idl orders.odcs.yaml --output orders.avdl
```

Running this against the [example `orders` contract](https://github.com/datacontract/datacontract-cli/blob/main/examples/orders/orders.odcs.yaml) produces:

```text
/** Tracks customer orders and their line items for analytics and reporting. */
protocol Orders {
    /** One row per customer order. */
    record orders {
        /** Unique identifier of the order. */
        string order_id;
        /** Timestamp when the order was placed. */
        string order_timestamp;
        /** Reference to the customer who placed the order. */
        string customer_id;
        /** Total amount of the order in cents. */
        double order_total;
        /** Current fulfilment status of the order. */
        string status;
    }
    /** One row per line item within an order. */
    record line_items {
        /** Unique identifier of the line item. */
        string line_item_id;
        /** Reference to the parent order. */
        string order_id;
        /** Stock keeping unit of the purchased product. */
        string sku;
        /** Number of units purchased. */
        int quantity;
    }
}
```

{/* END AUTOGENERATED EXAMPLE */}

---

## Export: Avro


Converts the data contract into an Avro schema. It supports specifying custom Avro properties for logical types and default values.

{/* AUTOGENERATED EXAMPLE: do not edit by hand; regenerate with the update script. */}

```bash
datacontract export avro orders.odcs.yaml --schema-name orders --output orders.avsc
```

Running this against the [example `orders` contract](https://github.com/datacontract/datacontract-cli/blob/main/examples/orders/orders.odcs.yaml) produces:

```json
{
  "type": "record",
  "name": "orders",
  "doc": "One row per customer order.",
  "fields": [
    {
      "name": "order_id",
      "doc": "Unique identifier of the order.",
      "type": "string"
    },
    {
      "name": "order_timestamp",
      "doc": "Timestamp when the order was placed.",
      "type": {
        "type": "int",
        "logicalType": "date"
      }
    },
    {
      "name": "customer_id",
      "doc": "Reference to the customer who placed the order.",
      "type": "string"
    },
    {
      "name": "order_total",
      "doc": "Total amount of the order in cents.",
      "type": "bytes"
    },
    {
      "name": "status",
      "doc": "Current fulfilment status of the order.",
      "type": "string"
    }
  ]
}
```

{/* END AUTOGENERATED EXAMPLE */}

## Custom Avro properties

A **`config` map on property level** may include additional key-value pairs. At the moment, [`logicalType`](https://avro.apache.org/docs/1.11.0/spec.html#Logical+Types) and `default` are supported.

```yaml
schema:
  - name: orders
    properties:
      - name: my_field_1
        description: Example for AVRO with Timestamp (microsecond precision)
        logicalType: integer
        physicalType: long
        examples:
          - 1672534861000000  # 2023-01-01 01:01:01 in microseconds
        required: true
        config:
          avroLogicalType: local-timestamp-micros
          avroDefault: 1672534861000000
```

- `avroLogicalType` — the Avro logical type of the property (here `local-timestamp-micros`).
- `avroDefault` — the default value for the property in Avro.

---

## Export: BigQuery


Converts the data contract to a [BigQuery table schema](https://cloud.google.com/bigquery/docs/schemas) JSON.

{/* AUTOGENERATED EXAMPLE: do not edit by hand; regenerate with the update script. */}

```bash
datacontract export bigquery orders.odcs.yaml --schema-name orders --server bigquery --output orders.bigquery.json
```

Running this against the [example `orders` contract](https://github.com/datacontract/datacontract-cli/blob/main/examples/orders/orders.odcs.yaml) produces:

```json
{
  "kind": "bigquery#table",
  "tableReference": {
    "datasetId": "orders",
    "projectId": "my-gcp-project",
    "tableId": "orders"
  },
  "description": "One row per customer order.",
  "schema": {
    "fields": [
      {
        "name": "order_id",
        "type": "STRING",
        "mode": "REQUIRED",
        "description": "Unique identifier of the order.",
        "maxLength": null
      },
      {
        "name": "order_timestamp",
        "type": "TIMESTAMP",
        "mode": "REQUIRED",
        "description": "Timestamp when the order was placed."
      },
      {
        "name": "customer_id",
        "type": "STRING",
        "mode": "REQUIRED",
        "description": "Reference to the customer who placed the order.",
        "maxLength": null
      },
      {
        "name": "order_total",
        "type": "NUMERIC",
        "mode": "REQUIRED",
        "description": "Total amount of the order in cents.",
        "precision": null,
        "scale": null
      },
      {
        "name": "status",
        "type": "STRING",
        "mode": "REQUIRED",
        "description": "Current fulfilment status of the order.",
        "maxLength": null
      }
    ]
  }
}
```

{/* END AUTOGENERATED EXAMPLE */}

---

## Export: Custom (Jinja)


Converts the data contract into any custom format using a [Jinja](https://jinja.palletsprojects.com/) template. Specify the template path with `--template`.

```bash
datacontract export custom --template template.txt orders.odcs.yaml
```

## Template variables

You can use the ODCS object directly. `data_contract` is an `OpenDataContractStandard` instance; top-level fields include `name`, `id`, `version`, `schema_` (the list of schemas), `servers`, `team`, etc.

Given this `template.txt`:

```text
title: {{ data_contract.name }}
schemas:
{%- for schema in data_contract.schema_ %}
  - name: {{ schema.name }}
{%- endfor %}
```

Running it against the [example `orders` contract](https://github.com/datacontract/datacontract-cli/blob/main/examples/orders/orders.odcs.yaml) produces:

```text
title: Orders
schemas:
  - name: orders
  - name: line_items
```

## Per-schema templates

Add `--schema-name` to render a single schema. The template then also receives:

- `schema_name` (str)
- `schema` (the `SchemaObject` from ODCS)

```bash
datacontract export custom datacontract.odcs.yaml --template template.sql --schema-name orders
```

---

## Export: Data Caterer


Converts the data contract to a data-generation task in YAML that can be ingested by [Data Caterer](https://github.com/data-catering/data-caterer). This lets you generate production-like data in any environment based on your contract.

{/* AUTOGENERATED EXAMPLE: do not edit by hand; regenerate with the update script. */}

```bash
datacontract export data-caterer orders.odcs.yaml
```

Running this against the [example `orders` contract](https://github.com/datacontract/datacontract-cli/blob/main/examples/orders/orders.odcs.yaml) produces:

```yaml
name: Orders
steps:
- name: orders
  type: snowflake
  options: {}
  fields:
  - name: order_id
    type: string
    options:
      isUnique: true
      isPrimaryKey: true
  - name: order_timestamp
    type: timestamp
  - name: customer_id
    type: string
  - name: order_total
    type: double
  - name: status
    type: string
- name: line_items
  type: snowflake
  options: {}
  fields:
  - name: line_item_id
    type: string
    options:
      isUnique: true
      isPrimaryKey: true
  - name: order_id
    type: string
  - name: sku
    type: string
  - name: quantity
    type: integer
```

{/* END AUTOGENERATED EXAMPLE */}

You can further customize generation by adding [additional metadata in the YAML](https://data.catering/setup/generator/data-generator/).

---

## Export: DBML


Converts the data contract to [DBML](https://dbml.dbdiagram.io/) (Database Markup Language).

{/* AUTOGENERATED EXAMPLE: do not edit by hand; regenerate with the update script. */}

```bash
datacontract export dbml orders.odcs.yaml --output orders.dbml
```

Running this against the [example `orders` contract](https://github.com/datacontract/datacontract-cli/blob/main/examples/orders/orders.odcs.yaml) produces:

```text
Project "Orders" {
    note: '''Tracks customer orders and their line items for analytics and reporting.'''
}

Table orders {
    note: "One row per customer order."
    order_id string [pk, unique, not null, note: "Unique identifier of the order."]
    order_timestamp date [not null, note: "Timestamp when the order was placed."]
    customer_id string [not null, note: "Reference to the customer who placed the order."]
    order_total number [not null, note: "Total amount of the order in cents."]
    status string [not null, note: "Current fulfilment status of the order."]
}

Table line_items {
    note: "One row per line item within an order."
    line_item_id string [pk, unique, not null, note: "Unique identifier of the line item."]
    order_id string [not null, note: "Reference to the parent order."]
    sku string [not null, note: "Stock keeping unit of the purchased product."]
    quantity integer [not null, note: "Number of units purchased."]
}
```

{/* END AUTOGENERATED EXAMPLE */}

If a server is selected via `--server`, the logical data types are converted to that database's specific types (based on the server `type`). If no server is selected, the logical data types are exported.

---

## Export: dbt Models


Converts the data contract to dbt models in YAML format, with support for SQL dialects.

{/* AUTOGENERATED EXAMPLE: do not edit by hand; regenerate with the update script. */}

```bash
datacontract export dbt-models orders.odcs.yaml
```

Running this against the [example `orders` contract](https://github.com/datacontract/datacontract-cli/blob/main/examples/orders/orders.odcs.yaml) produces (excerpt):

```yaml
version: 2
models:
- name: orders
  config:
    meta:
      data_contract: urn:datacontract:checkout:orders
    materialized: table
    contract:
      enforced: true
  description: One row per customer order.
  columns:
  - name: order_id
    data_type: VARCHAR
    description: Unique identifier of the order.
    constraints:
    - type: not_null
    - type: unique
  - name: order_timestamp
    data_type: TIMESTAMP_TZ
    description: Timestamp when the order was placed.
    constraints:
    - type: not_null
  - name: customer_id
    data_type: VARCHAR
    description: Reference to the customer who placed the order.
    constraints:
    - type: not_null
  - name: order_total
    data_type: NUMBER
    description: Total amount of the order in cents.
    constraints:
    - type: not_null
  - name: status
    data_type: VARCHAR
    description: Current fulfilment status of the order.
    constraints:
    - type: not_null
- name: line_items
  config:
    meta:
# …
```

{/* END AUTOGENERATED EXAMPLE */}

If a server is selected via `--server` (based on its `type`), the dbt column `data_types` match the expected data types of that server. If no server is selected, it defaults to `snowflake`.

See the [dbt Integration](../dbt.md) guide for the full picture, including `datacontract dbt sync`.

---

## Export: dbt Sources


Converts the data contract to a dbt `sources` YAML definition.

{/* AUTOGENERATED EXAMPLE: do not edit by hand; regenerate with the update script. */}

```bash
datacontract export dbt-sources orders.odcs.yaml
```

Running this against the [example `orders` contract](https://github.com/datacontract/datacontract-cli/blob/main/examples/orders/orders.odcs.yaml) produces (excerpt):

```yaml
version: 2
sources:
- name: urn:datacontract:checkout:orders
  description: Tracks customer orders and their line items for analytics and reporting.
  tables:
  - name: orders
    description: One row per customer order.
    columns:
    - name: order_id
      data_tests:
      - not_null
      - unique
      data_type: VARCHAR
      description: Unique identifier of the order.
    - name: order_timestamp
      data_tests:
      - not_null
      data_type: TIMESTAMP_TZ
      description: Timestamp when the order was placed.
    - name: customer_id
      data_tests:
      - not_null
      data_type: VARCHAR
      description: Reference to the customer who placed the order.
    - name: order_total
      data_tests:
      - not_null
      data_type: NUMBER
      description: Total amount of the order in cents.
    - name: status
      data_tests:
      - not_null
      data_type: VARCHAR
      description: Current fulfilment status of the order.
  - name: line_items
    description: One row per line item within an order.
    columns:
    - name: line_item_id
      data_tests:
      - not_null
# …
```

{/* END AUTOGENERATED EXAMPLE */}

As with [`dbt-models`](./dbt-models.md), selecting a server maps logical types to that server's data types; otherwise `snowflake` is used.

---

## Export: dbt Staging SQL


Generates a dbt staging SQL file that selects the contract's fields.

{/* AUTOGENERATED EXAMPLE: do not edit by hand; regenerate with the update script. */}

```bash
datacontract export dbt-staging-sql orders.odcs.yaml --schema-name orders
```

Running this against the [example `orders` contract](https://github.com/datacontract/datacontract-cli/blob/main/examples/orders/orders.odcs.yaml) produces:

```sql
    select
        order_id, order_timestamp, customer_id, order_total, status
    from {{ source('urn:datacontract:checkout:orders', 'orders') }}
```

{/* END AUTOGENERATED EXAMPLE */}

For fully customized staging models, use the [`custom`](./custom.md) exporter with a Jinja template.

---

## Export: DCS


Exports the data contract to the DCS (Data Contract Specification) format, for downstream tools that still expect it. For the other direction, see [Migrate from DCS to ODCS](../migrate-dcs-to-odcs.md).

{/* AUTOGENERATED EXAMPLE: do not edit by hand; regenerate with the update script. */}

```bash
datacontract export dcs orders.odcs.yaml --output datacontract.yaml
```

Running this against the [example `orders` contract](https://github.com/datacontract/datacontract-cli/blob/main/examples/orders/orders.odcs.yaml) produces (excerpt):

```yaml
id: urn:datacontract:checkout:orders
info:
  title: Orders
  version: 1.0.0
  status: active
  description: Tracks customer orders and their line items for analytics and reporting.
servers:
  production:
    type: snowflake
    account: my-account
    database: ANALYTICS
    schema: ORDERS
  bigquery:
    type: bigquery
    project: my-gcp-project
    dataset: orders
terms:
  usage: Used by the analytics team to build revenue dashboards and by finance for
    reconciliation.
  limitations: Not suitable for real-time fraud detection; data is loaded in hourly
    batches.
  description: Tracks customer orders and their line items for analytics and reporting.
models:
  orders:
    description: One row per customer order.
    type: table
    fields:
      order_id:
        type: string
        required: true
        primaryKey: true
        unique: true
        description: Unique identifier of the order.
      order_timestamp:
# …
```

{/* END AUTOGENERATED EXAMPLE */}

---

## Export: DQX


Converts the contract's [quality rules](../quality-rules/index.md) to [Databricks DQX](https://databrickslabs.github.io/dqx/) checks.

{/* AUTOGENERATED EXAMPLE: do not edit by hand; regenerate with the update script. */}

```bash
datacontract export dqx user_interactions.odcs.yaml --schema-name user_interactions
```

Running this against the [example `user_interactions` contract](https://github.com/datacontract/datacontract-cli/blob/main/examples/user_interactions/user_interactions.odcs.yaml) produces:

```yaml
- criticality: error
  check:
    function: is_not_null
    arguments:
      column: interaction_id
- criticality: error
  check:
    function: is_not_null
    arguments:
      column: user_id
- criticality: error
  check:
    function: is_in_list
    arguments:
      allowed:
      - click
      - view
      - purchase
      column: interaction_type
- criticality: warn
  check:
    function: is_in_range
    arguments:
      min_limit: 0
      max_limit: 1000
      column: interaction_value
```

{/* END AUTOGENERATED EXAMPLE */}

---

## Export: Excel


Converts a data contract into an ODCS Excel template — a user-friendly spreadsheet for authoring, sharing, and managing data contracts.

```bash
datacontract export excel orders.odcs.yaml --output orders.xlsx
```

Running this against the [example `orders` contract](https://github.com/datacontract/datacontract-cli/blob/main/examples/orders/orders.odcs.yaml) produces an `orders.xlsx` workbook with the ODCS template sheets — fundamentals, servers, the `orders` and `line_items` schemas, quality, and service levels — pre-filled from the contract.

The Excel format enables:

- **User-friendly authoring** in Excel's familiar interface.
- **Easy sharing** as standard Excel files.
- **Collaboration** with non-technical stakeholders.
- **Round-trip conversion** back to YAML via [`import excel`](../imports/excel.md).

For the template structure, see the [ODCS Excel Template repository](https://github.com/datacontract/open-data-contract-standard-excel-template).

---

## Export: Go


Generates Go `struct` types from the data contract.

{/* AUTOGENERATED EXAMPLE: do not edit by hand; regenerate with the update script. */}

```bash
datacontract export go orders.odcs.yaml --output orders.go
```

Running this against the [example `orders` contract](https://github.com/datacontract/datacontract-cli/blob/main/examples/orders/orders.odcs.yaml) produces:

```go
package main

type Orders struct {
    OrderId string `json:"order_id" avro:"order_id"`  // Unique identifier of the order.
    OrderTimestamp time.Time `json:"order_timestamp" avro:"order_timestamp"`  // Timestamp when the order was placed.
    CustomerId string `json:"customer_id" avro:"customer_id"`  // Reference to the customer who placed the order.
    OrderTotal float64 `json:"order_total" avro:"order_total"`  // Total amount of the order in cents.
    Status string `json:"status" avro:"status"`  // Current fulfilment status of the order.
}

type LineItems struct {
    LineItemId string `json:"line_item_id" avro:"line_item_id"`  // Unique identifier of the line item.
    OrderId string `json:"order_id" avro:"order_id"`  // Reference to the parent order.
    Sku string `json:"sku" avro:"sku"`  // Stock keeping unit of the purchased product.
    Quantity int `json:"quantity" avro:"quantity"`  // Number of units purchased.
}
```

{/* END AUTOGENERATED EXAMPLE */}

---

## Export: Great Expectations


Transforms a data contract into a comprehensive [Great Expectations](https://greatexpectations.io/) JSON suite. If the contract includes multiple models, specify the model with `--schema-name`.

{/* AUTOGENERATED EXAMPLE: do not edit by hand; regenerate with the update script. */}

```bash
datacontract export great-expectations orders.odcs.yaml --schema-name orders
```

Running this against the [example `orders` contract](https://github.com/datacontract/datacontract-cli/blob/main/examples/orders/orders.odcs.yaml) produces:

```json
{
  "name": "orders.1.0.0",
  "expectations": [
    {
      "type": "expect_table_columns_to_match_ordered_list",
      "kwargs": {
        "column_list": [
          "order_id",
          "order_timestamp",
          "customer_id",
          "order_total",
          "status"
        ]
      },
      "meta": {}
    },
    {
      "type": "expect_column_values_to_be_of_type",
      "kwargs": {
        "column": "order_id",
        "type_": "VARCHAR"
      },
      "meta": {}
    },
    {
      "type": "expect_column_values_to_be_unique",
      "kwargs": {
        "column": "order_id"
      },
      "meta": {}
    },
    {
      "type": "expect_column_values_to_be_of_type",
      "kwargs": {
        "column": "order_timestamp",
        "type_": "TIMESTAMP"
      },
      "meta": {}
    },
    {
      "type": "expect_column_values_to_be_of_type",
      "kwargs": {
        "column": "customer_id",
        "type_": "VARCHAR"
      },
      "meta": {}
    },
    {
      "type": "expect_column_values_to_be_of_type",
      "kwargs": {
        "column": "order_total",
        "type_": "NUMBER"
      },
      "meta": {}
    },
    {
      "type": "expect_column_values_to_be_of_type",
      "kwargs": {
        "column": "status",
        "type_": "VARCHAR"
      },
      "meta": {}
    }
  ],
  "meta": {}
}
```

{/* END AUTOGENERATED EXAMPLE */}

The export builds expectations from the model definition (with a fixed mapping) and from the `quality` rules of each model (see the [expectations gallery](https://greatexpectations.io/expectations/)).

## Additional options

- `suite_name` — the name of the expectation suite. Defaults to a name derived from the model name(s).
- `engine` — the execution engine: `pandas` (in-memory dataframes), `spark` (Spark dataframes), or `sql` (SQL databases).
- `sql_server_type` — the SQL server type to connect with when `engine` is `sql`. Ensures the correct SQL dialect and connection settings are applied.

---

## Export: HTML


Generates a standalone, self-contained HTML page documenting the data contract.

```bash
datacontract export html orders.odcs.yaml --output orders.html
```

Running this against the [example `orders` contract](https://github.com/datacontract/datacontract-cli/blob/main/examples/orders/orders.odcs.yaml) produces a single self-contained `orders.html` file (no external assets) that renders:

- the contract metadata (name, version, status, description),
- each schema (`orders`, `line_items`) as a table of fields with their types, constraints, and descriptions,
- the configured servers, quality checks, and service-level properties.

Open the file in any browser or publish it as static documentation.

---

## Export: Iceberg


Exports to an [Iceberg Table JSON Schema Definition](https://iceberg.apache.org/spec/#appendix-c-json-serialization).

This export supports a single model at a time, because Iceberg's schema definition is for a single table (1 model → 1 table). Use `--schema-name` to select the model.

{/* AUTOGENERATED EXAMPLE: do not edit by hand; regenerate with the update script. */}

```bash
datacontract export iceberg orders.odcs.yaml --schema-name orders --output orders.iceberg.json
```

Running this against the [example `orders` contract](https://github.com/datacontract/datacontract-cli/blob/main/examples/orders/orders.odcs.yaml) produces:

```json
{"type":"struct","fields":[{"id":1,"name":"order_id","type":"string","required":true},{"id":2,"name":"order_timestamp","type":"timestamptz","required":true},{"id":3,"name":"customer_id","type":"string","required":true},{"id":4,"name":"order_total","type":"decimal(38, 0)","required":true},{"id":5,"name":"status","type":"string","required":true}],"schema-id":0,"identifier-field-ids":[1]}
```

{/* END AUTOGENERATED EXAMPLE */}

```json
{
  "type": "struct",
  "fields": [
    { "id": 1, "name": "order_id", "type": "string", "required": true },
    { "id": 2, "name": "order_timestamp", "type": "timestamptz", "required": true }
  ],
  "schema-id": 0,
  "identifier-field-ids": [1]
}
```

---

## Export: JSON Schema


Converts the data contract to a [JSON Schema](https://json-schema.org/) document.

{/* AUTOGENERATED EXAMPLE: do not edit by hand; regenerate with the update script. */}

```bash
datacontract export jsonschema orders.odcs.yaml --schema-name orders --output orders.schema.json
```

Running this against the [example `orders` contract](https://github.com/datacontract/datacontract-cli/blob/main/examples/orders/orders.odcs.yaml) produces:

```json
{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "properties": {
    "order_id": {
      "type": "string",
      "primaryKey": true,
      "unique": true,
      "description": "Unique identifier of the order."
    },
    "order_timestamp": {
      "type": "string",
      "format": "date",
      "description": "Timestamp when the order was placed."
    },
    "customer_id": {
      "type": "string",
      "description": "Reference to the customer who placed the order."
    },
    "order_total": {
      "type": "number",
      "description": "Total amount of the order in cents."
    },
    "status": {
      "type": "string",
      "description": "Current fulfilment status of the order."
    }
  },
  "required": [
    "order_id",
    "order_timestamp",
    "customer_id",
    "order_total",
    "status"
  ],
  "description": "One row per customer order."
}
```

{/* END AUTOGENERATED EXAMPLE */}

---

## Export: Markdown


Generates Markdown documentation for the data contract.

{/* AUTOGENERATED EXAMPLE: do not edit by hand; regenerate with the update script. */}

```bash
datacontract export markdown orders.odcs.yaml --output orders.md
```

Running this against the [example `orders` contract](https://github.com/datacontract/datacontract-cli/blob/main/examples/orders/orders.odcs.yaml) produces (excerpt):

```markdown
# urn:datacontract:checkout:orders
## Info
*Tracks customer orders and their line items for analytics and reporting.*
- **name:** Orders
- **version:** 1.0.0
- **status:** active

## Terms of Use
### Usage
Used by the analytics team to build revenue dashboards and by finance for reconciliation.

### Purpose
Tracks customer orders and their line items for analytics and reporting.

### Limitations
Not suitable for real-time fraud detection; data is loaded in hourly batches.

## Servers
| Name | Type | Attributes |
| ---- | ---- | ---------- |
| production | snowflake | *No description.*• **account:** my-account• **database:** ANALYTICS• **schema_:** ORDERS |
| bigquery | bigquery | *No description.*• **dataset:** orders• **project:** my-gcp-project |

## Schema
### orders
*One row per customer order.*

| Field | Type | Attributes |
| ----- | ---- | ---------- |
|  order_id | string | *Unique identifier of the order.*• `primaryKey`• `required`• `unique`• **examples:** ['ORD-1001', 'ORD-1002'] |
|  order_timestamp | date | *Timestamp when the order was placed.*• `required` |
|  customer_id | string | *Reference to the customer who placed the order.*• `required` |
|  order_total | number | *Total amount of the order in cents.*• `required`• **quality:** [{'description': 'Order total is never negative.', 'type': 'sql', 'mustBe': 0, 'query': 'SELECT COUNT(*) FROM orders WHERE order_total < 0'}] |
|  status | string | *Current fulfilment status of the order.*• `required`• **examples:** ['pending', 'shipped', 'delivered'] |
<!-- … -->
```

{/* END AUTOGENERATED EXAMPLE */}

---

## Export: Mermaid


Generates a [Mermaid](https://mermaid.js.org/) entity-relationship diagram of the contract's schemas.

{/* AUTOGENERATED EXAMPLE: do not edit by hand; regenerate with the update script. */}

```bash
datacontract export mermaid orders.odcs.yaml
```

Running this against the [example `orders` contract](https://github.com/datacontract/datacontract-cli/blob/main/examples/orders/orders.odcs.yaml) produces:

```mermaid
erDiagram
        "**orders**" {
        order_id🔑🔒 string
        order_timestamp date
        customer_id string
        order_total number
        status string
}
        "**line_items**" {
        line_item_id🔑🔒 string
        order_id string
        sku string
        quantity integer
}
```

{/* END AUTOGENERATED EXAMPLE */}

---

## Export: ODCS


Exports the data contract to the [Open Data Contract Standard](../open-data-contract-standard.md) (ODCS) YAML format. Useful for normalizing or converting an in-memory contract back to canonical ODCS.

This is also the command that converts a legacy DCS contract — see [Migrate from DCS to ODCS](../migrate-dcs-to-odcs.md).

{/* AUTOGENERATED EXAMPLE: do not edit by hand; regenerate with the update script. */}

```bash
datacontract export odcs orders.odcs.yaml --output orders.normalized.yaml
```

Running this against the [example `orders` contract](https://github.com/datacontract/datacontract-cli/blob/main/examples/orders/orders.odcs.yaml) produces (excerpt):

```yaml
version: 1.0.0
kind: DataContract
apiVersion: v3.1.0
id: urn:datacontract:checkout:orders
name: Orders
tags:
- checkout
- orders
status: active
servers:
- server: production
  type: snowflake
  account: my-account
  database: ANALYTICS
  schema: ORDERS
- server: bigquery
  type: bigquery
  dataset: orders
  project: my-gcp-project
description:
  usage: Used by the analytics team to build revenue dashboards and by finance for
    reconciliation.
  purpose: Tracks customer orders and their line items for analytics and reporting.
  limitations: Not suitable for real-time fraud detection; data is loaded in hourly
    batches.
schema:
- name: orders
  physicalType: table
  description: One row per customer order.
  physicalName: orders
  properties:
  - name: order_id
# …
```

{/* END AUTOGENERATED EXAMPLE */}

---

## Export: Protobuf


Converts the data contract to a [Protocol Buffers](https://protobuf.dev/) schema.

{/* AUTOGENERATED EXAMPLE: do not edit by hand; regenerate with the update script. */}

```bash
datacontract export protobuf orders.odcs.yaml --output orders.proto
```

Running this against the [example `orders` contract](https://github.com/datacontract/datacontract-cli/blob/main/examples/orders/orders.odcs.yaml) produces:

```protobuf
syntax = "proto3";

package example;

// One row per customer order.
message Orders {
  // Unique identifier of the order.
  string order_id = 1;
  // Timestamp when the order was placed.
  string order_timestamp = 2;
  // Reference to the customer who placed the order.
  string customer_id = 3;
  // Total amount of the order in cents.
  double order_total = 4;
  // Current fulfilment status of the order.
  string status = 5;
}

// One row per line item within an order.
message LineItems {
  // Unique identifier of the line item.
  string line_item_id = 1;
  // Reference to the parent order.
  string order_id = 2;
  // Stock keeping unit of the purchased product.
  string sku = 3;
  // Number of units purchased.
  int32 quantity = 4;
}
```

{/* END AUTOGENERATED EXAMPLE */}

## Configuration

You can customize the generated Protobuf package name using the data contract's `customProperties`:

```yaml
kind: DataContract
apiVersion: v3.1.0
id: my-contract
version: 1.0.0
customProperties:
  - property: protoPackageName
    value: com.example.mydata
schema:
  - name: Orders
    properties:
      # ... fields ...
```

### Supported Properties

- **`protoPackageName`** (optional) — The package name for the generated Protobuf file. Defaults to `"example"` if not specified.

---

## Export: Pydantic Model


Generates [Pydantic](https://docs.pydantic.dev/) model classes from the data contract.

{/* AUTOGENERATED EXAMPLE: do not edit by hand; regenerate with the update script. */}

```bash
datacontract export pydantic-model orders.odcs.yaml --output orders.py
```

Running this against the [example `orders` contract](https://github.com/datacontract/datacontract-cli/blob/main/examples/orders/orders.odcs.yaml) produces:

```python
import datetime, typing, pydantic, decimal
'Tracks customer orders and their line items for analytics and reporting.'

class Orders(pydantic.BaseModel):
    """One row per customer order."""
    order_id: str
    'Unique identifier of the order.'
    order_timestamp: datetime.datetime
    'Timestamp when the order was placed.'
    customer_id: str
    'Reference to the customer who placed the order.'
    order_total: float
    'Total amount of the order in cents.'
    status: str
    'Current fulfilment status of the order.'

class Line_items(pydantic.BaseModel):
    """One row per line item within an order."""
    line_item_id: str
    'Unique identifier of the line item.'
    order_id: str
    'Reference to the parent order.'
    sku: str
    'Stock keeping unit of the purchased product.'
    quantity: int
    'Number of units purchased.'
```

{/* END AUTOGENERATED EXAMPLE */}

---

## Export: RDF


Converts the data contract into an RDF representation. You can add a base URL used as the default prefix to resolve relative IRIs inside the document.

```bash
datacontract export rdf orders.odcs.yaml --output orders.ttl
```

Running this against the [example `orders` contract](https://github.com/datacontract/datacontract-cli/blob/main/examples/orders/orders.odcs.yaml) produces (excerpt):

```turtle
@prefix odcs: <https://github.com/bitol-io/open-data-contract-standard/> .
@prefix odcsx: <https://github.com/bitol-io/open-data-contract-standard/extension/> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .

<urn:datacontract:checkout:orders> a odcs:DataContract ;
    odcs:apiVersion "v3.1.0" ;
    odcs:id "urn:datacontract:checkout:orders" ;
    odcs:info [ a odcs:Info ;
            odcs:description "Tracks customer orders and their line items for analytics and reporting." ;
            odcs:name "Orders" ;
            odcs:version "1.0.0" ] ;
    odcs:kind "DataContract" ;
    odcs:schema_ <line_items>,
        <orders> ;
    odcs:server <bigquery>,
        <production> .

<bigquery> a odcs:Server ;
    odcsx:dataset "orders" ;
    odcsx:project "my-gcp-project" ;
    odcsx:server "bigquery" ;
    odcs:type "bigquery" .

<line_items> a odcs:Schema ;
    odcs:description "One row per line item within an order." ;
    odcs:property [ a odcs:Property ;
            odcs:description "Unique identifier of the line item." ;
            odcsx:primaryKey true ;
            odcs:logicalType "string" ;
            odcs:name "line_item_id" ;
# …
```

The contract is mapped onto concepts of a Data Contract Ontology (`DataContract`, `Server`, `Model`). Having the contract as an RDF graph enables interoperability with other formats, storage in a knowledge graph, semantic search, linking to established ontologies, OWL reasoning, and graph algorithms across multiple contracts.

---

## Export: SodaCL


Converts the contract's schema and [quality rules](../quality-rules/index.md) into [SodaCL](https://docs.soda.io/soda-cl/soda-cl-overview.html) checks.

{/* AUTOGENERATED EXAMPLE: do not edit by hand; regenerate with the update script. */}

```bash
datacontract export sodacl orders.odcs.yaml --output sodacl.yaml
```

Running this against the [example `orders` contract](https://github.com/datacontract/datacontract-cli/blob/main/examples/orders/orders.odcs.yaml) produces (excerpt):

```yaml
checks for line_items:
- schema:
    fail:
      when required column missing:
      - line_item_id
    name: line_items__line_item_id__field_is_present
- schema:
    fail:
      when wrong column type:
        line_item_id: VARCHAR
    name: line_items__line_item_id__field_type
- missing_count(line_item_id) = 0:
    name: line_items__line_item_id__field_required
- duplicate_count(line_item_id) = 0:
    name: line_items__line_item_id__field_unique
- schema:
    fail:
      when required column missing:
      - order_id
    name: line_items__order_id__field_is_present
- schema:
    fail:
      when wrong column type:
        order_id: VARCHAR
    name: line_items__order_id__field_type
- missing_count(order_id) = 0:
    name: line_items__order_id__field_required
- schema:
    fail:
      when required column missing:
      - sku
    name: line_items__sku__field_is_present
# …
```

{/* END AUTOGENERATED EXAMPLE */}

---

## Export: Spark


Converts the data contract into a Spark `StructType` schema. The returned value is Python code representing the model schemas.

{/* AUTOGENERATED EXAMPLE: do not edit by hand; regenerate with the update script. */}

```bash
datacontract export spark orders.odcs.yaml
```

Running this against the [example `orders` contract](https://github.com/datacontract/datacontract-cli/blob/main/examples/orders/orders.odcs.yaml) produces (excerpt):

```python
orders = StructType([
    StructField("order_id",
        StringType(),
        False,
        {"comment": "Unique identifier of the order."}
    ),
    StructField("order_timestamp",
        TimestampType(),
        False,
        {"comment": "Timestamp when the order was placed."}
    ),
    StructField("customer_id",
        StringType(),
        False,
        {"comment": "Reference to the customer who placed the order."}
    ),
    StructField("order_total",
        DecimalType(10, 0),
        False,
        {"comment": "Total amount of the order in cents."}
    ),
    StructField("status",
        StringType(),
        False,
        {"comment": "Current fulfilment status of the order."}
    )
])

line_items = StructType([
    StructField("line_item_id",
        StringType(),
        False,
        {"comment": "Unique identifier of the line item."}
    ),
# …
```

{/* END AUTOGENERATED EXAMPLE */}

For Spark data types, see the [Spark documentation](https://spark.apache.org/docs/latest/sql-ref-datatypes.html).

---

## Export: SQL Query


Exports a data contract to a SQL `SELECT` query over the contract's fields.

{/* AUTOGENERATED EXAMPLE: do not edit by hand; regenerate with the update script. */}

```bash
datacontract export sql-query orders.odcs.yaml --schema-name orders
```

Running this against the [example `orders` contract](https://github.com/datacontract/datacontract-cli/blob/main/examples/orders/orders.odcs.yaml) produces:

```sql
-- Data Contract: urn:datacontract:checkout:orders
-- SQL Dialect: snowflake
select
    order_id,
    order_timestamp,
    customer_id,
    order_total,
    status
from orders
```

{/* END AUTOGENERATED EXAMPLE */}

---

## Export: SQL DDL

#  Export: SQL

Converts a data contract into a SQL data definition language (DDL) file.

{/* AUTOGENERATED EXAMPLE: do not edit by hand; regenerate with the update script. */}

```bash
datacontract export sql orders.odcs.yaml --output orders.sql
```

Running this against the [example `orders` contract](https://github.com/datacontract/datacontract-cli/blob/main/examples/orders/orders.odcs.yaml) produces:

```sql
-- Data Contract: urn:datacontract:checkout:orders
-- SQL Dialect: snowflake
CREATE TABLE orders (
  order_id VARCHAR not null primary key COMMENT 'Unique identifier of the order.',
  order_timestamp TIMESTAMP_TZ not null COMMENT 'Timestamp when the order was placed.',
  customer_id VARCHAR not null COMMENT 'Reference to the customer who placed the order.',
  order_total NUMBER not null COMMENT 'Total amount of the order in cents.',
  status VARCHAR not null COMMENT 'Current fulfilment status of the order.'
) COMMENT='One row per customer order.';
CREATE TABLE line_items (
  line_item_id VARCHAR not null primary key COMMENT 'Unique identifier of the line item.',
  order_id VARCHAR not null COMMENT 'Reference to the parent order.',
  sku VARCHAR not null COMMENT 'Stock keeping unit of the purchased product.',
  quantity NUMBER not null COMMENT 'Number of units purchased.'
) COMMENT='One row per line item within an order.';
```

{/* END AUTOGENERATED EXAMPLE */}

The SQL dialect is determined from the `servers` block in the data contract (e.g. `type: postgres`, `type: snowflake`). Alternatively, pass it explicitly:

```bash
datacontract export sql datacontract.yaml --dialect postgres --output output.sql
```

Supported dialects: `postgres`, `mysql`, `snowflake`, `databricks`, `sqlserver`, `trino`, `oracle`, `clickhouse` (and `auto`, the default, which detects the dialect from the server type).

:::note[Databricks `variant` columns]
If an error is thrown when deploying SQL DDLs with `variant` columns on Databricks, set:

```python
spark.conf.set("spark.databricks.delta.schema.typeCheck.enabled", "false")
```
:::

## ClickHouse

```bash
datacontract export sql datacontract.yaml --dialect clickhouse --output ddl.sql
```

ClickHouse requires a table engine (default `ENGINE = MergeTree()`) and uses `ORDER BY` as its primary key mechanism. Primary key fields in the contract are translated to the `ORDER BY` clause.

```bash
# Custom engine
datacontract export sql datacontract.yaml --dialect clickhouse \
  --clickhouse-engine "ReplicatedMergeTree('/clickhouse/tables/{shard}/table', '{replica}')"

# Custom ORDER BY (defaults to primary key columns if defined)
datacontract export sql datacontract.yaml --dialect clickhouse \
  --clickhouse-order-by "event_date DESC, event_id"
```

| Option | Description |
|---|---|
| `--clickhouse-engine` | The table engine. Default: `MergeTree()`. |
| `--clickhouse-order-by` | Comma-separated `ORDER BY` columns. Defaults to primary key columns. |

Override any field's ClickHouse type individually by setting the custom property `clickhouseType` in the data contract.

---

## Export: SQLAlchemy


Generates [SQLAlchemy](https://www.sqlalchemy.org/) ORM models from the data contract.

{/* AUTOGENERATED EXAMPLE: do not edit by hand; regenerate with the update script. */}

```bash
datacontract export sqlalchemy orders.odcs.yaml --output orders_models.py
```

Running this against the [example `orders` contract](https://github.com/datacontract/datacontract-cli/blob/main/examples/orders/orders.odcs.yaml) produces:

```python
from sqlalchemy.orm import DeclarativeBase
from sqlalchemy import Column, Date, Integer, Numeric, String, Text, VARCHAR, BigInteger, Float, Double, Boolean, Date, ARRAY, LargeBinary
from sqlalchemy import TIMESTAMP
'Tracks customer orders and their line items for analytics and reporting.'

class Base(DeclarativeBase):
    pass

class Orders(Base):
    """One row per customer order."""
    __tablename__ = 'orders'
    __table_args__ = {'comment': 'One row per customer order.', 'schema': None}
    order_id = Column(String(None), nullable=False, comment='Unique identifier of the order.', primary_key=True)
    order_timestamp = Column(Date, nullable=False, comment='Timestamp when the order was placed.', primary_key=None)
    customer_id = Column(String(None), nullable=False, comment='Reference to the customer who placed the order.', primary_key=None)
    order_total = Column(Numeric(None, None), nullable=False, comment='Total amount of the order in cents.', primary_key=None)
    status = Column(String(None), nullable=False, comment='Current fulfilment status of the order.', primary_key=None)

class Line_items(Base):
    """One row per line item within an order."""
    __tablename__ = 'line_items'
    __table_args__ = {'comment': 'One row per line item within an order.', 'schema': None}
    line_item_id = Column(String(None), nullable=False, comment='Unique identifier of the line item.', primary_key=True)
    order_id = Column(String(None), nullable=False, comment='Reference to the parent order.', primary_key=None)
    sku = Column(String(None), nullable=False, comment='Stock keeping unit of the purchased product.', primary_key=None)
    quantity = Column(Integer, nullable=False, comment='Number of units purchased.', primary_key=None)
```

{/* END AUTOGENERATED EXAMPLE */}

---

## Data Source Reference


Lookup material for every supported data source: all **authentication** options (environment variables) and how **data types** are mapped when importing a contract and checked when testing one. For task-oriented walkthroughs, see [Test your Data](../testing/index.md).

## How authentication works

Connection details (host, catalog, location, …) live in the contract's `servers` block; credentials are provided as environment variables. Variables are also loaded from a `.env` file in the current working directory (or the nearest parent directory containing one); already-set environment variables take precedence.

## How data types work

A contract property carries up to two type declarations:

- **`logicalType`** — one of nine portable ODCS types: `string`, `integer`, `number`, `boolean`, `date`, `timestamp`, `time`, `object`, `array`.
- **`physicalType`** — free text for the native type in the data source (e.g. `VARCHAR(255)`, `NUMBER(38,0)`). It is not validated by `datacontract lint`; at test time it is interpreted in the SQL dialect of the server under test.

When you run `datacontract test`, type checks work in one of two modes:

1. **Native type check** (`Check that field x has physical type y`) — on sources with catalog introspection (Snowflake, BigQuery, Databricks, Postgres, Redshift, SQL Server, Oracle, Trino, Athena), the declared `physicalType` is compared against the actual column type from the catalog. Timezone variants of timestamps are interchangeable; length/precision is only enforced when the contract declares it (`varchar` matches `varchar(255)`, but `varchar(255)` does not match `varchar(100)`). A `physicalType` that can't be interpreted in the server's dialect degrades to the logical check or a warning — never a hard failure.
2. **Logical type check** (`Check that field x has type y`) — everywhere else (and as fallback), both the declared and the actual type are normalized to one of the nine ODCS categories and compared. `integer` and `number` are mutually compatible; a bare `object` or `array` matches any structure with the same base.

For file sources with `format: csv`, `json`, or `avro`, no type checks are generated — the file is read *as* the contract's types, and violations surface as read errors (plus JSON Schema validation for `format: json`).

Independent of the type checks, `logicalTypeOptions` (`minimum`, `maximum`, `minLength`, `maxLength`, `pattern`, `enum`, …) generate value checks that behave identically on every source.

## Data sources

  
    
    Amazon AthenaAuthentication and data types
  
  
    
    Amazon RedshiftAuthentication and data types
  
  
    
    Amazon S3Authentication and data types
  
  
    
    Apache ImpalaAuthentication and data types
  
  
    
    Azure Blob / ADLSAuthentication and data types
  
  
    
    DatabricksAuthentication and data types
  
  
    
    Google BigQueryAuthentication and data types
  
  
    
    Google Cloud StorageAuthentication and data types
  
  
    
    HTTP APIAuthentication and data types
  
  
    
    KafkaAuthentication and data types
  
  
    
    Local filesData types for CSV, JSON, Parquet, Delta
  
  
    
    Microsoft SQL ServerAuthentication and data types
  
  
    
    MySQLAuthentication and data types
  
  
    
    OracleAuthentication and data types
  
  
    
    PostgresAuthentication and data types
  
  
    
    SnowflakeAuthentication and data types
  
  
    
    Spark DataFrameData types for in-memory DataFrames
  
  
    
    TrinoAuthentication and data types

---

## HTTP API Reference


Authentication options and data type handling for [HTTP API connections](../testing/api.md).

## Server

```yaml
servers:
  - server: production
    type: api
    location: "https://api.example.com/orders"
    format: json
```

## Authentication

| Variable | Example | Description |
|---|---|---|
| `DATACONTRACT_API_HEADER_AUTHORIZATION` | `Bearer <token>` | Value for the `authorization` header (optional) |

The `location` (URL) and `delimiter` (`new_line`, `array`, or `none`) come from the contract's `servers` block. Only GET requests are supported.

## Data types

The response is downloaded and tested like a local JSON file: records are read as the contract's types and validated against a JSON Schema derived from the contract's `logicalType`s. The nine ODCS logical types map to JSON Schema types (`string`, `integer`, `number`, `boolean`, `object`, `array`; `date`/`timestamp`/`time` become `string` with `format: date`/`date-time`/`time`); non-required fields also accept `null`. `physicalType` is not checked. Value constraints come from `logicalTypeOptions` (`pattern`, `minimum`, `enum`, …).

---

## Amazon Athena Reference


Authentication options and data type handling for [Athena connections](../testing/athena.md).

## Server

```yaml
servers:
  - server: athena
    type: athena
    catalog: awsdatacatalog # default
    schema: my_database     # in Athena, this is called "database"
    regionName: eu-central-1
    stagingDir: s3://my-bucket/athena-results/
```

## Authentication

Athena authenticates as an AWS principal, so an existing AWS session is enough — `aws sso login`, `AWS_PROFILE`, EC2/ECS/EKS instance roles, or GitHub OIDC in CI. Unlike Redshift, no database user is involved, so nothing has to be minted or configured.

Set the variables below only to override that with static keys:

| Variable | Example | Description |
|---|---|---|
| `DATACONTRACT_S3_REGION` | `eu-central-1` | Region of the Athena service |
| `DATACONTRACT_S3_ACCESS_KEY_ID` | `AKIAXV5Q5Q...` | AWS Access Key ID |
| `DATACONTRACT_S3_SECRET_ACCESS_KEY` | `93S7LRrJ...` | AWS Secret Access Key |
| `DATACONTRACT_S3_SESSION_TOKEN` | `AQoDYXdzEJr...` | AWS temporary session token (optional) |

`catalog`, `schema` (the Athena database), `regionName`, and `stagingDir` (required, for query results) come from the contract's `servers` block; `DATACONTRACT_S3_REGION` takes precedence over `regionName` when both are set. The credentials need `athena:StartQueryExecution` plus read access to the data and write access to `stagingDir`, and `glue:GetTables` to import.

## Data types

### Importing

`datacontract import athena` reads the table metadata from the AWS Glue Data Catalog, where Athena keeps it. Glue stores the Hive spelling of two types, which the import rewrites to what Athena reports back — `string` → `varchar` and `binary` → `varbinary` — so the physical type checks pass on the first test run. Every other type already compares equal in the Athena dialect (`int`/`integer`, `array<int>`/`array(integer)`, `struct`/`row(...)`, `decimal`/`decimal(10,2)`).

Types map through the shared Glue mapping: `string`/`varchar`/`char` → `string`, `int`/`bigint`/`smallint`/`tinyint` → `integer`, `float`/`double`/`decimal` → `number`, `boolean` → `boolean`, `date` → `date`, `timestamp` → `timestamp`, `array<...>` → `array`, `struct<...>` → `object`, `binary` → `string` (format `binary`), `map<...>` → no logical type.

### Testing

Athena supports **native type introspection**: the declared `physicalType` is checked against the actual column type from the Athena catalog. Timezone variants of timestamps are interchangeable; parameters are only enforced when declared. A `physicalType` that isn't valid Athena SQL falls back to the logical type category comparison.

---

## Azure Blob / ADLS Reference


Authentication options and data type handling for [Azure storage connections](../testing/azure.md).

## Server

```yaml
servers:
  - server: production
    type: azure
    location: abfss://datameshdatabricksdemo.dfs.core.windows.net/inventory_events/*.parquet
    format: parquet
```

## Authentication

Authentication uses an Azure Service Principal (App Registration) with a secret:

| Variable | Example | Description |
|---|---|---|
| `DATACONTRACT_AZURE_TENANT_ID` | `79f5b80f-...` | The Azure Tenant ID |
| `DATACONTRACT_AZURE_CLIENT_ID` | `3cf7ce49-...` | The Application/Client ID of the app registration |
| `DATACONTRACT_AZURE_CLIENT_SECRET` | `yZK8Q~GWO1M...` | The client secret value |

The service principal needs the **Storage Blob Data Reader** role. The [metadata-check path](../testing/azure.md#metadata-checks) alternatively accepts `DATACONTRACT_AZURE_CONNECTION_STRING` or `DATACONTRACT_AZURE_STORAGE_ACCOUNT_KEY`.

Supported `location` URL formats (in the `servers` block):

- `https://<account>.blob.core.windows.net/<container>/<prefix>`
- `abfss://<container>@<account>.dfs.core.windows.net/<prefix>`
- `azure://<container>@<account>.blob.core.windows.net/<prefix>`
- `wasbs://<container>@<account>.blob.core.windows.net/<prefix>`

## Data types

Files on Azure storage are read with DuckDB. Type handling depends on the `format` in the `servers` block:

| `format` | How types are handled |
|---|---|
| `csv` | The file is read **as the contract's types** — no type checks are generated; a value that can't be coerced surfaces as a read error. |
| `json` | Same as CSV, plus every record is validated against a JSON Schema derived from the contract's `logicalType`s. |
| `parquet` | Column types come from the Parquet file; the contract's `logicalType` is checked by category. |
| `delta` | Column types come from the Delta table; the contract's `logicalType` is checked by category. |

`physicalType` is never checked against file sources. Schema objects with `logicalType: blob` / `physicalType: file` switch to [metadata checks](../testing/azure.md#metadata-checks) against blob properties instead of reading file contents.

{/* AUTOGENERATED TYPE MAPPING: do not edit by hand; regenerate with update_reference_types.py */}

### Logical type mapping

When no `physicalType` is declared, the CLI derives the DuckDB column type from the `logicalType` when reading files. This table is generated from the converters in the CLI's code:

| `logicalType` | CSV read type | Parquet read type |
|---|---|---|
| `string` | `VARCHAR` | `VARCHAR` |
| `integer` | `BIGINT` | `INTEGER` |
| `number` | `DOUBLE` | `DECIMAL` |
| `boolean` | `BOOLEAN` | `BOOLEAN` |
| `date` | `DATE` | `DATE` |
| `timestamp` | `TIMESTAMP` | `TIMESTAMP WITH TIME ZONE` |
| `time` | `TIME` | `TIME` |
| `object` | `VARCHAR` | `STRUCT()` |
| `array` | `VARCHAR` | `VARCHAR[]` |

{/* END AUTOGENERATED TYPE MAPPING */}

---

## Google BigQuery Reference


Authentication options and data type handling for [BigQuery connections](../testing/bigquery.md).

## Server

```yaml
servers:
  - server: production
    type: bigquery
    project: datameshexample-product
    dataset: datacontract_cli_test_dataset
```

## Authentication

Authentication uses a Service Account Key or Application Default Credentials (ADC) — including Workload Identity Federation (WIF), the GCE metadata server, and `gcloud auth application-default login`. The account needs the **BigQuery Job User** and **BigQuery Data Viewer** roles.

| Variable | Example | Description |
|---|---|---|
| `DATACONTRACT_BIGQUERY_ACCOUNT_INFO_JSON_PATH` | `~/service-access-key.json` | Service Account key JSON file used by `test`. If unset, ADC/WIF is used. |
| `DATACONTRACT_BIGQUERY_BILLING_PROJECT` | `my-compute-project` | Optional. Project to bill query jobs to. Requires `bigquery.jobUser` on the billing project and `bigquery.dataViewer` on the data project. |

`project` and `dataset` come from the contract's `servers` block. `datacontract import bigquery` always uses ADC — set `GOOGLE_APPLICATION_CREDENTIALS` to point it at a key file.

## Data types

### Importing

`datacontract import bigquery` maps BigQuery types as follows; the original BigQuery type is kept as `physicalType`. An unknown type fails the import with an error.

| BigQuery type | `logicalType` |
|---|---|
| `STRING` | `string` (`maxLength` if declared) |
| `BYTES` | `array` |
| `INTEGER`, `INT64` | `integer` |
| `FLOAT`, `FLOAT64` | `number` |
| `NUMERIC`, `BIGNUMERIC` | `number` (precision/scale as custom properties) |
| `BOOLEAN`, `BOOL` | `boolean` |
| `TIMESTAMP`, `DATETIME` | `timestamp` |
| `DATE` | `date` |
| `TIME` | `time` |
| `GEOGRAPHY`, `JSON` | `object` |
| `INTERVAL` | `string` |
| `RECORD`, `STRUCT` | `object` with nested `properties` |
| `RANGE` | `array` with typed `items` |
| any type with mode `REPEATED` | `array` with typed `items` |

Fields with mode `REQUIRED` become `required: true`. Tables, external tables, and snapshots become schema `physicalType: table`; views and materialized views become `view`.

### Testing

BigQuery supports **native type introspection**: the declared `physicalType` is checked against the actual column type from the BigQuery API. Timezone variants of timestamps are interchangeable; parameters (precision, length) are only enforced when the contract declares them. If the `physicalType` cannot be interpreted as a BigQuery type, the check falls back to the logical type category comparison.

{/* AUTOGENERATED TYPE MAPPING: do not edit by hand; regenerate with update_reference_types.py */}

### Logical type mapping

When no `physicalType` is declared, the CLI derives the native type from the `logicalType` — for example in `datacontract export sql` and the dbt exports. This table is generated from the converter in the CLI's code:

| `logicalType` | BigQuery type |
|---|---|
| `string` | `STRING` |
| `integer` | `INTEGER` |
| `number` | `NUMERIC` |
| `boolean` | `BOOL` |
| `date` | `DATE` |
| `timestamp` | `TIMESTAMP` |
| `time` | `TIME` |
| `object` | `STRUCT<>` |
| `array` | `ARRAY<STRING>` |

{/* END AUTOGENERATED TYPE MAPPING */}

---

## Databricks Reference


Authentication options and data type handling for [Databricks connections](../testing/databricks.md).

## Server

```yaml
servers:
  - server: production
    type: databricks
    catalog: acme_catalog_prod
    schema: orders_latest
```

## Authentication

| Variable | Example | Description |
|---|---|---|
| `DATACONTRACT_DATABRICKS_SERVER_HOSTNAME` | `dbc-abcdefgh-1234.cloud.databricks.com` | Host of the SQL warehouse or compute cluster |
| `DATACONTRACT_DATABRICKS_HTTP_PATH` | `/sql/1.0/warehouses/b053a3ff...` | HTTP path to the SQL warehouse or compute cluster |
| `DATACONTRACT_DATABRICKS_TOKEN` | `dapia0000...` | A personal access token (PAT) |
| `DATACONTRACT_DATABRICKS_CLIENT_ID` | `00000000-...` | Service principal client ID for OAuth M2M auth |
| `DATACONTRACT_DATABRICKS_CLIENT_SECRET` | `dose0000...` | Service principal OAuth secret (used with the client ID) |
| `DATACONTRACT_DATABRICKS_PROFILE` | `my-profile` | A profile from `~/.databrickscfg` (Databricks SDK unified auth) |
| `DATACONTRACT_DATABRICKS_AUTH_TYPE` | `databricks-oauth` | Explicit connector auth type, e.g. for interactive U2M browser login |

The authentication method is selected from the variables you set, in this order: PAT → OAuth service principal (`CLIENT_ID` + `CLIENT_SECRET`) → config profile → explicit `AUTH_TYPE`. `catalog` and `schema` come from the contract's `servers` block.

`datacontract import unity` needs only the hostname plus a PAT or profile; `datacontract test` additionally needs `DATACONTRACT_DATABRICKS_HTTP_PATH`. When a `spark` session is passed programmatically (notebook/pipeline), no credentials are needed at all.

## Data types

### Importing

`datacontract import unity` keeps the full Unity Catalog type text as `physicalType` (e.g. `decimal(10,2)`, `struct<x:int>`, `map<string,int>`) — also duplicated in the `databricksType` custom property — and maps to `logicalType`:

| Databricks type | `logicalType` |
|---|---|
| `STRING`, `VARCHAR`, `CHAR` | `string` |
| `INT`, `BIGINT`, `SMALLINT`, `TINYINT` | `integer` |
| `FLOAT`, `DOUBLE`, `DECIMAL` | `number` |
| `BOOLEAN` | `boolean` |
| `DATE` | `date` |
| `TIMESTAMP` | `timestamp` |
| `BINARY` | `string` (format `binary`) |
| `ARRAY<...>` | `array` (nested `items` when pyspark is available) |
| `STRUCT<...>` | `object` (nested `properties` when pyspark is available) |
| `MAP<...>`, `INTERVAL`, `VARIANT` | *(unset)* — `physicalType` is still written |

`datacontract import spark` maps Spark session types the same way, with two caveats: `TimestampType` and `TimestampNTZType` are currently imported as `date`, and `BinaryType` as `array`.

### Testing

Databricks supports **native type introspection**: the declared `physicalType` is checked against the actual `information_schema` column type. Timezone variants of timestamps are interchangeable; parameters are only enforced when declared. A `physicalType` that isn't valid Databricks SQL falls back to the logical type category comparison. When testing a Spark DataFrame (`type: dataframe`), only the logical type category is checked.

{/* AUTOGENERATED TYPE MAPPING: do not edit by hand; regenerate with update_reference_types.py */}

### Logical type mapping

When no `physicalType` is declared, the CLI derives the native type from the `logicalType` — for example in `datacontract export sql` and the dbt exports. This table is generated from the converter in the CLI's code:

| `logicalType` | Databricks type |
|---|---|
| `string` | `STRING` |
| `integer` | `INT` |
| `number` | `DECIMAL(38,0)` |
| `boolean` | `BOOLEAN` |
| `date` | `DATE` |
| `timestamp` | `TIMESTAMP` |
| `time` | `STRING` |
| `object` | `STRUCT<>` |
| `array` | `ARRAY<STRING>` |

{/* END AUTOGENERATED TYPE MAPPING */}

---

## Spark DataFrame Reference


Data type handling for [Spark DataFrame connections](../testing/dataframe.md). No environment variables are needed — the existing Spark session is passed programmatically.

## Data types

### Importing

`datacontract import spark --tables <name>` maps Spark types as follows; `physicalType` is the Spark simple string (e.g. `bigint`, `decimal(10,2)`, `array<int>`).

| Spark type | `logicalType` |
|---|---|
| `StringType`, `VarcharType` | `string` |
| `IntegerType`, `ShortType`, `LongType` | `integer` |
| `FloatType`, `DoubleType`, `DecimalType` | `number` |
| `BooleanType` | `boolean` |
| `DateType` | `date` |
| `TimestampType`, `TimestampNTZType` | `date` |
| `StructType` | `object` with nested `properties` |
| `ArrayType` | `array` with typed `items` |
| `MapType`, `VariantType` | `object` |
| `BinaryType` | `array` |

### Testing

DataFrames are registered as temporary views named after the contract's schema objects. There is no catalog to introspect, so the `logicalType` is checked by category (`Check that field x has type y`) against the DataFrame's Spark types, with `integer` and `number` treated as compatible.

{/* AUTOGENERATED TYPE MAPPING: do not edit by hand; regenerate with update_reference_types.py */}

### Logical type mapping

When no `physicalType` is declared, the CLI derives the native type from the `logicalType` — for example in `datacontract export sql` and the dbt exports. This table is generated from the converter in the CLI's code:

| `logicalType` | Spark type |
|---|---|
| `string` | `STRING` |
| `integer` | `INT` |
| `number` | `DECIMAL(38,0)` |
| `boolean` | `BOOLEAN` |
| `date` | `DATE` |
| `timestamp` | `TIMESTAMP` |
| `time` | `STRING` |
| `object` | `STRUCT<>` |
| `array` | `ARRAY<STRING>` |

{/* END AUTOGENERATED TYPE MAPPING */}

---

## Google Cloud Storage Reference


Authentication options and data type handling for [GCS connections](../testing/gcs.md). GCS is accessed through the S3 interoperability layer (`type: s3` with `endpointUrl: https://storage.googleapis.com`).

## Server

```yaml
servers:
  - server: production
    type: gcs
    location: s3://my-bucket/orders/*.json # duckdb reads GCS over the S3-compatible endpoint
    format: json
    delimiter: new_line # new_line, array, or none
```

## Authentication

| Variable | Example | Description |
|---|---|---|
| `DATACONTRACT_S3_ACCESS_KEY_ID` | `GOOG1EZZZ...` | The GCS [HMAC Key](https://cloud.google.com/storage/docs/authentication/hmackeys) ID |
| `DATACONTRACT_S3_SECRET_ACCESS_KEY` | `PDWWpb...` | The GCS [HMAC Key](https://cloud.google.com/storage/docs/authentication/hmackeys) secret |

The `location` in the `servers` block must use the `s3://` scheme (not `gs://`).

## Data types

Same as [Amazon S3](./s3.md#data-types): CSV and JSON files are read as the contract's types (no type checks; JSON additionally validated against a JSON Schema derived from the contract), Parquet and Delta column types are checked against the contract's `logicalType` by category. `physicalType` is never checked against file sources.

{/* AUTOGENERATED TYPE MAPPING: do not edit by hand; regenerate with update_reference_types.py */}

### Logical type mapping

When no `physicalType` is declared, the CLI derives the DuckDB column type from the `logicalType` when reading files. This table is generated from the converters in the CLI's code:

| `logicalType` | CSV read type | Parquet read type |
|---|---|---|
| `string` | `VARCHAR` | `VARCHAR` |
| `integer` | `BIGINT` | `INTEGER` |
| `number` | `DOUBLE` | `DECIMAL` |
| `boolean` | `BOOLEAN` | `BOOLEAN` |
| `date` | `DATE` | `DATE` |
| `timestamp` | `TIMESTAMP` | `TIMESTAMP WITH TIME ZONE` |
| `time` | `TIME` | `TIME` |
| `object` | `VARCHAR` | `STRUCT()` |
| `array` | `VARCHAR` | `VARCHAR[]` |

{/* END AUTOGENERATED TYPE MAPPING */}

---

## Apache Impala Reference


Authentication options and data type handling for [Impala connections](../testing/impala.md).

## Server

```yaml
servers:
  - server: production
    type: impala
    host: my-impala-host
    port: 443
    database: my_database # optional default database
```

## Authentication

| Variable | Example | Description |
|---|---|---|
| `DATACONTRACT_IMPALA_USERNAME` | `analytics_user` | Username |
| `DATACONTRACT_IMPALA_PASSWORD` | `mysecretpassword` | Password |
| `DATACONTRACT_IMPALA_USE_SSL` | `true` | Whether to use SSL (defaults to true) |
| `DATACONTRACT_IMPALA_AUTH_MECHANISM` | `LDAP` | Authentication mechanism (defaults to LDAP) |
| `DATACONTRACT_IMPALA_USE_HTTP_TRANSPORT` | `true` | Whether to use HTTP transport (defaults to true) |
| `DATACONTRACT_IMPALA_HTTP_PATH` | `cliservice` | HTTP path for the Impala service (defaults to cliservice) |

`host`, `port`, and `database` come from the contract's `servers` block.

## Data types

### Importing

There is no direct Impala importer. Import a `SHOW CREATE TABLE` DDL with `datacontract import sql --dialect spark` (Impala DDL is Hive-compatible): `STRING`/`VARCHAR`/`CHAR` → `string`, `INT`/`BIGINT`/`SMALLINT`/`TINYINT` → `integer`, `FLOAT`/`DOUBLE`/`DECIMAL` → `number`, `BOOLEAN` → `boolean`, `DATE` → `date`, `TIMESTAMP` → `timestamp`, `ARRAY<...>` → `array`, `STRUCT<...>` → `object`, `BINARY` → `string` (format `binary`), `MAP<...>` → no logical type.

### Testing

Impala does **not** support native type introspection — the declared `physicalType` is not compared against the catalog. Instead, the `logicalType` is checked by category (`Check that field x has type y`): the actual column type is normalized to one of the nine ODCS categories and compared, with `integer` and `number` treated as compatible.

---

## Kafka Reference


Authentication options and data type handling for [Kafka connections](../testing/kafka.md).

## Server

```yaml
servers:
  - server: production
    type: kafka
    host: abc-12345.eu-central-1.aws.confluent.cloud:9092
    topic: orders
    format: json # or avro
```

## Authentication

| Variable | Example | Description |
|---|---|---|
| `DATACONTRACT_KAFKA_SASL_USERNAME` | `xxx` | The SASL username (key) |
| `DATACONTRACT_KAFKA_SASL_PASSWORD` | `xxx` | The SASL password (secret) |
| `DATACONTRACT_KAFKA_SASL_MECHANISM` | `PLAIN` | Default `PLAIN`; also `SCRAM-SHA-256`, `SCRAM-SHA-512` |

If no username/password is set, the CLI connects without authentication (e.g. a local broker). `host`, `topic`, and `format` come from the contract's `servers` block. Kafka checks run on Spark and require a JDK (17 or 21).

## Data types

### Importing

For Avro-encoded topics, import the schema with `datacontract import avro`. The Avro type is kept as `physicalType`:

| Avro type | `logicalType` |
|---|---|
| `string` | `string` |
| `int`, `long` | `integer` |
| `float`, `double` | `number` |
| `boolean` | `boolean` |
| `record` | `object` with nested `properties` |
| `array` | `array` |
| `map` | `object` (values not expanded) |
| `enum` | `string` (symbols in the `avroSymbols` custom property) |
| `bytes`, `fixed` | `array` |

Avro logical type annotations take precedence: `decimal` → `number` (with precision/scale), `date` → `date`, `uuid`/`duration`/`time-millis`/`time-micros` → `string`. Unions must be `[null, T]` and make the field optional.

### Testing

Messages are read via Spark. For `format: json` and `format: avro`, no type checks are generated — messages are decoded as the contract's types, and violations surface as decode errors or value-check failures from `logicalTypeOptions`.

---

## Local Files Reference


Data type handling for [local file connections](../testing/local.md). No environment variables are needed.

## Server

```yaml
servers:
  - server: local
    type: local
    path: ./*.parquet # glob patterns and a {model} placeholder are supported
    format: parquet   # parquet, json, csv, or delta
```

No environment variables are needed for local files. Data type inference and per-format type handling: **[Local Files Reference](../reference/local.md)**.

## Data types

### Importing

`datacontract import csv` samples the file with DuckDB and infers per column:

| Inferred DuckDB type | `logicalType` |
|---|---|
| `BOOLEAN` | `boolean` |
| `INTEGER`, `BIGINT` | `integer` |
| `DOUBLE` | `number` |
| `VARCHAR` | `string` |

It also profiles the data: `required` (no nulls), `unique` (all distinct), `examples`, and `minimum`/`maximum` for numeric columns. Values that all match an email or UUID pattern get `logicalTypeOptions.format`.

`datacontract import json` inspects the first 20 records: booleans, integers, and floats map to `boolean`/`integer`/`number`; strings map to `string` (with a detected `format` of `date`, `date-time`, `email`, or `uuid` in `logicalTypeOptions`); objects and arrays map to `object`/`array` with nested properties. When records disagree, the wider type wins (integer + float → `number`; anything + string → `string`).

`datacontract import parquet` reads types from the Parquet metadata.

### Testing

Type handling depends on the `format` in the `servers` block:

| `format` | How types are handled |
|---|---|
| `csv` | The file is read **as the contract's types** — no type checks are generated; a value that can't be coerced surfaces as a read error. |
| `json` | Same as CSV, plus every record is validated against a JSON Schema derived from the contract's `logicalType`s (with `format` options like `date-time`, `uuid`). |
| `parquet` | Column types come from the Parquet file; the contract's `logicalType` is checked by category (`Check that field x has type y`). |
| `delta` | Column types come from the Delta table; the contract's `logicalType` is checked by category. |

`physicalType` is never checked against file sources — declare `logicalType` (and `logicalTypeOptions` for value constraints). The `path` supports glob patterns and a `{model}` placeholder.

{/* AUTOGENERATED TYPE MAPPING: do not edit by hand; regenerate with update_reference_types.py */}

### Logical type mapping

When no `physicalType` is declared, the CLI derives the DuckDB column type from the `logicalType` when reading files. This table is generated from the converters in the CLI's code:

| `logicalType` | CSV read type | Parquet read type |
|---|---|---|
| `string` | `VARCHAR` | `VARCHAR` |
| `integer` | `BIGINT` | `INTEGER` |
| `number` | `DOUBLE` | `DECIMAL` |
| `boolean` | `BOOLEAN` | `BOOLEAN` |
| `date` | `DATE` | `DATE` |
| `timestamp` | `TIMESTAMP` | `TIMESTAMP WITH TIME ZONE` |
| `time` | `TIME` | `TIME` |
| `object` | `VARCHAR` | `STRUCT()` |
| `array` | `VARCHAR` | `VARCHAR[]` |

{/* END AUTOGENERATED TYPE MAPPING */}

---

## MySQL Reference


Authentication options and data type handling for [MySQL connections](../testing/mysql.md).

## Server

```yaml
servers:
  - server: mysql
    type: mysql
    host: localhost
    port: 3306
    database: mydb
```

## Authentication

| Variable | Example | Description |
|---|---|---|
| `DATACONTRACT_MYSQL_USERNAME` | `root` | Username |
| `DATACONTRACT_MYSQL_PASSWORD` | `mysecretpassword` | Password |

`host`, `port` (default 3306), and `database` come from the contract's `servers` block.

## Data types

### Importing

`datacontract import sql --dialect mysql` maps DDL types as follows; the normalized SQL type is kept as `physicalType`.

| MySQL type | `logicalType` |
|---|---|
| `VARCHAR`, `CHAR`, `TEXT` | `string` (`maxLength` for varchar/char) |
| `INT`, `INTEGER`, `BIGINT`, `SMALLINT` | `integer` |
| `NUMERIC`, `DECIMAL`, `FLOAT`, `DOUBLE` | `number` |
| `BOOLEAN` | `boolean` |
| `DATE` | `date` |
| `TIME` | `time` |
| `DATETIME`, `TIMESTAMP` | `timestamp` |
| `BLOB`, `BINARY`, `VARBINARY` | `string` (format `binary`) |
| `JSON` | `object` |
| `ENUM(...)`, `YEAR`, `TINYINT(1)`, `INT UNSIGNED` | *(unset)* — `physicalType` is still written |

### Testing

MySQL is reached through DuckDB's MySQL scanner and does **not** support native type introspection — the declared `physicalType` is not compared against the catalog. Instead, the `logicalType` is checked by category (`Check that field x has type y`), with `integer` and `number` treated as compatible.

{/* AUTOGENERATED TYPE MAPPING: do not edit by hand; regenerate with update_reference_types.py */}

### Logical type mapping

When no `physicalType` is declared, the CLI derives the native type from the `logicalType` — for example in `datacontract export sql` and the dbt exports. This table is generated from the converter in the CLI's code:

| `logicalType` | MySQL type |
|---|---|
| `string` | `varchar(255)` |
| `integer` | `int` |
| `number` | `decimal` |
| `boolean` | `boolean` |
| `date` | `date` |
| `timestamp` | `timestamp` |
| `time` | `time` |
| `object` | `json` |
| `array` | `json` |

{/* END AUTOGENERATED TYPE MAPPING */}

---

## Oracle Reference


Authentication options and data type handling for [Oracle connections](../testing/oracle.md).

## Server

```yaml
servers:
  - server: oracle
    type: oracle
    host: localhost
    port: 1521
    serviceName: ORCL
    schema: ADMIN
```

## Authentication

| Variable | Example | Description |
|---|---|---|
| `DATACONTRACT_ORACLE_USERNAME` | `system` | Username |
| `DATACONTRACT_ORACLE_PASSWORD` | `0x162e53` | Password |
| `DATACONTRACT_ORACLE_CLIENT_DIR` | `C:\oracle\client` | Path to an Oracle Instant Client installation (required for thick mode) |

`host`, `port` (default 1521), `service_name`, and `schema` come from the contract's `servers` block.

## Data types

### Importing

`datacontract import sql --dialect oracle` maps DDL types as follows; the normalized SQL type is kept as `physicalType`.

| Oracle type | `logicalType` |
|---|---|
| `VARCHAR2`, `NVARCHAR2`, `CHAR`, `NCHAR` | `string` (`maxLength` for varchar/char) |
| `CLOB`, `NCLOB` | `string` |
| `NUMBER` | `number` (precision/scale as custom properties) |
| `BINARY_FLOAT`, `BINARY_DOUBLE`, `FLOAT` | `number` |
| `DATE` | `date` |
| `TIMESTAMP`, `TIMESTAMP WITH (LOCAL) TIME ZONE` | `timestamp` |
| `RAW`, `BLOB`, `BFILE` | `string` (format `binary`) |
| `XMLTYPE`, `ROWID`, `INTERVAL` types | *(unset)* — `physicalType` is still written |

### Testing

Oracle supports **native type introspection**: the declared `physicalType` is checked against `ALL_TAB_COLUMNS`. Length is reconstructed for character and `RAW` types, precision/scale for `NUMBER`; parameters are only enforced when the contract declares them. Types that sqlglot can't parse (`ROWID`, `INTERVAL`) are compared by name. A `physicalType` that can't be interpreted falls back to the logical type category comparison.

{/* AUTOGENERATED TYPE MAPPING: do not edit by hand; regenerate with update_reference_types.py */}

### Logical type mapping

When no `physicalType` is declared, the CLI derives the native type from the `logicalType` — for example in `datacontract export sql` and the dbt exports. This table is generated from the converter in the CLI's code:

| `logicalType` | Oracle type |
|---|---|
| `string` | `NVARCHAR2(2000)` |
| `integer` | `NUMBER` |
| `number` | `NUMBER` |
| `boolean` | `CHAR(1)` |
| `date` | `DATE` |
| `timestamp` | `TIMESTAMP(6) WITH TIME ZONE` |
| `time` | `DATE` |
| `object` | `CLOB` |
| `array` | `CLOB` |

{/* END AUTOGENERATED TYPE MAPPING */}

---

## Postgres Reference


Authentication options and data type handling for [Postgres connections](../testing/postgres.md).

## Server

```yaml
servers:
  - server: postgres
    type: postgres
    host: localhost
    port: 5432
    database: postgres
    schema: public
```

## Authentication

| Variable | Example | Description |
|---|---|---|
| `DATACONTRACT_POSTGRES_USERNAME` | `postgres` | Username |
| `DATACONTRACT_POSTGRES_PASSWORD` | `mysecretpassword` | Password |

`host`, `port` (default 5432), `database`, and `schema` come from the contract's `servers` block — or, for `datacontract import postgres`, from `--source`, `--port`, `--database`, and `--schema`.

## Data types

### Importing

`datacontract import postgres` reads the declared type from `information_schema.columns` and keeps it as `physicalType` exactly as the test path reads it back (`character varying(36)`, `numeric(10,2)`, `timestamp with time zone`), so an imported contract passes `datacontract test` without hand-editing.

`datacontract import sql --dialect postgres` maps DDL types as follows; the normalized SQL type is kept as `physicalType`.

| Postgres type | `logicalType` |
|---|---|
| `VARCHAR`, `CHAR`, `TEXT` | `string` (`maxLength` for varchar/char) |
| `SMALLINT`, `INT`, `INTEGER`, `BIGINT` | `integer` |
| `NUMERIC`, `DECIMAL`, `REAL`, `DOUBLE PRECISION`, `MONEY` | `number` |
| `BOOLEAN` | `boolean` |
| `DATE` | `date` |
| `TIME` | `time` |
| `TIMESTAMP`, `TIMESTAMPTZ` | `timestamp` |
| `BYTEA` | `string` (format `binary`) |
| `JSON` | `object` |
| `XML` | `string` |
| `UUID`, `SERIAL`, `BIGSERIAL`, `JSONB`, `INTERVAL`, `INET`, array types (`INT[]`) | *(unset)* — `physicalType` is still written |

### Testing

Postgres supports **native type introspection**: the declared `physicalType` is checked against `information_schema.columns`. Note that on Postgres `text` and `varchar` are **distinct types** — declare the one the column actually has. `decimal` and `numeric` are interchangeable, as are timezone variants of timestamps; length/precision is only enforced when declared (`varchar` matches `varchar(255)`, `varchar(255)` does not match `varchar(100)`). A `physicalType` that isn't valid Postgres SQL falls back to the logical type category comparison.

The same applies to Postgres-compatible databases tested with `type: postgres` (e.g. RisingWave).

{/* AUTOGENERATED TYPE MAPPING: do not edit by hand; regenerate with update_reference_types.py */}

### Logical type mapping

When no `physicalType` is declared, the CLI derives the native type from the `logicalType` — for example in `datacontract export sql` and the dbt exports. This table is generated from the converter in the CLI's code:

| `logicalType` | Postgres type |
|---|---|
| `string` | `text` |
| `integer` | `integer` |
| `number` | `numeric` |
| `boolean` | `boolean` |
| `date` | `date` |
| `timestamp` | `timestamptz` |
| `time` | `time` |
| `object` | `jsonb` |
| `array` | `text[]` |

{/* END AUTOGENERATED TYPE MAPPING */}

---

## Amazon Redshift Reference


Authentication options and data type handling for [Redshift connections](../testing/redshift.md).

## Server

```yaml
servers:
  - server: redshift
    type: redshift
    host: my-workgroup.123456789012.us-east-1.redshift-serverless.amazonaws.com
    port: 5439
    database: dev
    schema: analytics
```

## Authentication

`datacontract test` and `datacontract import redshift` authenticate identically. `host`, `port` (default 5439), `database`, and `schema` come from the contract's `servers` block; for the import they are passed as `--source`, `--port`, `--database`, and `--schema`.

| Variable | Example | Description |
|---|---|---|
| `DATACONTRACT_REDSHIFT_AUTHENTICATION` | `iam` | `password` or `iam`; only needed to override what is inferred |
| `DATACONTRACT_REDSHIFT_USERNAME` | `awsuser` | Database user. Required for `password`; in `iam` mode it selects the legacy API (see below) |
| `DATACONTRACT_REDSHIFT_PASSWORD` | `mysecretpassword` | Password (`password` mode only) |
| `DATACONTRACT_REDSHIFT_SSLMODE` | `verify-full` | TLS mode passed to the driver. Defaults to `require` in `iam` mode, and to the driver's own default (`prefer`) otherwise |

### IAM authentication

The method is inferred: setting `DATACONTRACT_REDSHIFT_PASSWORD` selects a database login, otherwise resolvable AWS credentials select IAM. Set `DATACONTRACT_REDSHIFT_AUTHENTICATION` explicitly only to override that. With IAM, the CLI asks AWS for temporary database credentials and uses them to log in — no database password anywhere. The AWS credentials themselves come from the same variables Athena uses (`DATACONTRACT_S3_ACCESS_KEY_ID`, `DATACONTRACT_S3_SECRET_ACCESS_KEY`, `DATACONTRACT_S3_SESSION_TOKEN`, `DATACONTRACT_S3_REGION`), and fall back to the standard AWS chain: `aws sso login`, `AWS_PROFILE`, EC2/ECS/EKS instance roles, or GitHub OIDC in CI.

Which AWS API is called depends on the endpoint and whether a username is set:

| Endpoint | Username set | API |
|---|---|---|
| Serverless | — | `redshift-serverless:GetCredentials` |
| Provisioned | no | `redshift:GetClusterCredentialsWithIAM` — the database user is derived from your IAM identity |
| Provisioned | yes | `redshift:GetClusterCredentials` — credentials are requested for that database user |

The workgroup or cluster identifier and the region are derived from the endpoint host in the `servers` block, so a standard endpoint needs no extra configuration. Custom domains and VPC endpoints don't follow that naming, so they need an override:

| Variable | Example | Description |
|---|---|---|
| `DATACONTRACT_REDSHIFT_WORKGROUP` | `my-workgroup` | Serverless workgroup name |
| `DATACONTRACT_REDSHIFT_CLUSTER_IDENTIFIER` | `my-cluster` | Provisioned cluster identifier |
| `DATACONTRACT_REDSHIFT_REGION` | `eu-central-1` | AWS region (otherwise `DATACONTRACT_S3_REGION`, then the host) |
| `DATACONTRACT_REDSHIFT_DURATION_SECONDS` | `3600` | Lifetime of the temporary credentials (900–3600) |
| `DATACONTRACT_REDSHIFT_AUTO_CREATE` | `true` | Create the database user if it doesn't exist (`GetClusterCredentials` only) |
| `DATACONTRACT_REDSHIFT_DB_GROUPS` | `readers,analysts` | Database groups the user joins for the session (`GetClusterCredentials` only) |

The AWS identity needs permission to call the API from the table above — `redshift-serverless:GetCredentials` on the workgroup, or `redshift:GetClusterCredentialsWithIAM` / `redshift:GetClusterCredentials` on the `dbname:`/`dbuser:` resources (plus `redshift:JoinGroup` when using `DB_GROUPS`). Credentials are minted per connection and expire, which is why a long-running process may need to reconnect.

## Data types

### Importing

`datacontract import redshift` reads the declared types from the `SVV_COLUMNS` catalog view and writes them as `physicalType` in the catalog's own spelling, including length and precision (`character varying(36)`, `numeric(10,2)`). That is exactly what the physical type check reads back during a test, so an imported contract passes on the first run. The `logicalType` is derived with the same mapping as the SQL import below.

`datacontract import sql --dialect redshift` maps DDL types like the [Postgres dialect](./postgres.md#importing): `VARCHAR`/`CHAR`/`TEXT` → `string`, integer types → `integer`, `NUMERIC`/`DECIMAL`/`REAL`/`DOUBLE PRECISION` → `number`, `BOOLEAN` → `boolean`, `DATE` → `date`, `TIME` → `time`, `TIMESTAMP`/`TIMESTAMP WITH TIME ZONE` → `timestamp`. Redshift-specific types without a portable equivalent (`SUPER`, `GEOMETRY`) get no `logicalType`; the `physicalType` is still written.

### Testing

Redshift supports **native type introspection**: the declared `physicalType` is checked against `information_schema.columns`. `decimal` and `numeric` are interchangeable, as are timezone variants of timestamps; length/precision is only enforced when declared. A `physicalType` that can't be interpreted falls back to the logical type category comparison.

{/* AUTOGENERATED TYPE MAPPING: do not edit by hand; regenerate with update_reference_types.py */}

### Logical type mapping

When no `physicalType` is declared, the CLI derives the native type from the `logicalType` — for example in `datacontract export sql` and the dbt exports. This table is generated from the converter in the CLI's code:

| `logicalType` | Redshift type |
|---|---|
| `string` | `text` |
| `integer` | `integer` |
| `number` | `numeric` |
| `boolean` | `boolean` |
| `date` | `date` |
| `timestamp` | `timestamptz` |
| `time` | `time` |
| `object` | `jsonb` |
| `array` | `text[]` |

{/* END AUTOGENERATED TYPE MAPPING */}

---

## Amazon S3 Reference


Authentication options and data type handling for [S3 connections](../testing/s3.md).

## Server

### JSON

```yaml
servers:
  - server: production
    type: s3
    endpointUrl: https://minio.example.com # not needed with AWS S3
    location: s3://bucket-name/path/*/*.json
    format: json
    delimiter: new_line # new_line, array, or none
```

### Delta

```yaml
servers:
  - server: production
    type: s3
    endpointUrl: https://minio.example.com # not needed with AWS S3
    location: s3://bucket-name/path/table.delta # folder with parquet files and _delta_log
    format: delta
```

## Authentication

An existing AWS session is used when no key is set — `aws sso login`, `AWS_PROFILE`, EC2/ECS/EKS instance roles, or GitHub OIDC in CI. With nothing resolvable the objects are read unsigned, which is what public buckets need.

The variables below override that:

| Variable | Example | Description |
|---|---|---|
| `DATACONTRACT_S3_REGION` | `eu-central-1` | Region of the S3 bucket |
| `DATACONTRACT_S3_ACCESS_KEY_ID` | `AKIAXV5Q5Q...` | AWS Access Key ID |
| `DATACONTRACT_S3_SECRET_ACCESS_KEY` | `93S7LRrJ...` | AWS Secret Access Key |
| `DATACONTRACT_S3_SESSION_TOKEN` | `AQoDYXdzEJr...` | AWS temporary session token (optional) |

If `DATACONTRACT_S3_ACCESS_KEY_ID` is not set, no credentials are configured — public buckets and ambient credentials still work. For S3-compatible storage (MinIO), set `endpointUrl` in the `servers` block; the CLI then uses path-style addressing.

## Data types

Files on S3 are read with DuckDB. Type handling depends on the `format` in the `servers` block:

| `format` | How types are handled |
|---|---|
| `csv` | The file is read **as the contract's types** — no type checks are generated; a value that can't be coerced surfaces as a read error. |
| `json` | Same as CSV, plus every record is validated against a JSON Schema derived from the contract's `logicalType`s (with `format` options like `date-time`, `uuid`). |
| `parquet` | Column types come from the Parquet file; the contract's `logicalType` is checked by category (`Check that field x has type y`). |
| `delta` | Column types come from the Delta table; the contract's `logicalType` is checked by category. |

`physicalType` is never checked against file sources — declare `logicalType` (and `logicalTypeOptions` for value constraints). See [Local files](./local.md) for how CSV/JSON imports infer types.

{/* AUTOGENERATED TYPE MAPPING: do not edit by hand; regenerate with update_reference_types.py */}

### Logical type mapping

When no `physicalType` is declared, the CLI derives the DuckDB column type from the `logicalType` when reading files. This table is generated from the converters in the CLI's code:

| `logicalType` | CSV read type | Parquet read type |
|---|---|---|
| `string` | `VARCHAR` | `VARCHAR` |
| `integer` | `BIGINT` | `INTEGER` |
| `number` | `DOUBLE` | `DECIMAL` |
| `boolean` | `BOOLEAN` | `BOOLEAN` |
| `date` | `DATE` | `DATE` |
| `timestamp` | `TIMESTAMP` | `TIMESTAMP WITH TIME ZONE` |
| `time` | `TIME` | `TIME` |
| `object` | `VARCHAR` | `STRUCT()` |
| `array` | `VARCHAR` | `VARCHAR[]` |

{/* END AUTOGENERATED TYPE MAPPING */}

---

## Snowflake Reference


Authentication options and data type handling for [Snowflake connections](../testing/snowflake.md).

## Server

```yaml
servers:
  - server: snowflake
    type: snowflake
    account: abcdefg-xn12345
    database: ORDER_DB
    schema: ORDERS_PII_V2
```

## Authentication

Any `DATACONTRACT_SNOWFLAKE_`-prefixed variable is passed (lowercased, prefix stripped) as a connection parameter to the [snowflake-connector-python](https://docs.snowflake.com/en/developer-guide/python-connector/python-connector-api#connect) driver. Set the variables required by your workspace's `authenticator` mode.

| Connection parameter | Environment variable |
|---|---|
| `username` | `DATACONTRACT_SNOWFLAKE_USERNAME` |
| `password` | `DATACONTRACT_SNOWFLAKE_PASSWORD` |
| `warehouse` | `DATACONTRACT_SNOWFLAKE_WAREHOUSE` |
| `role` | `DATACONTRACT_SNOWFLAKE_ROLE` |
| `connection_timeout` | `DATACONTRACT_SNOWFLAKE_CONNECTION_TIMEOUT` |
| `authenticator` | `DATACONTRACT_SNOWFLAKE_AUTHENTICATOR` |
| `private_key` | `DATACONTRACT_SNOWFLAKE_PRIVATE_KEY` |
| `private_key_passphrase` | `DATACONTRACT_SNOWFLAKE_PRIVATE_KEY_PASSPHRASE` |
| `private_key_path` | `DATACONTRACT_SNOWFLAKE_PRIVATE_KEY_PATH` |

`account`, `database`, and `schema` come from the contract's `servers` block.

### Import-specific options

`datacontract import snowflake` additionally supports:

| Variable | Description |
|---|---|
| `DATACONTRACT_SNOWFLAKE_PRIVATE_KEY_FILE` | Path to a private key file (key-pair auth) |
| `DATACONTRACT_SNOWFLAKE_PRIVATE_KEY_FILE_PWD` | Passphrase for the private key file |
| `DATACONTRACT_SNOWFLAKE_HOME` | Directory containing a [connections.toml](https://docs.snowflake.com/en/developer-guide/python-connector/python-connector-connect#connecting-using-the-connections-toml-file) |
| `DATACONTRACT_SNOWFLAKE_CONNECTIONS_FILE` | Path to a connections.toml file |
| `DATACONTRACT_SNOWFLAKE_DEFAULT_CONNECTION_NAME` | Connection name within connections.toml |

The `SNOWFLAKE_`-prefixed equivalents work as fallbacks. If no password is set, the import falls back to browser-based SSO (`externalbrowser`).

## Data types

### Importing

`datacontract import snowflake` reads `INFORMATION_SCHEMA.COLUMNS` and maps types as follows. The `physicalType` keeps the full native type including length/precision (e.g. `NUMBER(38, 0)`, `TEXT(16777216)`).

| Snowflake type | `logicalType` | Notes |
|---|---|---|
| `TEXT`, `VARCHAR` | `string` | `maxLength` from `CHARACTER_MAXIMUM_LENGTH` |
| `NUMBER` (incl. `INT`/`BIGINT` aliases) | `number` | precision/scale as custom properties |
| `FLOAT`, `DOUBLE` | `number` | |
| `BOOLEAN` | `boolean` | |
| `DATE` | `date` | |
| `TIMESTAMP_NTZ` / `_LTZ` / `_TZ` | `timestamp` | |
| `TIME` | `time` | |
| `BINARY` | `string` (format `binary`) | |
| `ARRAY` | `array` | |
| `VARIANT`, `OBJECT`, `GEOGRAPHY`, `GEOMETRY` | *(unset)* | `physicalType` is still written |

Columns also get `required` (from `IS_NULLABLE`), `unique` (from `IS_IDENTITY`), and custom properties for `ordinalPosition`, `default`, `precision`, `scale`, `characterSet`, and `collation`.

### Testing

Snowflake supports **native type introspection**: the declared `physicalType` is checked against the actual catalog type. Snowflake-specific leniency: `VARCHAR`/`TEXT`/`NVARCHAR` are treated as the same family, as are `DECIMAL`/`INT`/`BIGINT`/`SMALLINT`/`TINYINT` (Snowflake stores all of them as `NUMBER`), and `DOUBLE`/`FLOAT`. Structured `OBJECT(...)`/`ARRAY(...)`/`MAP(...)` columns are introspected via `SHOW COLUMNS` and compared including their nesting.

{/* AUTOGENERATED TYPE MAPPING: do not edit by hand; regenerate with update_reference_types.py */}

### Logical type mapping

When no `physicalType` is declared, the CLI derives the native type from the `logicalType` — for example in `datacontract export sql` and the dbt exports. This table is generated from the converter in the CLI's code:

| `logicalType` | Snowflake type |
|---|---|
| `string` | `STRING` |
| `integer` | `NUMBER` |
| `number` | `NUMBER` |
| `boolean` | `BOOLEAN` |
| `date` | `DATE` |
| `timestamp` | `TIMESTAMP_TZ` |
| `time` | `TIME` |
| `object` | `OBJECT` |
| `array` | `ARRAY` |

{/* END AUTOGENERATED TYPE MAPPING */}

---

## Microsoft SQL Server Reference


Authentication options and data type handling for [SQL Server connections](../testing/sqlserver.md).

## Server

```yaml
servers:
  - server: production
    type: sqlserver
    host: localhost
    port: 1433
    database: mydb
    schema: dbo
    driver: ODBC Driver 18 for SQL Server
```

## Authentication

| Variable | Example | Description |
|---|---|---|
| `DATACONTRACT_SQLSERVER_AUTHENTICATION` | `sql` | `sql` (default), `cli` (uses `az login`), `windows`, `ActiveDirectoryPassword`, `ActiveDirectoryServicePrincipal`, `ActiveDirectoryInteractive` |
| `DATACONTRACT_SQLSERVER_USERNAME` | `root` | Username (for `sql`, `ActiveDirectoryPassword`, `ActiveDirectoryInteractive`) |
| `DATACONTRACT_SQLSERVER_PASSWORD` | `toor` | Password (for `sql` and `ActiveDirectoryPassword`) |
| `DATACONTRACT_SQLSERVER_CLIENT_ID` | `a3cf5d29-...` | Client ID (for `ActiveDirectoryServicePrincipal`) |
| `DATACONTRACT_SQLSERVER_CLIENT_SECRET` | `kX9~Qr2Lm...` | Client secret (for `ActiveDirectoryServicePrincipal`) |
| `DATACONTRACT_SQLSERVER_TRUST_SERVER_CERTIFICATE` | `True` | Trust self-signed certificate |
| `DATACONTRACT_SQLSERVER_ENCRYPTED_CONNECTION` | `True` | Use SSL |
| `DATACONTRACT_SQLSERVER_DRIVER` | `ODBC Driver 18 for SQL Server` | ODBC driver name |

The `cli` mode reuses an `az login` session through the Azure default credential chain and requires ODBC Driver 18.1 or newer. `host`, `port`, `database`, `schema`, and `driver` come from the contract's `servers` block.

## Data types

### Importing

`datacontract import sql --dialect sqlserver` maps DDL types as follows; the normalized SQL type is kept as `physicalType`.

| SQL Server type | `logicalType` |
|---|---|
| `VARCHAR`, `NVARCHAR`, `CHAR`, `NCHAR`, `TEXT`, `NTEXT` | `string` (`maxLength` for varchar/char) |
| `INT`, `INTEGER`, `BIGINT`, `SMALLINT`, `TINYINT` | `integer` |
| `NUMERIC`, `DECIMAL`, `FLOAT`, `REAL`, `MONEY` | `number` |
| `BIT` | `boolean` |
| `DATE` | `date` |
| `TIME` | `time` |
| `DATETIME`, `DATETIME2`, `DATETIMEOFFSET`, `SMALLDATETIME` | `timestamp` |
| `UNIQUEIDENTIFIER` | `string` (format `uuid`) |
| `BINARY`, `VARBINARY`, `IMAGE` | `string` (format `binary`) |
| `XML` | `string` |

### Testing

SQL Server supports **native type introspection**: the declared `physicalType` is checked against `information_schema.columns`. `varchar` and `nvarchar` are distinct types, as are `text` and `varchar` — declare what the column actually is. Timezone/precision variants of the datetime family are interchangeable; length is only enforced when declared (`varchar(max)` is introspected as such). A `physicalType` that isn't valid T-SQL falls back to the logical type category comparison.

{/* AUTOGENERATED TYPE MAPPING: do not edit by hand; regenerate with update_reference_types.py */}

### Logical type mapping

When no `physicalType` is declared, the CLI derives the native type from the `logicalType` — for example in `datacontract export sql` and the dbt exports. This table is generated from the converter in the CLI's code:

| `logicalType` | SQL Server type |
|---|---|
| `string` | `varchar` |
| `integer` | `INT` |
| `number` | `numeric` |
| `boolean` | `bit` |
| `date` | `date` |
| `timestamp` | `datetimeoffset` |
| `time` | `time` |
| `object` | `nvarchar(max)` |
| `array` | *(not supported)* |

{/* END AUTOGENERATED TYPE MAPPING */}

---

## Trino Reference


Authentication options and data type handling for [Trino connections](../testing/trino.md).

## Server

```yaml
servers:
  - server: trino
    type: trino
    host: localhost
    port: 8080
    catalog: my_catalog
    schema: my_schema
```

## Authentication

| Variable | Example | Description |
|---|---|---|
| `DATACONTRACT_TRINO_USERNAME` | `trino` | Username for `basic` auth |
| `DATACONTRACT_TRINO_PASSWORD` | `mysecretpassword` | Password for `basic` auth |
| `DATACONTRACT_TRINO_AUTHENTICATION` | `oauth2` | `basic` (default), `jwt`, or `oauth2` |
| `DATACONTRACT_TRINO_JWT_TOKEN` | `eyJhbGciOi...` | JWT bearer token for `jwt` auth |

`host`, `port`, `catalog`, and `schema` come from the contract's `servers` block.

## Data types

### Importing

There is no direct Trino importer. Import a `SHOW CREATE TABLE` DDL with `datacontract import sql --dialect postgres` (Trino's ANSI-style DDL generally parses with the postgres dialect): `VARCHAR`/`CHAR` → `string`, `INTEGER`/`BIGINT`/`SMALLINT`/`TINYINT` → `integer`, `DECIMAL`/`DOUBLE`/`REAL` → `number`, `BOOLEAN` → `boolean`, `DATE` → `date`, `TIME` → `time`, `TIMESTAMP` → `timestamp`, `ARRAY(...)` → `array`, `JSON` → `object`.

### Testing

Trino supports **native type introspection**: the declared `physicalType` is checked against `information_schema.columns`. Length/precision is only enforced when the contract declares it (`varchar` matches `varchar(10)`); timezone variants of timestamps are interchangeable. A `physicalType` that isn't valid Trino SQL falls back to the logical type category comparison.

{/* AUTOGENERATED TYPE MAPPING: do not edit by hand; regenerate with update_reference_types.py */}

### Logical type mapping

When no `physicalType` is declared, the CLI derives the native type from the `logicalType` — for example in `datacontract export sql` and the dbt exports. This table is generated from the converter in the CLI's code:

| `logicalType` | Trino type |
|---|---|
| `string` | `varchar` |
| `integer` | `integer` |
| `number` | `decimal` |
| `boolean` | `boolean` |
| `date` | `date` |
| `timestamp` | `timestamp(3) with time zone` |
| `time` | `time` |
| `object` | `json` |
| `array` | `json` |

{/* END AUTOGENERATED TYPE MAPPING */}

---

## Commands


The `datacontract` CLI groups its functionality into the following commands. Run `datacontract --help` or `datacontract <command> --help` at any time to see the latest usage.

| Command | Description |
|---|---|
| [`init`](./init.md) | Create an empty data contract from a template. |
| [`edit`](./edit.md) | Edit a contract in the Data Contract Editor (web UI). |
| [`lint`](./lint.md) | Validate that the contract is correctly formatted. |
| [`changelog`](./changelog.md) | Show a changelog between two contracts. |
| [`test`](./test.md) | Run schema and quality tests on configured servers. |
| [`dbt`](./dbt.md) | Generate dbt tests from a contract and run them (`dbt sync`). |
| [`ci`](./ci.md) | Run tests for CI/CD pipelines with annotations and summaries. |
| [`export`](./export.md) | Convert a contract to a target format. |
| [`import`](./import.md) | Create a contract from a source format. |
| [`catalog`](./catalog.md) | Create an HTML catalog of data contracts. |
| [`publish`](./publish.md) | Publish a contract to Entropy Data. |
| [`api`](./api.md) | Start the CLI as a web server with a REST API. |

## Common usage

```bash
datacontract init odcs.yaml
datacontract edit odcs.yaml
datacontract lint odcs.yaml
datacontract changelog v1.odcs.yaml v2.odcs.yaml
datacontract test odcs.yaml
datacontract dbt sync orders.odcs.yaml --project-dir ./warehouse
datacontract export html datacontract.yaml --output odcs.html
datacontract import sql --source my-ddl.sql --dialect postgres --output odcs.yaml
```

Most commands accept a `--debug` flag for verbose logging.

---

## api

# `datacontract api`

Start the CLI as a web server application with a REST API. The OpenAPI documentation (Swagger UI) is available at `http://localhost:4242`. Requires the `api` extra. See the [API](../api.md) guide.

```bash
datacontract api [OPTIONS]
```

| Option | Default | Description |
|---|---|---|
| `--port` | `4242` | Bind the socket to this port. |
| `--host` | `127.0.0.1` | Bind to this host (use `0.0.0.0` for Docker). |
| `--debug` / `--no-debug` | off | Enable debug logging. |

Protect the API by setting `DATACONTRACT_CLI_API_KEY`; requests must then include an `x-api-key` header.

```bash
datacontract api --port 4242 --host 0.0.0.0
```

---

## catalog

# `datacontract catalog`

Create an HTML catalog of data contracts.

```bash
datacontract catalog [OPTIONS]
```

| Option | Default | Description |
|---|---|---|
| `--files` | `*.yaml` | Glob pattern for the contract files to include (applies recursively to subfolders). |
| `--output` | `catalog/` | Output directory for the catalog HTML files. |
| `--json-schema` | — | Location (URL or path) of the ODCS JSON Schema. |
| `--debug` / `--no-debug` | off | Enable debug logging. |

```bash
# Create a catalog in the current folder
datacontract catalog --output "."

# Create a catalog based on a filename convention
datacontract catalog --files "*.odcs.yaml"
```

---

## changelog

# `datacontract changelog`

Show a changelog between two data contracts.

```bash
datacontract changelog V1 V2
```

| Argument | Description |
|---|---|
| `V1` *(required)* | Location (URL or path) of the source (before) contract. |
| `V2` *(required)* | Location (URL or path) of the target (after) contract. |

| Option | Default | Description |
|---|---|---|
| `--inline-references` / `--no-inline-references` | `--inline-references` | Resolve and inline external references. |
| `--debug` / `--no-debug` | `--no-debug` | Enable debug logging. |

```bash
datacontract changelog v1.odcs.yaml v2.odcs.yaml
```

---

## ci

# `datacontract ci`

Run tests for CI/CD pipelines. Wraps [`test`](./test.md) with CI-specific features: GitHub Actions annotations and step summary, Azure DevOps annotations, machine-readable output, and exit-code control. See [Scheduling and CI/CD](../ci-cd.md) for full pipeline examples.

```bash
datacontract ci [LOCATIONS]...
```

| Argument | Description |
|---|---|
| `LOCATIONS...` | One or more locations (URL or path) of contract files. Supports globs, e.g. `contracts/*.yaml`. |

| Option | Default | Description |
|---|---|---|
| `--server` | `all` | Which server to test, or `all`. |
| `--publish` | — | URL to publish the results to. |
| `--output` | — | File path to write results to. |
| `--output-format` | — | `json` or `junit`. |
| `--json` | — | Print test results as JSON to stdout. |
| `--fail-on` | `error` | Minimum severity that causes a non-zero exit: `warning`, `error`, or `never`. |
| `--logs` / `--no-logs` | `--no-logs` | Print logs. |
| `--ssl-verification` / `--no-...` | on | SSL verification when publishing. |
| `--inline-references` / `--no-...` | on | Resolve and inline external references. |
| `--debug` / `--no-debug` | off | Enable debug logging. |

```bash
datacontract ci datacontract.yaml --output test-results.xml --output-format junit
```

---

## dbt

# `datacontract dbt`

Work with data contracts in your dbt project.

- `datacontract dbt sync` — merge an ODCS contract's schema (columns, descriptions, tags) and tests into your dbt project.
- `datacontract dbt test` — run the generated, contract-managed tests.

See the [dbt Integration](../dbt.md) guide.

## `datacontract dbt sync`

Merge one or more ODCS contracts' schema (column data types, descriptions, tags, model metadata) and tests into your dbt project. Modifies the existing dbt model YAML in place (preserving comments and formatting) and, if needed, creates a new model YAML sidecar (next to a model's `.sql`) or singular SQL tests (under `<test-paths>/datacontract_cli/`).

```bash
datacontract dbt sync [CONTRACT] ...
```

| Argument | Description |
|---|---|
| `CONTRACT` | One or more paths or globs of ODCS contracts to sync. If omitted, every `*.odcs.yaml` under `--project-dir` (and its subdirectories) is synced. |

| Option                                         | Default                                       | Description |
|------------------------------------------------|-----------------------------------------------|---|
| `--project-dir`                                | current dir                                   | Path to the dbt project root (must contain `dbt_project.yml`). |
| `--schema-name`                                | `all`                                         | Which ODCS schema to sync, by name. |
| `--model-resolution`                           | `name`                                        | Map an ODCS schema to a dbt model name: `name` or `physicalName`. |
| `--prune` / `--no-prune`                       | `--no-prune`                                  | Remove model columns and tags that are not declared in the contract. |
| `--target`                                     | —                                             | Forwarded to `dbt test --target`. |
| `--profiles-dir`                               | —                                             | Forwarded to `dbt test --profiles-dir`. |
| `--run-tests` / `--skip-tests`                 | `--skip-tests`                                | Run `dbt test` after syncing (requires dbt installed and on PATH). Required by `--publish`/`--server`. |
| `--publish`                                    | —                                             | URL to publish the results to. |
| `--server`                                     | The only one in the contract, else `--target` | Selected ODCS server. Its `type` is the dialect for mapping the contract's types to column `data_type`s, and its name is used for published results. |
| `--ssl-verification` / `--no-ssl-verification` | `--ssl-verification`                          | SSL verification when publishing. |
| `--debug` / `--no-debug`                       | `--no-debug`                                  | Enable debug logging. |

```bash
datacontract dbt sync orders.odcs.yaml --project-dir ./warehouse
```

### Example

Given this contract for IDs and email addresses in the `orders` schema:

```yaml title="orders-v1.odcs.yaml"
apiVersion: v3.0.0
kind: DataContract
id: orders
version: 1.0.0
status: active
schema:
  - name: orders
    description: Customer orders
    properties:
      - name: order_id
        physicalType: integer
        primaryKey: true
      - name: customer_email
        description: Billing email address
        logicalType: string
        physicalType: text
        required: true
        logicalTypeOptions:
          maxLength: 320
```

Let's assume that a minimal model properties file already exists. This is what it looks like after running
`datacontract dbt sync orders-v1.odcs.yaml` (highlighted lines got added):

```yaml title="models/orders.yml" {5-8,12-37,41-55}
version: 2
models:
  - name: orders
    description: Customer orders
    config:
      meta:
        datacontract_cli:
          contract_id: orders
    columns:
      - name: order_id
        data_type: integer
        data_tests:
          - not_null:
              config:
                severity: warn
                meta:
                  datacontract_cli:
                    check: orders__order_id__field_required
                    include_in_tests: true
                    contract_versions:
                      - 1.0.0
                    generated: true
              description: Check that field order_id has no missing values
          - unique:
              config:
                severity: warn
                meta:
                  datacontract_cli:
                    check: orders__order_id__field_unique
                    include_in_tests: true
                    contract_versions:
                      - 1.0.0
                    generated: true
              description: Check that field order_id has no duplicate values
        meta:
          datacontract_cli:
            generated: true
      - name: customer_email
        data_type: text
        description: Billing email address
        data_tests:
          - not_null:
              config:
                severity: warn
                meta:
                  datacontract_cli:
                    check: orders__customer_email__field_required
                    include_in_tests: true
                    contract_versions:
                      - 1.0.0
                    generated: true
              description: Check that field customer_email has no missing values
        meta:
          datacontract_cli:
            generated: true
```

The generated `config.meta.datacontract_cli` block is how `dbt sync`/`dbt test` recognize and scope managed tests; the per-column `meta.datacontract_cli.generated` marks a column the CLI added.

The `maxLength` bound becomes a self-contained singular SQL test (no `dbt_utils` needed). Its `config()` header carries the same `datacontract_cli` metadata:

```sql title="tests/datacontract_cli/orders/orders__1_0_0__orders__customer_email__length.sql"
-- AUTO-GENERATED by `datacontract dbt sync`. Do not edit.
-- Source contract: orders@1.0.0 (model: orders, check: customer_email__length)
{{ config(severity='warn', meta={"datacontract_cli": {"check": "orders__customer_email__field_length", "contract_versions": ["1.0.0"], "generated": true, "include_in_tests": true, "model": "orders", "field": "customer_email", "description": "Check that field customer_email has a length of at most 320"}}) }}
SELECT *
FROM {{ ref('orders') }}
WHERE "customer_email" IS NOT NULL
  AND (LENGTH("customer_email") > 320)
```

## `datacontract dbt test`

Run the contract-managed dbt tests that `datacontract dbt sync` generated, scoped to the requested contracts' models, report the results, and optionally publish them. Never modifies project files — run `datacontract dbt sync` first to (re)generate the tests. With multiple contracts, each contract's results are reported (and published) separately, followed by a summary.

```bash
datacontract dbt test [CONTRACT]...
```

| Argument | Description |
|---|---|
| `CONTRACT` | One or more paths or globs of ODCS contracts to test. If omitted, every `*.odcs.yaml` under `--project-dir` (and its subdirectories) is tested. Non-ODCS files are skipped with a warning, and a glob that matches nothing is ignored. |

| Option | Default | Description |
|---|---|---|
| `--project-dir` | current dir | Path to the dbt project root (must contain `dbt_project.yml`). |
| `--target` | — | Forwarded to `dbt test --target`. |
| `--profiles-dir` | — | Forwarded to `dbt test --profiles-dir`. |
| `--publish` | — | URL to publish the results to. |
| `--server` | The only one in the contract, else `--target` | ODCS server name for published test results. |
| `--ssl-verification` / `--no-...` | on | SSL verification when publishing. |
| `--debug` / `--no-debug` | off | Enable debug logging. |

```bash
datacontract dbt test orders.odcs.yaml --project-dir ./warehouse
```

---

## edit

# `datacontract edit`

Edit a data contract file in the [Data Contract Editor](../editor.md) (web UI). Starts a local web server that opens the editor for the given file; saving writes back to the local file. Requires the `api` extra.

```bash
datacontract edit [LOCATION]
```

| Argument | Default | Description |
|---|---|---|
| `LOCATION` | `datacontract.yaml` | Path of the data contract YAML to edit. If it does not exist, you are asked whether to initialize a new one. If a URL is given, you are asked whether to download a local copy. |

| Option | Default | Description |
|---|---|---|
| `--port` | `4243` | Bind the server to this port. |
| `--host` | `127.0.0.1` | Bind to this host (use `0.0.0.0` for Docker). |
| `--editor-version` | bundled | Version of the `datacontract-editor` npm package to load from the CDN. |
| `--editor-assets-url` | — | Base URL for editor assets (takes precedence over `--editor-version`). |
| `--open` / `--no-open` | `--open` | Open the editor in the default browser. |
| `--debug` / `--no-debug` | `--no-debug` | Enable debug logging. |

```bash
datacontract edit datacontract.yaml
```

See the [Data Contract Editor](../editor.md) guide for details.

---

## export

# `datacontract export`

Convert a data contract to a target format. See the [Exports](../exports/index.md) section for every format and its options.

```bash
datacontract export <format> [LOCATION] [--output FILE] [--server NAME] [--schema-name NAME]
```

Run `datacontract export <format> --help` to see format-specific options.

```bash
datacontract export html datacontract.yaml --output datacontract.html
```

For SQL dialects (`postgres`, `mysql`, `snowflake`, `databricks`, `sqlserver`, `trino`, `oracle`, `clickhouse`), use `datacontract export sql --dialect <dialect>`.

Available formats: `avro`, `avro-idl`, `bigquery`, `custom`, `data-caterer`, `dbml`, `dbt-models`, `dbt-sources`, `dbt-staging-sql`, `dcs`, `dqx`, `excel`, `go`, `great-expectations`, `html`, `iceberg`, `jsonschema`, `markdown`, `mermaid`, `odcs`, `protobuf`, `pydantic-model`, `rdf`, `sodacl`, `spark`, `sql`, `sql-query`, `sqlalchemy`.

---

## import

# `datacontract import`

Create a data contract from a source format. See the [Imports](../imports/index.md) section for every format and its options.

```bash
datacontract import <format> --source <source> [--output FILE]
```

Run `datacontract import <format> --help` to see format-specific options.

```bash
datacontract import sql --source ddl.sql --dialect postgres --output datacontract.yaml
```

Available formats: `adls`, `athena`, `avro`, `bigquery`, `csv`, `databricks`, `dbml`, `dbt`, `excel`, `gcs`, `glue`, `iceberg`, `json`, `jsonschema`, `mysql`, `odcs`, `oracle`, `parquet`, `postgres`, `powerbi`, `protobuf`, `redshift`, `s3`, `snowflake`, `spark`, `sql`, `sqlserver`, `trino`.

---

## init

# `datacontract init`

Create an empty data contract from a template.

```bash
datacontract init [LOCATION]
```

| Argument | Default | Description |
|---|---|---|
| `LOCATION` | `datacontract.yaml` | Path of the data contract file to create. |

| Option | Default | Description |
|---|---|---|
| `--template` | — | URL of a template or data contract to start from. |
| `--overwrite` / `--no-overwrite` | `--no-overwrite` | Replace an existing file. |
| `--debug` / `--no-debug` | `--no-debug` | Enable debug logging. |

```bash
datacontract init datacontract.yaml
```

---

## lint

# `datacontract lint`

Validate that the `datacontract.yaml` is correctly formatted against the ODCS JSON Schema.

```bash
datacontract lint [LOCATION]
```

| Argument | Default | Description |
|---|---|---|
| `LOCATION` | `datacontract.yaml` | Location (URL or path) of the data contract YAML. |

| Option | Default | Description |
|---|---|---|
| `--json-schema` | — | Location (URL or path) of the ODCS JSON Schema. |
| `--output` | stdout | File path to write the results to. |
| `--output-format` | — | `json` or `junit`. |
| `--all-errors` | — | Report all JSON Schema validation errors instead of stopping after the first. |
| `--inline-references` / `--no-inline-references` | `--inline-references` | Resolve and inline external references. |
| `--debug` / `--no-debug` | `--no-debug` | Enable debug logging. |

```bash
datacontract lint datacontract.yaml
```

---

## publish

# `datacontract publish`

Publish the data contract to [Entropy Data](https://www.entropy-data.com/).

```bash
datacontract publish [LOCATION]
```

| Argument | Default | Description |
|---|---|---|
| `LOCATION` | `datacontract.yaml` | Location (URL or path) of the data contract YAML. |

| Option | Default | Description |
|---|---|---|
| `--json-schema` | — | Location (URL or path) of the ODCS JSON Schema. |
| `--ssl-verification` / `--no-...` | on | SSL verification when publishing. |
| `--debug` / `--no-debug` | off | Enable debug logging. |

```bash
datacontract publish datacontract.yaml
```

---

## test

# `datacontract test`

Run schema and quality tests on the servers configured in the contract. See **[Test your Data](../testing/index.md)** for connection details and supported data sources.

```bash
datacontract test [LOCATION]
```

| Argument | Default | Description |
|---|---|---|
| `LOCATION` | `datacontract.yaml` | Location (URL or path) of the data contract YAML. |

| Option | Default | Description |
|---|---|---|
| `--server` | `all` | Which server to test (the key in the `servers` block), or `all`. |
| `--schema-name` | `all` | Which schema to test, or `all`. |
| `--checks` | all | Comma-separated categories: `schema`, `quality`, `servicelevel`, `custom`. |
| `--publish` | — | URL to publish the results to after the test. |
| `--output` | — | File path to write results to. |
| `--output-format` | — | `json` or `junit`. |
| `--include-failed-samples` / `--no-...` | off | Collect a small sample of failed rows. |
| `--logs` / `--no-logs` | `--no-logs` | Print logs. |
| `--ssl-verification` / `--no-...` | on | SSL verification when publishing. |
| `--inline-references` / `--no-...` | on | Resolve and inline external references. |
| `--debug` / `--no-debug` | off | Enable debug logging. |

```bash
datacontract test datacontract.yaml --server production
```

---

## Python Library


Everything the CLI does is also available as a Python library through the `DataContract` class. This is useful for embedding data contract checks in pipelines, notebooks, orchestrators (Airflow, Dagster, Prefect), or your own tooling.

```bash
pip install 'datacontract-cli[all]'
```

## Test a data contract

```python
from datacontract.data_contract import DataContract

data_contract = DataContract(data_contract_file="datacontract.yaml")
run = data_contract.test()

if not run.has_passed():
    print("Data quality validation failed.")
    # Abort the pipeline, alert, or take corrective action...
```

### Inspecting the result

`test()` (and `lint()`) return a `Run` object:

```python
run = data_contract.test()

print(run.result)          # "passed", "failed", "warning", or "error"
print(run.has_passed())    # True / False

for check in run.checks:
    print(check.result, check.name, check.reason)
```

## Constructor options

The `DataContract` constructor accepts the contract from a file, a string, or an in-memory ODCS object, plus the same options as the CLI:

```python
from datacontract.data_contract import DataContract

DataContract(
    data_contract_file="datacontract.yaml",  # or data_contract_str=... / data_contract=<ODCS object>
    server="production",                      # which server to test (default: all)
    schema_name="orders",                     # which schema to test (default: "all")
    check_categories={"schema", "quality"},   # subset of: schema, quality, servicelevel, custom
    publish_url="https://api.entropy-data.com/api/test-results",
    inline_references=True,
    include_failed_samples=False,
)
```

| Argument | Description |
|---|---|
| `data_contract_file` | Path or URL to the contract. |
| `data_contract_str` | The contract as a YAML string. |
| `data_contract` | An in-memory `OpenDataContractStandard` object. |
| `server` | Server to test against (the key in `servers`). |
| `schema_name` | Which schema/model to test (default `"all"`). |
| `check_categories` | Set of categories to run: `schema`, `quality`, `servicelevel`, `custom`. |
| `spark` | A `SparkSession`, for the `dataframe` / Databricks engines. |
| `duckdb_connection` | An existing DuckDB connection. |
| `publish_url` | URL to publish test results to. |
| `inline_references` | Resolve external references (default `True`). |
| `include_failed_samples` | Collect a sample of failing rows (default `False`). |

## Lint a data contract

```python
from datacontract.data_contract import DataContract

run = DataContract(data_contract_file="datacontract.yaml").lint()
assert run.has_passed()
```

## Export

`export()` returns the converted artifact as a string (or bytes for binary formats such as Excel). Pass the target format and, optionally, a schema and format-specific keyword arguments.

```python
from datacontract.data_contract import DataContract

data_contract = DataContract(data_contract_file="datacontract.yaml", server="snowflake")

sql = data_contract.export("sql")
print(sql)

# Format-specific options are passed as keyword arguments
html = data_contract.export("html")
with open("datacontract.html", "w") as f:
    f.write(html)
```

See [Exports](./exports/index.md) for the full list of formats.

## Import

`DataContract.import_from_source()` is a class method that returns an ODCS (`OpenDataContractStandard`) object. Format-specific options are passed as keyword arguments.

```python
from datacontract.data_contract import DataContract

odcs = DataContract.import_from_source(
    format="sql",
    source="my_ddl.sql",
    dialect="postgres",
)

# Wrap it to export or test
data_contract = DataContract(data_contract=odcs)
print(data_contract.export("odcs"))
```

See [Imports](./imports/index.md) for the full list of formats.

## Compare two contracts (changelog)

```python
from datacontract.data_contract import DataContract

v1 = DataContract(data_contract_file="v1.odcs.yaml")
v2 = DataContract(data_contract_file="v2.odcs.yaml")

result = v1.changelog(v2)
print(result)
```

## Spark DataFrames and Databricks

Pass a `SparkSession` to test in-memory DataFrames (registered as temporary views) or to run inside a Databricks notebook:

```python
from datacontract.data_contract import DataContract

df.createOrReplaceTempView("my_table")

data_contract = DataContract(
    data_contract_file="datacontract.yaml",
    spark=spark,
)
run = data_contract.test()
assert run.result == "passed"
```

See [Spark DataFrame](./testing/dataframe.md) and [Databricks](./testing/databricks.md) for details.

## Credentials

Server credentials are read from environment variables (or a `.env` file), exactly as with the CLI — see [Test your Data](./testing/index.md). Set them before constructing `DataContract`.

---

## Adopting Data Contracts


There are two proven ways to introduce data contracts with the Data Contract CLI. Pick the one that matches your situation.

## Data-first approach

Create a data contract based on the **actual data**. This is the fastest way to get started and to get feedback from data consumers.

1. Use an existing physical schema (e.g. SQL DDL) as a starting point to define your logical data model in the contract. Right after the import, double-check that the actual data meets the imported model:

   ```bash
   datacontract import sql --source ddl.sql
   datacontract test
   ```

2. Add [quality checks](./quality-rules/index.md) and additional type constraints one by one, making sure the data still adheres to the contract:

   ```bash
   datacontract test
   ```

3. Validate that the `datacontract.yaml` is correctly formatted and adheres to the [Open Data Contract Standard](./open-data-contract-standard.md):

   ```bash
   datacontract lint
   ```

4. Set up a CI pipeline that runs daily for continuous quality checks. Use the [`ci`](./commands/ci.md) command for CI-optimized output (GitHub Actions annotations and step summary, Azure DevOps annotations). You can also report results to tools like [Entropy Data](https://entropy-data.com):

   ```bash
   datacontract ci --publish https://api.entropy-data.com/api/test-results
   ```

   See [Scheduling and CI/CD](./ci-cd.md).

## Contract-first approach

Create a data contract based on the **requirements** from use cases, before the data product exists.

1. Start with a `datacontract.yaml` template:

   ```bash
   datacontract init
   ```

2. Create the model and quality guarantees based on your business requirements. Fill in the terms, descriptions, etc., then validate the format:

   ```bash
   datacontract lint
   ```

3. Use the [export](./exports/index.md) functions to start building the providing data product as well as the integration into consuming data products:

   ```bash
   # data provider
   datacontract export dbt-models
   # data consumer
   datacontract export dbt-sources
   datacontract export dbt-staging-sql
   ```

4. Test that your data product implementation adheres to the contract:

   ```bash
   datacontract test
   ```

---

## Sync with dbt


The Data Contract CLI integrates with [dbt](https://www.getdbt.com/) in multiple ways:

- **Sync**: Merge the schema and tests of one or multiple data contracts into an existing dbt
  project (`datacontract dbt sync`)
- **Test**: Run the CLI-generated dbt tests, optionally scoped to a contract (
  `datacontract dbt test`)
- **Export**: Do a one-time export from a data contract into a dbt model schema, a sources YAML, or a staging SQL file (`datacontract export`)
- **Import**: Create a data contract from a dbt manifest file (`datacontract import dbt`)

## `datacontract dbt sync`

`dbt sync` reads one or more ODCS data contracts and applies the specified schema and tests to an
existing dbt project. It updates the model properties files (YAML) in-place and adds singular SQL
tests if needed.

```bash
# Auto-discover every *.odcs.yaml in a dbt project, update models and test
datacontract dbt sync

# Delete columns, tests etc. that are not specified in the contract
datacontract dbt sync --prune

# Specify contract and dbt project directory
datacontract dbt sync orders.odcs.yaml --project-dir ./warehouse

# Several contracts, or a glob
datacontract dbt sync orders.odcs.yaml customers.odcs.yaml
datacontract dbt sync "contracts/*.odcs.yaml"

# Run the generated tests (or as separate command: datacontract dbt test)
datacontract dbt sync --run-tests

# Generate, run, and publish results to an Entropy Data instance 
datacontract dbt sync orders.odcs.yaml --run-tests --publish https://api.entropy-data.com/api/test-results
```

A list of all options can be found at the [`dbt` command reference](./commands/dbt.md#datacontract-dbt-sync).

### What is synced?

All CLI-managed content in the dbt project gets marked with custom meta tags. By default, schemas
or tests that are neither specified in the contract nor generated by the CLI are kept, unless
`--prune` is specified.

| ODCS source | dbt target                                                 |
|---|------------------------------------------------------------|
| `schema` / `property` description | model / column `description`                               |
| `property.logicalType` / `physicalType` | column `data_type`, mapped to the server dialect           |
| `property` | column entry                                               |
| `schema.tags` / `property.tags` | `config.tags`                                              |
| `property.required`, single `primaryKey`, `property.quality.mustBe` | native dbt tests (`not_null`, `unique`, `accepted_values`) |
| composite `primaryKey`, `logicalTypeOptions` bounds (`minLength`, `maxLength`, `pattern`, `minimum`, …), `quality` with a `query` + bound, `schema.quality` `rowCount` | singular SQL tests at `<testsRoot>/datacontract_cli/`      |

:::tip
See the [`dbt` command reference](./commands/dbt.md#datacontract-dbt-sync) for a worked example and all options.
:::

### Versioned models

`dbt sync` works natively with [dbt model versions](https://docs.getdbt.com/docs/mesh/govern/model-versions). To specify multiple versions using data contracts, create one contract file per version. Their filename must include an integer version number with a preceding `v`, e.g. `orders-v01.odcs.yaml`.

If `dbt sync` is fed with a subset of the versioned contracts, the other versions remain unchanged.

## `datacontract dbt test`

Runs the contract-managed tests that `dbt sync` generated, reports the results, and optionally publishes them. Like `dbt sync`, the command accepts multiple contracts.

```bash
# Every contract in the current project
datacontract dbt test

# A single contract and dbt target in a specified project
datacontract dbt test orders.odcs.yaml --project-dir ./warehouse --target dev
```

See the [`dbt` command reference](./commands/dbt.md).

## dbt exporters

Convert a contract into dbt artifacts:

- **[`dbt-models`](./exports/dbt-models.md)** — dbt model schema YAML. If a server is selected via `--server`, the dbt `data_types` match the expected data types of that server; otherwise it defaults to `snowflake`.
- **[`dbt-sources`](./exports/dbt-sources.md)** — dbt sources YAML.
- **[`dbt-staging-sql`](./exports/dbt-staging-sql.md)** — a dbt staging SQL file.

```bash
datacontract export dbt-models datacontract.yaml --server snowflake
```

## dbt importer

Generate a contract from a dbt project's `manifest.json`:

```bash
# Import specific tables from a dbt manifest
datacontract import dbt --source manifest.json --model orders --model line_items

# Import all tables
datacontract import dbt --source manifest.json
```

---

## API(Docs)


The Data Contract CLI can run as a web server that exposes a REST API for data contract testing, linting, exporting, and changelogs. This is useful for integrating data contract checks into other services.

You can try a public demo at [api.datacontract.com](https://api.datacontract.com). Note that the demo endpoint cannot connect to your secured data sources.

## Starting the API

The API requires the `api` extra:

```bash
pip install 'datacontract-cli[api]'
```

Start the server:

```bash
datacontract api
```

| Option | Default | Description |
|---|---|---|
| `--port` | `4242` | Bind the socket to this port. |
| `--host` | `127.0.0.1` | Bind to this host. In Docker, use `0.0.0.0`. |
| `--debug` / `--no-debug` | `--no-debug` | Enable debug logging. |

You can pass extra keyword arguments through to `uvicorn.run()`, e.g.:

```bash
datacontract api --port 1234 --root_path /datacontract
```

## OpenAPI / Swagger UI

Once running, open the interactive OpenAPI documentation (Swagger UI) at
[http://localhost:4242](http://localhost:4242). You can execute the commands directly from the UI.

## Test a data contract

POST a data contract as the request body to `/test` and receive the test results as JSON:

```bash
curl -X POST "http://localhost:4242/test?server=production" \
  --data-binary @datacontract.yaml
```

You can also send the YAML inline with `-H 'Content-Type: application/yaml'`.

## Export a data contract

```bash
curl -X POST "http://localhost:4242/export?format=sql" \
  --data-binary @datacontract.yaml
```

## Changelog between two contracts

POST a JSON body with `v1` (before) and `v2` (after) as YAML strings. The response is a JSON object with `summary` and `entries`:

```bash
curl -X POST "http://localhost:4242/changelog" \
  -H "Content-Type: application/json" \
  -d '{
    "v1": "'"$(cat v1.odcs.yaml)"'",
    "v2": "'"$(cat v2.odcs.yaml)"'"
  }'
```

## Configure server credentials

To connect to a data source, set the required credentials as environment variables **before starting the API** (see [Test your Data](./testing/index.md#configuring-the-connection)). For example, for Snowflake:

```bash
export DATACONTRACT_SNOWFLAKE_USERNAME=123
export DATACONTRACT_SNOWFLAKE_PASSWORD=
export DATACONTRACT_SNOWFLAKE_WAREHOUSE=
export DATACONTRACT_SNOWFLAKE_ROLE=
```

## Secure the API

Set `DATACONTRACT_CLI_API_KEY` to a secret value (such as a random UUID) to require authentication. Requests must then include the header `x-api-key` with the correct key.

```bash
export DATACONTRACT_CLI_API_KEY=<your-secret-key-such-as-a-random-uuid>
```

:::warning
Securing the API is highly recommended. Data contract tests may otherwise be subject to SQL injection or leak sensitive information.
:::

## Run as a Docker container

The pre-built image can run the API in any container environment (Docker Compose, Kubernetes, Azure Container Apps, Google Cloud Run, …):

```yaml
services:
  datacontract-api:
    image: datacontract/cli:latest
    ports:
      - "4242:4242"
    environment:
      - DATACONTRACT_CLI_API_KEY=a079ce4c-af90-45ab-abe5-a8d7697f60d6
    command: ["api", "--host", "0.0.0.0"]
```

See the [`api` command reference](./commands/api.md).

---

## Extending the CLI


The CLI uses factory patterns for export and import, so you can register your own **custom exporter** (a new output format) or **custom importer** (a new input format) and use them through the same `DataContract` API.

:::note
For producing a custom text output from a Jinja template, you usually don't need code — use the built-in [`export custom`](./exports/custom.md) exporter. Register a custom exporter class (below) when you need full programmatic control.
:::

## Custom exporter

Implement `Exporter.export(...)` and register the class with the exporter factory. The `data_contract` argument is an `OpenDataContractStandard` (ODCS) instance, so you read `data_contract.name`, `data_contract.version`, `data_contract.schema_`, etc.

```python
from datacontract.data_contract import DataContract
from datacontract.export.exporter import Exporter
from datacontract.export.exporter_factory import exporter_factory

class MyExporter(Exporter):
    def export(self, data_contract, schema_name, server, sql_server_type, export_args) -> str:
        lines = [f"# {data_contract.name} v{data_contract.version}"]
        for schema in data_contract.schema_ or []:
            columns = ", ".join(p.name for p in (schema.properties or []))
            lines.append(f"{schema.name}: {columns}")
        return "\n".join(lines)

# Register the exporter under a new format name
exporter_factory.register_exporter("my_format", MyExporter)

# Use it like any built-in format
data_contract = DataContract(data_contract_file="datacontract.yaml")
print(data_contract.export("my_format"))
```

The `export()` method receives:

| Argument | Description |
|---|---|
| `data_contract` | The contract as an `OpenDataContractStandard` object. |
| `schema_name` | The selected schema, or `"all"`. |
| `server` | The selected server name (or `None`). |
| `sql_server_type` | The SQL dialect (`"auto"` by default). |
| `export_args` | A dict of any extra keyword arguments passed to `export(...)`. |

## Custom importer

Implement `Importer.import_source(...)` to build and return an `OpenDataContractStandard`, then register it with the importer factory.

```python
import json
from open_data_contract_standard.model import (
    OpenDataContractStandard,
    SchemaObject,
    SchemaProperty,
)
from datacontract.data_contract import DataContract
from datacontract.imports.importer import Importer
from datacontract.imports.importer_factory import importer_factory

class MyImporter(Importer):
    def import_source(self, source, import_args) -> OpenDataContractStandard:
        data = json.loads(source)
        return OpenDataContractStandard(
            apiVersion="v3.0.2",
            kind="DataContract",
            id=data["id"],
            name=data["title"],
            version=data["version"],
            description={"purpose": data.get("description")},
            schema=[
                SchemaObject(
                    name=model["name"],
                    properties=[
                        SchemaProperty(name=col["name"], logicalType=col.get("type"))
                        for col in model["columns"]
                    ],
                )
                for model in data.get("models", [])
            ],
        )

# Register the importer under a new format name
importer_factory.register_importer("my_format", MyImporter)

source = '{"id": "urn:my:contract", "title": "My Contract", "version": "1.0.0", "models": []}'
odcs = DataContract.import_from_source(format="my_format", source=source)

# import_from_source returns an ODCS object; wrap it to export or test
print(DataContract(data_contract=odcs).export("odcs"))
```

## Tips

- Register classes once at import time (e.g. in a small plugin module you import before using the CLI as a library).
- Custom formats are only available through the [Python library](./python-library.md), not the standalone `datacontract` command.
- See the built-in exporters/importers in the [source tree](https://github.com/datacontract/datacontract-cli/tree/main/datacontract) for complete, real-world implementations.

---

## Release Notes

<!-- AUTOGENERATED from CHANGELOG.md by docs/scripts/generate-release-notes.mjs. Do not edit by hand. -->

# Release Notes

Every released version of the Data Contract CLI, newest first. The CLI follows
[Semantic Versioning](https://semver.org/spec/v2.0.0.html); breaking changes are
marked as such in the entry.

- Install or upgrade with `uv tool install --upgrade datacontract-cli` or `pip install --upgrade datacontract-cli` — see [Installation](./installation.md).
- Packages: [PyPI](https://pypi.org/project/datacontract-cli/#history) · [Docker Hub](https://hub.docker.com/r/datacontract/cli/tags) · [GitHub releases](https://github.com/datacontract/datacontract-cli/releases)

## Unreleased {#unreleased}

### Added
- `datacontract import trino` creates a data contract from a Trino catalog, including a ready-to-test `servers` block
- `datacontract import oracle` creates a data contract from a live Oracle database, including a ready-to-test `servers` block
- `datacontract import gcs` and `datacontract import adls` create a data contract from files in Google Cloud Storage or Azure Blob Storage, including a ready-to-test `servers` block
- `datacontract import sqlserver` creates a data contract from a live SQL Server database, including a ready-to-test `servers` block
- `datacontract import mysql` creates a data contract from a live MySQL database, including a ready-to-test `servers` block
- The documentation has a [Release Notes](https://docs.datacontract.com/release-notes) page, generated from this changelog
- The documentation has a guide to [migrate contracts from DCS to ODCS](https://docs.datacontract.com/migrate-dcs-to-odcs)
- `datacontract import s3` creates a data contract from files in an S3 bucket, including a ready-to-test `servers` block
- `datacontract import athena` creates a data contract from an Amazon Athena database, including a ready-to-test `servers` block
- `datacontract import unity` is now `datacontract import databricks`; the `unity` format name keeps working
- Redshift infers the authentication method: a password means a database login, otherwise your AWS session is used for IAM. `DATACONTRACT_REDSHIFT_AUTHENTICATION` is no longer required and remains as an override
- `datacontract import postgres` creates a data contract from a live Postgres schema, including a ready-to-test `servers` block
- `datacontract import redshift` creates a data contract from an Amazon Redshift schema, including a ready-to-test `servers` block
- Redshift supports IAM authentication, using temporary credentials from your AWS session instead of a database password
- `datacontract import bigquery` now generates a `servers` block, so `datacontract test` works right after the import
- `datacontract test` verifies declared primary keys: each key column must have no missing values, and the key must have no duplicates (a composite key is checked as a tuple) ([#1220](https://github.com/datacontract/datacontract-cli/issues/1220) [@DMZ22](https://github.com/DMZ22))

### Fixed
- A data contract could inject SQL into the duckdb session through `endpointUrl`, which is interpolated into the statement that stores the S3, GCS and Azure credentials; every value is escaped now
- The `datacontract api` server accepted a local file path as the `schema` query parameter, so a caller could have it read files from the server's filesystem; only `http(s)` URLs are accepted now
- Trino physical type checks were silently skipped: its `information_schema` has no length or precision columns, so the catalog query failed and a wrong `physicalType` still passed
- `datacontract import athena` and `datacontract import glue` now honour `DATACONTRACT_S3_ACCESS_KEY_ID` and `DATACONTRACT_S3_SECRET_ACCESS_KEY`; the Glue catalog was read with ambient AWS credentials only
- S3 now uses an existing AWS session (`aws sso login`, `AWS_PROFILE`, instance roles) when no access key is configured; previously such a setup failed with `403 Forbidden`
- Documented that Athena authenticates with an existing AWS session (`aws sso login`, `AWS_PROFILE`, instance roles); static access keys were presented as the only option
- `regionName` in an Athena `servers` block was ignored, so the region could only be set via `DATACONTRACT_S3_REGION`
- `datacontract import glue` mapped `timestamp` columns to `logicalType: date` instead of `timestamp`
- Testing and importing Redshift failed with `codec not available in Python: 'UNICODE'`
- Error messages no longer drop bracketed text such as `pip install "botocore[crt]"`
- BigQuery export failed on fields with `logicalType: time`
- Testing Parquet files failed for `number` fields without a declared precision and scale
- CSV and JSON imports now write detected formats (`email`, `uuid`, `date-time`) to `logicalTypeOptions.format` instead of a custom property, so they are validated by `datacontract test`
- SQL imports now map `TIME` types with precision or time zone (e.g. `TIME(9)`) to `logicalType: time`; previously the logical type was left unset
- `datacontract test --checks quality` now runs `rowCount` quality rules, which were wrongly categorized as schema checks

## 1.0.14 — 2026-07-23 {#v1-0-14}

### Added
- `datacontract export protobuf` supports a customizable package name via the `protoPackageName` custom property ([#1381](https://github.com/datacontract/datacontract-cli/issues/1381) [@Schokuroff](https://github.com/Schokuroff))
- `datacontract export protobuf` emits `optional` for non-required message/object fields ([#1390](https://github.com/datacontract/datacontract-cli/issues/1390) [@Schokuroff](https://github.com/Schokuroff))

### Fixed
- `datacontract dbt sync` resolves `{object}`/`{property}` placeholders in custom `sql` quality checks to the dbt `ref()` and column name ([#1397](https://github.com/datacontract/datacontract-cli/issues/1397))
- SyntaxWarning during installation: `datacontract/lint/resolve.py:72: SyntaxWarning: 'return' in a 'finally' block return except_message` is handled properly ([#1384](https://github.com/datacontract/datacontract-cli/issues/1384) [@Cupprum](https://github.com/Cupprum))
- `datacontract test` no longer reports "backend is not installed" for Athena and other ibis SQL backends when `packaging` is missing from the environment

## 1.0.13 — 2026-07-14 {#v1-0-13}

### Added
- `datacontract test` reports the nested types of a property that declares `properties:` or `items:` as a separate check
- `datacontract test` on Snowflake verifies the `physicalType` of nested properties against the real column type

### Fixed
- `datacontract test` on Snowflake matches a `physicalType` against the alias the catalog reports, such as `BIGINT` on a `NUMBER(38,0)` column
- `datacontract test` no longer reports a mismatch for a `physicalType` without precision, such as `NUMBER` on a `NUMBER(12,2)` column

## 1.0.12 — 2026-07-10 {#v1-0-12}

### Fixed
- `datacontract test` now recursively verifies nested `logicalType` for Snowflake structured `OBJECT`/`ARRAY` columns ([#1373](https://github.com/datacontract/datacontract-cli/issues/1373))
- `datacontract test` now fails for complex types when the nested type definition cannot be verified (e.g. if array<integer> is required, array<json> will no longer be accepted) ([#1373](https://github.com/datacontract/datacontract-cli/issues/1373))

## 1.0.11 — 2026-07-09 {#v1-0-11}

### Added
- `datacontract import` can now import BigQuery type `INTERVAL` ([#1367](https://github.com/datacontract/datacontract-cli/issues/1367),[#1372](https://github.com/datacontract/datacontract-cli/issues/1372) [@fantastisch](https://github.com/fantastisch))

### Fixed
- Failed business definition IRI lookups now suggest the `ENTROPY_DATA_HOST` value to set when the IRI host does not match the configured entropy-data host.
- `datacontract test` no longer reports a physical type mismatch for BigQuery type aliases, such as a `physicalType` of `INTEGER` on an `INT64` column ([#1371](https://github.com/datacontract/datacontract-cli/issues/1371) [@fantastisch](https://github.com/fantastisch))
- `datacontract test` no longer fails with `CANNOT_CONVERT_COLUMN_INTO_BOOL` on Databricks when a Spark session is used

## 1.0.10 — 2026-07-08 {#v1-0-10}

### Added
- extended `datacontract dbt sync`:
  - now edits existing properties files (schema.yaml) in-place instead of creating new ones
  - preserve manual edits to a properties file
  - edit schema incl. descriptions, column types and tags, not only tests
  - add `--prune` flag to remove everything that's not specified in the contract (models, tags, checks) - per default, only generated content gets removed
  - support for multiple contract versions (`versions:` block)
  - possibility to sync multiple contracts at once
  - store the contract id and version in the `meta:` block
- added `datacontract dbt test`: Use local `dbt` to run all tests that have been generated using `datacontract dbt sync` earlier (scoped to a single data contract, or all data contracts in the opened dbt project)
  - optionally publish to Entropy Data

### Changed
- `datacontract dbt sync`:
  - no longer executes tests per default (use `--run-tests` or run `datacontract dbt test` afterwards)
- `datacontract test` now verifies a field's `physicalType` against the column's real native type from the platform catalog (length and precision included), taking precedence over `logicalType` ([#1354](https://github.com/datacontract/datacontract-cli/issues/1354))
- `datacontract test` JSON output now includes `datacontractCliVersion` ([#1353](https://github.com/datacontract/datacontract-cli/issues/1353) [@hk8suva](https://github.com/hk8suva))
- `datacontract test` type-check errors now report the first failing field and a count of the remaining errors ([#1334](https://github.com/datacontract/datacontract-cli/issues/1334) [@jorgengranseth](https://github.com/jorgengranseth))

### Fixed
- `datacontract test` no longer fails the type check for SQL Server `uniqueidentifier` (UUID) columns with "the column type could not be determined" ([#1354](https://github.com/datacontract/datacontract-cli/issues/1354))
- `datacontract test` against BigQuery no longer fails SQL quality checks with `'RowIterator' object has no attribute 'fetchone'`
- `datacontract import` against BigQuery applies correct `logicalType` for BigQuery types `TIMESTAMP`, `DATETIME`, and `TIME` ([#1366](https://github.com/datacontract/datacontract-cli/issues/1366) [@fantastisch](https://github.com/fantastisch)) 

## 1.0.9 — 2026-06-26 {#v1-0-9}

### Fixed
- `datacontract test` against Kafka no longer reports every field as null for plain (non-Confluent Schema Registry) Avro messages ([#1344](https://github.com/datacontract/datacontract-cli/issues/1344))
- Honor the `pattern` argument in `invalidValues` quality checks ([#1346](https://github.com/datacontract/datacontract-cli/issues/1346))

## 1.0.8 — 2026-06-25 {#v1-0-8}

### Fixed
- `datacontract test` against Redshift no longer fails with `relation "pg_catalog.pg_enum" does not exist`. Redshift rides the Postgres ibis backend, whose schema introspection joins `pg_catalog.pg_enum` to detect enum columns — a relation Redshift does not expose. Introspection now omits that join (Redshift has no enum types).

## 1.0.7 — 2026-06-25 {#v1-0-7}

### Added
- New global option `--system-truststore` (env `DATACONTRACT_SYSTEM_TRUSTSTORE`) to verify TLS using the operating system's certificate trust store, for use behind corporate proxies or with internal CAs.

## 1.0.6 — 2026-06-24 {#v1-0-6}

### Fixed
- `datacontract test` against Redshift no longer fails with `column "current_schema" does not exist`. Redshift rides the Postgres ibis backend, whose introspection resolved the active schema with `SELECT current_schema` (no parentheses) — valid on PostgreSQL but rejected by Redshift, which only supports `current_schema()`. The configured server `schema` is now passed explicitly during introspection, skipping that query.

## 1.0.5 — 2026-06-24 {#v1-0-5}

### Fixed
- `datacontract test` now only supports logicalTypes. Previously physicalType was preferrerd and used even if logicalType did not exist. 
- `datacontract test` field type check now compares the full structured type tree for `object` and `array` logical types.
- Unknown and unsupported types are silently ignored rather than failing the check. Specifically the `map` type is not supported until ODCS version v3.2.0 and is also ignored. 
- `datacontract --help` no longer fails with `ModuleNotFoundError: No module named 'ibis'` when the optional `ibis` extra is not installed.
- `datacontract test` against Oracle now qualifies tables with the configured server `schema` (owner), fixing `Could not read model '': ` when the login user differs from the table owner.

## 1.0.4 — 2026-06-22 {#v1-0-4}

### Added
- `datacontract test` validates Azure Blob Storage / ADLS Gen2 file metadata against a data contract used as a storage policy ([#1227](https://github.com/datacontract/datacontract-cli/issues/1227))
- `datacontract test` against Trino now supports `DATACONTRACT_TRINO_AUTHENTICATION=jwt` with `DATACONTRACT_TRINO_JWT_TOKEN`, and `DATACONTRACT_TRINO_AUTHENTICATION=oauth2` for the interactive browser flow.
- `datacontract export sql --dialect clickhouse`: export data contracts to ClickHouse SQL DDL. ([#1293](https://github.com/datacontract/datacontract-cli/issues/1293))
- Example data contracts and import sources under `examples/`, used as the worked examples on the docs export and import pages.

### Fixed
- `datacontract import unity` now imports struct and array columns as structured ODCS types instead of only a flat type string: structs get nested `properties`, arrays get `items` (parsed recursively from Unity's `type_json`, including descriptions on nested fields). Arrays also get the correct `logicalType: array` (previously `object`); this logical type fix applies to the `sql` and `snowflake` importers as well. Map columns and other unmappable SQL types now leave `logicalType` unset (instead of the invalid `object`) until ODCS v3.2 adds `logicalType: map` (RFC 0030). ([#1280](https://github.com/datacontract/datacontract-cli/issues/1280))
- `datacontract export dcs` no longer crashes on data contracts with a structured description or a standard server.
- `datacontract test` against Oracle releases before 23ai no longer fails every check with `ORA-00923: FROM keyword not found where expected` during schema introspection.
- `datacontract test` now recognizes Oracle `VARCHAR2`/`NVARCHAR2` (with or without a length, e.g. `VARCHAR2(4000)`) as string types in the field type check.

## 1.0.3 — 2026-06-15 {#v1-0-3}

### Added
- `datacontract import powerbi` imports a data contract from a Power BI semantic model (.pbit, .bim, or .json) ([#1232](https://github.com/datacontract/datacontract-cli/issues/1232),[#1233](https://github.com/datacontract/datacontract-cli/issues/1233) [@dmaresma](https://github.com/dmaresma))

### Changed
- `datacontract edit` shows the edited filename in the editor header, no longer offers New/Load Example/Open, and Cancel reverts to the file on disk

### Fixed
- `datacontract test` no longer aborts remaining quality checks after a SQL quality check falls back to native execution on DuckDB-backed sources ([#1302](https://github.com/datacontract/datacontract-cli/issues/1302))
- Athena: pass `DATACONTRACT_S3_SESSION_TOKEN` to the connection again, fixing `UnrecognizedClientException` with temporary AWS credentials ([#1309](https://github.com/datacontract/datacontract-cli/issues/1309))

## 1.0.2 — 2026-06-11 {#v1-0-2}

### Added
- `datacontract edit` now accepts a URL: it asks to download a local copy and opens that in the editor, using the configured API key (e.g., `ENTROPY_DATA_API_KEY`) to fetch the file.
- BigQuery: added support for a separate billing/compute project via the `DATACONTRACT_BIGQUERY_BILLING_PROJECT` environment variable. When set, query jobs are submitted to (and billed against) the billing project while data is read from the `project` specified in the server config. This enables cross-project and cross-organisation validation without requiring `bigquery.jobUser` on the data project.
### Fixed
- `datacontract test` no longer fails on Spark/Databricks tables containing columns of types ibis cannot represent, such as VariantType ([#1296](https://github.com/datacontract/datacontract-cli/issues/1296))
- The `postgres` and `redshift` extras now install `psycopg[binary]`, whose wheels bundle the libpq client library. Previously, `datacontract test` against PostgreSQL/Redshift failed on machines without PostgreSQL client libraries installed (`couldn't import psycopg: no pq wrapper available`).

## 1.0.1 — 2026-06-10 {#v1-0-1}

### Added
- Environment variables (e.g., credentials for `datacontract test`) are now also loaded from a `.env` file in the current working directory, walking up parent directories until one is found. Already-set environment variables take precedence over `.env` values, so exported variables and CI secrets keep working unchanged. ([#1295](https://github.com/datacontract/datacontract-cli/issues/1295))
- `datacontract edit <file>` opens a local data contract file in the [Data Contract Editor](https://github.com/datacontract/datacontract-editor) (web UI). Starts a local web server (default port 4243, requires the `api` extra) that serves the editor, writes saves directly back to the file, and doubles as the editor's test runner: "Run test" in the editor executes the data contract tests locally via the server's own `/test` endpoint. The editor is bundled with the CLI and works offline; `--editor-version` loads a specific editor version from the CDN instead, `--editor-assets-url` a self-hosted build. If the file does not exist, the command offers to initialize a new data contract (same template as `datacontract init`). Saving gives feedback in the editor and on the console.

### Fixed
- `datacontract test` no longer fails with `Compilation rule for RegexSearch operation is not defined` when a contract uses `logicalTypeOptions.pattern` (or `pattern`) against SQL Server. SQL Server has no native regex operator, so the ibis mssql backend cannot compile `re_search`; pattern checks now fall back to a `PATINDEX(...) > 0` LIKE match, restoring the behaviour of the former soda-core engine. Patterns that use real regex syntax (anchors, quantifiers, groups, `.`) cannot be expressed as a T-SQL LIKE pattern and now raise a clear error instead of failing cryptically. ([#1284](https://github.com/datacontract/datacontract-cli/issues/1284))

## 1.0.0 — 2026-06-04 {#v1-0-0}

### Breaking Changes
- Replaced the Soda Core quality/test engine with [ibis](https://ibis-project.org/). `datacontract test` now compiles schema and quality checks into ibis expressions (dialect-correct SQL per backend via sqlglot, local/remote files via DuckDB) instead of generating SodaCL. Install extras now pull `ibis-framework[<backend>]` instead of `soda-core-*`. Check semantics and pass/fail results are preserved for the supported sources (postgres, redshift, mysql, snowflake, bigquery, databricks, sqlserver, oracle, trino, athena, impala, kafka/dataframe via the ibis Spark backend, and local/S3/GCS/Azure files).
- Raw SodaCL custom quality checks (`quality.type: custom` with `engine: soda`) are no longer executed and now report a warning. Migrate them to `quality.type: sql` or a library metric (e.g. `metric: rowCount`).

### Added
- Python 3.13 and 3.14 support (`requires-python` now allows 3.10–3.14). On 3.13/3.14 the Spark-backed extras resolve to PySpark 4.0 (Spark 3.5 has no 3.13+ build); the Kafka/Avro connector jars already adapt to the runtime Spark/Scala version. `create_spark_session` now pins `PYSPARK_PYTHON`/`PYSPARK_DRIVER_PYTHON` to the running interpreter so Spark's Python workers match the driver.
- `datacontract test` against Databricks now supports more authentication methods beyond the personal access token (`DATACONTRACT_DATABRICKS_TOKEN`): an OAuth service principal for machine-to-machine auth (`DATACONTRACT_DATABRICKS_CLIENT_ID` + `DATACONTRACT_DATABRICKS_CLIENT_SECRET`), a local config profile via the Databricks SDK unified auth (`DATACONTRACT_DATABRICKS_PROFILE`, also covers Azure CLI/MSI), and an explicit connector auth type (`DATACONTRACT_DATABRICKS_AUTH_TYPE`, e.g. `databricks-oauth` for the interactive browser flow).
- `datacontract test` now records structured `diagnostics` on each check explaining why it passed or failed: the metric, measured value, threshold, and (for "bad row" metrics) the total row count and failed fraction. `invalid_count` checks also report the validity rule they enforced (e.g. `{"max_length": 20}`, `{"pattern": "^.+@.+$"}`). The diagnostics surface in the JSON output and the JUnit failure text. This replaces the Soda-specific `diagnostics` payload that the ibis migration had left unpopulated.
- `datacontract test` now honors ODCS `quality.unit: percent` on the count-of-bad-rows library metrics (`nullValues`, `missingValues`, `invalidValues`). The threshold is then compared against the failed fraction (0–100) of the model row count instead of an absolute count, so e.g. `metric: nullValues`, `unit: percent`, `mustBeLessThan: 5` passes when fewer than 5% of rows are null. Percent on metrics where a row fraction has no meaning (`rowCount`, `duplicateValues`) logs a warning and falls back to the absolute count. The measured `percent` is added to the check diagnostics.
- `datacontract test` now honors ODCS `quality.severity`: a non-blocking severity (`info`, `warning`, `low`, `minor`, `trivial`) downgrades a failing quality check to a `warning` instead of a `failed`, so it no longer fails the run. Any other severity (or none) still fails. The severity is recorded in the check diagnostics.
- `datacontract test --include-failed-samples` collects a small sample of the rows that failed each `missing`/`invalid`/`duplicate` check (off by default). Each sample is restricted to the contract's identifier columns (`unique` / `primaryKey` fields) plus the offending column; duplicate checks report the duplicated key values and their counts. Columns whose ODCS `classification` marks them sensitive (`pii`, `personal`, `confidential`, `restricted`, `sensitive`, `secret`) are omitted. Samples are capped at 5 rows per check and surface on `Check.failed_samples` in the JSON output and in the JUnit failure text. This is local-only and needs no Soda Cloud (soda-core itself collects failed-row samples only via Soda Cloud).

### Changed
- MySQL is tested through DuckDB's `mysql` extension instead of ibis's native MySQL backend, so the `mysql` extra stays pure-pip (no `mysqlclient` C build / system MySQL client libraries required).
- Bumped DuckDB to the 1.5.x line (from 1.0.x) with the bundled `duckdb-extension-*` wheels (httpfs/aws/azure) pinned in lockstep. The 1.5.x extension wheels publish arm64 Linux builds, so air-gapped installs on arm64 Linux are now supported (the previous platform skip markers were removed).
- The Kafka/Avro Spark connector jars are now derived from the installed PySpark at runtime (Spark version + Scala binary version), so both PySpark 3.5.x (Scala 2.12) and 4.x (Scala 2.13) work. The `kafka` and `databricks` extras allow `pyspark<5.0`.
- `import protobuf` now uses the pure-Python `proto-schema-parser` instead of the `protoc` system binary. The `protobuf` extra no longer requires `protoc` (or the `protobuf` runtime), so `.proto` import works out of the box, including transitive imports across subdirectories.
- Container image is now based on [Docker Hardened Images](https://hub.docker.com/hardened-images/catalog/dhi/python): signed, ships SBOM/VEX, and has tighter CVE patch SLAs than upstream Debian. Runs as nonroot (uid 65532) instead of root. `pip` / `uv` installs at build time are routed through Socket Firewall Free, which blocks malicious dependencies. ([#1275](https://github.com/datacontract/datacontract-cli/issues/1275), [#1277](https://github.com/datacontract/datacontract-cli/issues/1277))
- Container image now ships Eclipse Temurin JRE 17, so PySpark-backed engines (Kafka, Spark) actually run inside the image — previously they failed at `SparkSession` startup because the base image had no JVM. End users pulling `datacontract/cli` are unaffected by the build-side changes. ([#1277](https://github.com/datacontract/datacontract-cli/issues/1277))

### Fixed
- DuckDB S3 secret creation no longer fails (`Secret Validation Failure`) on DuckDB ≥1.5: explicit `KEY_ID`/`SECRET` now use the default `config` provider instead of `CREDENTIAL_CHAIN`.
- `import csv` no longer fails with a DuckDB binder error on DuckDB ≥1.5 (the uniqueness probe now uses `count(DISTINCT ...)` via SQL).

### Removed
- The `soda-core` runtime dependency and all `soda-core-*` install extras, plus the `setuptools` runtime shim they required. The `sodacl` export format (`datacontract export sodacl`) is unchanged and is now generated independently of any Soda runtime.
- The unused `details` field on test-result checks (`Run.checks[].details`). It was a Soda-era placeholder that was never populated; the new structured `diagnostics` field replaces it. The JUnit failure text no longer prints a `Details:` line.

## 0.12.5 — 2026-05-30 {#v0-12-5}

### Added
- Resolve `authoritativeDefinitions[type=definition]` on schema properties, filling fields the contract author left unset ([#1261](https://github.com/datacontract/datacontract-cli/issues/1261))
  - **Breaking:** Per default, any resolution failure of `authoritativeDefinitions[type in {definition, semantics}]` now rejects the contract on `lint`, `test`, `ci`, `export`, and `changelog`. ([#1261](https://github.com/datacontract/datacontract-cli/issues/1261))
  - Resolve `authoritativeDefinitions[type=semantics]` (and the legacy `type=semantic`) the same way. A `url:` that points at the configured entropy-data host is fetched directly; a `url:` that's an IRI (host doesn't match) is routed through `GET /api/semantics?iri=...` on the configured host, which uses the API key's organization to resolve. ([#1262](https://github.com/datacontract/datacontract-cli/issues/1262))
  - `--no-inline-references` flag to skip the HTTP fetch ([#1261](https://github.com/datacontract/datacontract-cli/issues/1261))
- When `--json-schema` points at a custom JSON Schema, the ODCS Pydantic step now accepts extra top-level fields the schema allows ([#1266](https://github.com/datacontract/datacontract-cli/issues/1266))

### Changed
- JSON Schema validation errors now identify the offending schema and property by name instead of array indices ([#1255](https://github.com/datacontract/datacontract-cli/issues/1255) [@dmaresma](https://github.com/dmaresma))
- `test`, `lint`, and `ci` now infer `--output-format` from the `--output` file extension when not given (`.json` → json, `.xml` → junit) ([#1156](https://github.com/datacontract/datacontract-cli/issues/1156) [@dallylee](https://github.com/dallylee))

### Fixed
- Schema type check no longer fails for `varchar(n)` columns on Databricks with PySpark 4.0+, and for `map` and `varchar` types nested inside `struct` columns; affected columns emit a warning and skip the type check instead ([#1219](https://github.com/datacontract/datacontract-cli/issues/1219),[#1245](https://github.com/datacontract/datacontract-cli/issues/1245) [@IchEssBlumen](https://github.com/IchEssBlumen))
- `WARNING`/`ERROR` log messages are no longer hidden by default for `import`, `export`, `changelog`, `catalog`, `dbt`, and `publish` ([#1264](https://github.com/datacontract/datacontract-cli/issues/1264))
- `datacontract test` against S3, GCS, and Azure no longer fails with `Failed to download extension` in air-gapped containers. The required DuckDB extensions are now bundled via the `s3`/`gcs`/`azure` install extras ([#1191](https://github.com/datacontract/datacontract-cli/issues/1191) [@ParenParikh](https://github.com/ParenParikh))
- JSON/CSV/Parquet with DuckDB: `field_is_present` check now correctly detects missing columns ([#1065](https://github.com/datacontract/datacontract-cli/issues/1065),[#1163](https://github.com/datacontract/datacontract-cli/issues/1163) [@hieusats](https://github.com/hieusats))
- `import dbt --model <name>.vN` now correctly imports the specified version of a versioned dbt model. Previously the filter compared the full `name.vN` string against `node["name"]` (which is always the bare base name), producing a silent empty contract ([#1249](https://github.com/datacontract/datacontract-cli/issues/1249) [@willbowditch](https://github.com/willbowditch))

## 0.12.4 — 2026-05-21 {#v0-12-4}

### Added
- `datacontract test` now logs the Data Contract CLI version and whether it ran as a local CLI or through the FastAPI server (including the request URL) as part of the test result logs

### Fixed
- Schema checks now resolve each property by its `physicalName` when set (falling back to `name`), matching the existing table-level resolution and the SQL/BigQuery exporters. Previously a property whose logical `name` differed from its physical column (e.g. `name: brand` with `physicalName: BRAND`) failed the presence and type checks even though the physical column existed ([#1246](https://github.com/datacontract/datacontract-cli/issues/1246))

## 0.12.3 — 2026-05-18 {#v0-12-3}

### Added
- new `datacontract dbt sync` command: generate dbt tests from an ODCS contract, then run `dbt test` for them, and optionally publish the results to Entropy Data ([#1222](https://github.com/datacontract/datacontract-cli/issues/1222), [#1235](https://github.com/datacontract/datacontract-cli/issues/1235))
- `redshift` server type for `datacontract test` (requires `pip install datacontract-cli[redshift]`). ([#1236](https://github.com/datacontract/datacontract-cli/issues/1236))

### Fixed
- SQL type converter: emit canonical `decimal`/`numeric` per dialect (Postgres → `numeric`, MySQL → `decimal`) so `test`'s column-type check matches `information_schema` ([#1237](https://github.com/datacontract/datacontract-cli/issues/1237))

## 0.12.2 — 2026-05-05 {#v0-12-2}

### Added
- `impala` extra (`pip install datacontract-cli[impala]`) — pulls in `soda-core-impala`. Impala engine support landed in [#965](https://github.com/datacontract/datacontract-cli/issues/965) but the install extra was never added; users had to install `soda-core-impala` manually. Also included in `[all]`.

### Changed
- **breaking:** drop the `dbt` extra and the `dbt-core` dependency. `import dbt` now reads `manifest.json` directly with no third-party dependency, and works without installing any extra. Minimum supported manifest schema version is v9 (dbt 1.5+). Users who installed `datacontract-cli[dbt]` should switch to plain `datacontract-cli`.
- **breaking:** the `protobuf` extra now requires the `protoc` compiler installed on the system. Replaces the bundled `grpcio-tools` (~50 MB of platform-specific protoc binaries) with the lighter `protobuf` runtime (`>=3.20,<7.0`). `import protobuf` raises a clear error with platform-specific install hints if `protoc` is not on `PATH`. Install with `brew install protobuf` (macOS), `sudo apt install protobuf-compiler` (Debian/Ubuntu), etc. — see README.

### Fixed
- README install table: add missing `csv`, `excel`, and `oracle` extras. The matching `[project.optional-dependencies]` entries already existed but were undocumented.
- quality: support `{object}` and `${object}` placeholder in SQL quality queries as the ODCS-spec name for the current schema object (alias for `{model}`/`{table}`) ([#676](https://github.com/datacontract/datacontract-cli/issues/676))
- `changelog` command help text now advertises `(url or path)` for V1/V2 arguments, clarifying that HTTP/HTTPS URLs are accepted ([#1162](https://github.com/datacontract/datacontract-cli/issues/1162))
- **breaking:** `test` command now exits non-zero when a server is specified, but soda-core fails to connect or authenticate ([#1181](https://github.com/datacontract/datacontract-cli/issues/1181))
- correct swapped `check_type` labels  `model_qualty_sql` and `field_quality_sql` ([#1187](https://github.com/datacontract/datacontract-cli/issues/1187))
- `import spark` now emits a native Spark SQL physicalType (e.g. `string`) instead of Python repr (e.g. `StringType()`). Contracts imported using Spark in v0.11.0–v0.12.1 did not perform type checks and must be re-imported. ([#1048](https://github.com/datacontract/datacontract-cli/issues/1048))
- Re-add `setuptools` as a base dependency. soda-core's `env_helper.py` imports `from distutils.util import strtobool`; `distutils` was removed from stdlib in Python 3.12 and stripped from python-build-standalone 3.11 builds. `setuptools` provides the `distutils` shim. Previously pulled in transitively via `grpcio-tools`; now required explicitly. Reverts [#1199](https://github.com/datacontract/datacontract-cli/issues/1199) — see soda-core#2091.
- SLA freshness checks now quote column identifiers with special characters ([#1202](https://github.com/datacontract/datacontract-cli/issues/1202))
- update field / model quotation for Impala, dataframe, and Kafka ([#1202](https://github.com/datacontract/datacontract-cli/issues/1202))

## 0.12.1 — 2026-04-21 {#v0-12-1}

### Fixed
- make `--schema` a deprecated alias for `--json-schema` to (will be removed in v0.13.0)

## 0.12.0 — 2026-04-20 {#v0-12-0}

This release introduces several changes to improve the usability of `datacontract-cli` for AI Agents.

- **Breaking**: Several changes in the CLI syntax ([#1157](https://github.com/datacontract/datacontract-cli/issues/1157)):
> Fix in v0.12.1: re-added `--schema` as alias for the new `--json-schema` (will be removed in v0.13.0)

| Command                                    | Old option                                    | New option                             |
  |--------------------------------------------|-----------------------------------------------|----------------------------------------|
  | `lint`, `test`, `ci`, `publish`, `catalog` | `--schema <PATH>` (will work until v0.13.0)   | `--json-schema <PATH>`                 |
  | `export`, `import`                         | `--format <FORMAT> <OPTIONS>`                 | `<FORMAT> <OPTIONS>` (drop `--format`) |
  | **Export options:**                        |                                               |                                        |
  | `export --format dbt`                      | `--format dbt`                                | `dbt-models` (format renamed)          |
  | `export --format great-expectations`       | `--sql-server-type <TYPE>`                    | `--dialect <TYPE>`                     |
  | `export --format rdf`                      | `--rdf-base <URI>`                            | `--base <URI>`                         |
  | `export --format sql`                      | `--sql-server-type <TYPE>`                    | `--dialect <TYPE>`                     |
  | `export --format sql-query`                | `--sql-server-type <TYPE>`                    | `--dialect <TYPE>`                     |
  | **Import options:**                        |                                               |                                        |
  | `import --format bigquery`                 | `--bigquery-[project\|dataset\|table] <NAME>` | `--[project\|dataset\|table] <NAME>`   |
  | `import --format dbt`                      | `--dbt-model <NAME>`                          | `--model <NAME>`                       |
  | `import --format glue`                     | `--source <NAME>`, `--glue-table <NAME>`      | `--database <NAME>`, `--table <NAME>`  |
  | `import --format iceberg`                  | `--iceberg-table <NAME>`                      | `--table <NAME>`                       |
  | `import --format unity`                    | `--unity-table-full-name <NAME>`              | `--table <NAME>`                       |
  | `import --format spark`                    | `--source <NAMES>`                            | `--tables <NAMES>`                     |
  | `import`                                   | `--template`                                  | dropped (was a no-op)                  |

The `--schema` option (referring to the ODCS JSON schema) was renamed to `--json-schema` to avoid confusion with `--schema-name`, which refers to the schema within the data contract to test for.

- Error messages for uncaught exceptions are shortened now. Pass `--debug` (or set `DATACONTRACT_CLI_DEBUG=1`) to see the full traceback. ([#1175](https://github.com/datacontract/datacontract-cli/issues/1175))
- Add example calls to `--help` outputs ([#1176](https://github.com/datacontract/datacontract-cli/issues/1176))
- Add explicit errors when required env vars for soda connections are missing ([#1177](https://github.com/datacontract/datacontract-cli/issues/1177))
- Validate some of the CLI options against their allowed values instead of accepting any string ([#1178](https://github.com/datacontract/datacontract-cli/issues/1178))

## 0.11.9 — 2026-04-20 {#v0-11-9}

### Added
- Added `--checks` option to `test` command to selectively run check categories: `schema`, `quality`, `servicelevel` ([#678](https://github.com/datacontract/datacontract-cli/issues/678))
- Added `--schema-name` option to `test` command to test a specific schema instead of all schemas ([#1079](https://github.com/datacontract/datacontract-cli/issues/1079),[#1085](https://github.com/datacontract/datacontract-cli/issues/1085) [@kelsoufi-sanofi](https://github.com/kelsoufi-sanofi))

### Fixed
- Move `precision`/`scale` for `number` types from `logicalTypeOptions` to `customProperties` ([#1145](https://github.com/datacontract/datacontract-cli/issues/1145),[#1160](https://github.com/datacontract/datacontract-cli/issues/1160) [@davidb-tada](https://github.com/davidb-tada))
- Emit placeholder server values in SQL importer so generated contracts pass lint ([#1146](https://github.com/datacontract/datacontract-cli/issues/1146),[#1152](https://github.com/datacontract/datacontract-cli/issues/1152) [@Ai-chan-0411](https://github.com/Ai-chan-0411))
- Fix Protobuf export for arrays of objects and improve message/enum naming to UpperCamelCase ([#1012](https://github.com/datacontract/datacontract-cli/issues/1012) [@Schokuroff](https://github.com/Schokuroff))
- Exit with code 1 when `--server` name is not found ([#1153](https://github.com/datacontract/datacontract-cli/issues/1153),[#1161](https://github.com/datacontract/datacontract-cli/issues/1161) [@Ai-chan-0411](https://github.com/Ai-chan-0411))

Thanks to [@kelsoufi-sanofi](https://github.com/kelsoufi-sanofi) for the new `--schema-name` option on `test`, and to [@Schokuroff](https://github.com/Schokuroff), [@Ai-chan-0411](https://github.com/Ai-chan-0411), and [@davidb-tada](https://github.com/davidb-tada) for their contributions.

## 0.11.8 — 2026-04-10 {#v0-11-8}

### Added
- Added `ci` command for CI/CD-optimized test runs: multi-file support, GitHub Actions annotations and step summary, Azure DevOps annotations, `--fail-on` flag, `--json` output ([#1114](https://github.com/datacontract/datacontract-cli/issues/1114))
- Added `changelog` command and API endpoint ([#1118](https://github.com/datacontract/datacontract-cli/issues/1118) [@davidb-tada](https://github.com/davidb-tada))
- Added opt-in `--all-errors` mode for `datacontract lint` to report all JSON Schema validation errors, with matching `all_errors` support in the Python library and API ([#1125](https://github.com/datacontract/datacontract-cli/issues/1125) [@jmbenedetto](https://github.com/jmbenedetto))
- Added `--schema-name` option to custom model export ([#978](https://github.com/datacontract/datacontract-cli/issues/978) [@AntoineGiraud](https://github.com/AntoineGiraud))

### Fixed
- Avro importer now raises an error for union fields with multiple non-null types, which are not supported by ODCS ([#1124](https://github.com/datacontract/datacontract-cli/issues/1124))
- Fix SQL export generating multiple PRIMARY KEY constraints for composite keys ([#1026](https://github.com/datacontract/datacontract-cli/issues/1026),[#1092](https://github.com/datacontract/datacontract-cli/issues/1092) [@barry0451](https://github.com/barry0451) [@dwestheide](https://github.com/dwestheide))
- Preserve parametrized physicalTypes for SQL export ([#1086](https://github.com/datacontract/datacontract-cli/issues/1086),[#1093](https://github.com/datacontract/datacontract-cli/issues/1093) [@barry0451](https://github.com/barry0451) [@alexander-griesbeck](https://github.com/alexander-griesbeck))
- Fix incorrect SQL type mappings: SQL Server `double`/`jsonb`, MySQL bare `varchar`, missing Trino types ([#1110](https://github.com/datacontract/datacontract-cli/issues/1110))
- Fix markdown export breaking table structure when extra field values contain pipe characters ([#832](https://github.com/datacontract/datacontract-cli/issues/832),[#1117](https://github.com/datacontract/datacontract-cli/issues/1117) [@barry0451](https://github.com/barry0451) [@grepwood](https://github.com/grepwood))
- Fix dbt import using incorrect physicalType instead of actual materialization type ([#1136](https://github.com/datacontract/datacontract-cli/issues/1136))
- Remove unnecessary numpy dependency from databricks and kafka extras ([#1135](https://github.com/datacontract/datacontract-cli/issues/1135) [@kayhendriksen](https://github.com/kayhendriksen))

Special thanks to [@davidb-tada](https://github.com/davidb-tada) for the outstanding contribution of the new `changelog` command and API endpoint! Also thanks to [@barry0451](https://github.com/barry0451) for multiple quality fixes across the SQL exporter and markdown export, and to [@AntoineGiraud](https://github.com/AntoineGiraud) and [@jmbenedetto](https://github.com/jmbenedetto) for their feature contributions.

## 0.11.7 — 2026-03-24 {#v0-11-7}

### Fixed
- Escape single quotes in string values for SodaCL checks ([#1090](https://github.com/datacontract/datacontract-cli/issues/1090))
- Escape BigQuery field and model names with backticks for SodaCL checks ([#736](https://github.com/datacontract/datacontract-cli/issues/736))
- Escape Databricks model names with backticks for SodaCL checks
- Fixed catalog export SpecView not having a tags property for the index.html template ([#1059](https://github.com/datacontract/datacontract-cli/issues/1059))
- Fix SQL importer type mappings: binary types, datetime/time, uuid now map to correct ODCS logicalType and format ([#790](https://github.com/datacontract/datacontract-cli/issues/790))

### Added
- Added support for MySQL for data contract tests ([#1101](https://github.com/datacontract/datacontract-cli/issues/1101))
- Support additional PyArrow types in Parquet importer ([#1091](https://github.com/datacontract/datacontract-cli/issues/1091))
- Populate `logicalTypeOptions.format` for SQL import from binary and uuid types ([#790](https://github.com/datacontract/datacontract-cli/issues/790))
- Snowflake DDL import with tags, descriptions, and template variable handling ([#790](https://github.com/datacontract/datacontract-cli/issues/790))

## 0.11.6 — 2026-03-17 {#v0-11-6}

### Fixed
- Fix parser error for CSV / Parquet table names containing special characters ([#1066](https://github.com/datacontract/datacontract-cli/issues/1066))
- Fix BigQuery export failing with "Unsupported type" for parameterized physicalType like `NUMERIC(18, 4)` ([#1083](https://github.com/datacontract/datacontract-cli/issues/1083))

### Added
- Added JSON output format for test results (`--output-format json`)
- Added Azure AD / Entra ID authentication support for SQL Server and Microsoft Fabric

## 0.11.5 — 2026-02-19 {#v0-11-5}

### Fixed

- Fix BigQuery import for repeated fields ([#1017](https://github.com/datacontract/datacontract-cli/issues/1017))
- Make Markdown export compatible with XHTML by replacing `` with `` ([#1030](https://github.com/datacontract/datacontract-cli/issues/1030))
- Add ADC/WIF and impersonation support for BigQuery ([#1064](https://github.com/datacontract/datacontract-cli/issues/1064))
- Fix Snowflake quoted identifiers by enabling double-quote quoting ([#1053](https://github.com/datacontract/datacontract-cli/issues/1053))
- Fix retention duration crash for numeric ODCS values ([#1051](https://github.com/datacontract/datacontract-cli/issues/1051))
- Fix physicalType bypass for precision and scale conversion ([#1043](https://github.com/datacontract/datacontract-cli/issues/1043))
- Fix mkdir TOCTOU race causing silent JUnit write failure ([#1050](https://github.com/datacontract/datacontract-cli/issues/1050))
- Fix validation failure for field names with special chars on Databricks ([#1049](https://github.com/datacontract/datacontract-cli/issues/1049))
- Add Azure support for field name quoting in schema checks ([#1025](https://github.com/datacontract/datacontract-cli/issues/1025))

## 0.11.4 — 2026-01-19 {#v0-11-4}

### Changed

- Made `duckdb` an optional dependency. Install with `pip install datacontract-cli[duckdb]` for local/S3/GCS/Azure file testing.
- Removed unused `fastparquet` and `numpy` core dependencies.

### Added

- Include searchable tags in catalog index.html

### Fixed

- Fixed example(s) field mapping for Data Contract Specification importer ([#992](https://github.com/datacontract/datacontract-cli/issues/992)).
- Spark exporter now supports decimal precision/scale via `customProperties` or parsing from `physicalType` (e.g., `decimal(10,2)`) ([#996](https://github.com/datacontract/datacontract-cli/issues/996))
- Fix catalog/HTML export failing on ODCS contracts with no schema or no properties ([#971](https://github.com/datacontract/datacontract-cli/issues/971))

## 0.11.3 — 2026-01-10 {#v0-11-3}

### Fixed

- Fix `datacontract init` to generate ODCS format instead of deprecated Data Contract Specification ([#984](https://github.com/datacontract/datacontract-cli/issues/984))
- Fix ODCS lint failing on optional relationship `type` field by updating open-data-contract-standard to v3.1.2 ([#971](https://github.com/datacontract/datacontract-cli/issues/971))
- Restrict DuckDB dependency to < 1.4.0 ([#972](https://github.com/datacontract/datacontract-cli/issues/972))
- Fixed schema evolution support for optional fields in CSV and Parquet formats. Optional fields marked with `required: false` are no longer incorrectly treated as required during validation, enabling proper schema evolution where optional fields can be added to contracts without breaking validation of historical data files ([#977](https://github.com/datacontract/datacontract-cli/issues/977))
- Fixed decimals in pydantic model export. Fields marked with `type: decimal` will be mapped to `decimal.Decimal` instead of `float`.
- Fix BigQuery test failure for fields with FLOAT or BOOLEAN types by mapping them to equivalent types (BOOL and FLOAT64)

## 0.11.2 — 2025-12-15 {#v0-11-2}

### Added
- Add Impala engine support for Soda scans via ODCS `impala` server type.
### Fixed
- Restrict DuckDB dependency to < 1.4.0 ([#972](https://github.com/datacontract/datacontract-cli/issues/972))

## 0.11.1 — 2025-12-14 {#v0-11-1}

This is a major release with breaking changes:
We switched the internal data model from [Data Contract Specification](https://datacontract-specification.com) to [Open Data Contract Standard](https://datacontract.com/#odcs) (ODCS).

Not all features that were available are supported in this version, as some features are not supported by the Open Data Contract Standard, such as:

- Internal definitions using `$ref` (you can refer to external definitions via `authoritativeDefinition`)
- Lineage (no real workaround, use customProperties or transformation object if needed)
- Support for different physical types (no real workaround, use customProperties if needed)
- Support for enums (use quality metric `invalidValues`)
- Support for properties with type map and defining `keys` and `values` (use logical type map)
- Support for `scale` and `precision` (define them in `physicalType`)

The reason for this change is that the Data Contract Specification is deprecated, we focus on best possible support for the Open Data Contract Standard.
We try to make this transition as seamless as possible. 
If you face issues, please open an issue on GitHub.

We continue support reading [Data Contract Specification](https://datacontract-specification.com) data contracts during v0.11.x releases until end of 2026.
To migrate existing data contracts to Open Data Contract Standard use this instruction: https://datacontract-specification.com/#migration

### Changed

- ODCS v3.1.0 is now the default format for all imports.
- Renamed `--model` option to `--schema-name` in the `export` command to align with ODCS terminology.
- Renamed exporter files from `*_converter.py` to `*_exporter.py` for consistency (internal change).

### Added

- If an ODCS slaProperty "freshness" is defined with a reference to the element (column), the CLI will now test freshness of the data.
- If an ODCS slaProperty "retention" is defined with a reference to the element (column), the CLI will now test retention of the data.
- Support for custom Soda quality checks in ODCS using `type: custom` and `engine: soda` with raw SodaCL implementation.

### Fixed

- Oracle: Fix `service_name` attribute access to use ODCS field name `serviceName`

### Removed

- The `breaking`, `changelog`, and `diff` commands are now deleted ([#925](https://github.com/datacontract/datacontract-cli/issues/925)).
- The `terraform` export format has been removed.

## 0.10.41 — 2025-12-02 {#v0-10-41}

### Changed

- Great Expectations export: Update to Great Expectations 1.x format ([#919](https://github.com/datacontract/datacontract-cli/issues/919))
  - Changed `expectation_suite_name` to `name` in suite output
  - Changed `expectation_type` to `type` in expectations
  - Removed `data_asset_type` field from suite output
  - **Breaking**: Users with custom quality definitions using `expectation_type` must update to use `type`

### Added

- test: Log server name and type in output ([#963](https://github.com/datacontract/datacontract-cli/issues/963))
- api: CORS is now enabled for all origins
- quality: Support `{schema}` and `${schema}` placeholder in SQL quality checks to reference the server's database schema ([#957](https://github.com/datacontract/datacontract-cli/issues/957))
- SQL Server: Support `DATACONTRACT_SQLSERVER_DRIVER` environment variable to specify the ODBC driver ([#959](https://github.com/datacontract/datacontract-cli/issues/959))
- Excel: Add Oracle server type support for Excel export/import ([#960](https://github.com/datacontract/datacontract-cli/issues/960))
- Excel: Add local/CSV server type support for Excel export/import ([#961](https://github.com/datacontract/datacontract-cli/issues/961))
- Excel Export: Complete server types (glue, kafka, postgres, s3, snowflake, sqlserver, custom)

### Fixed

- Protobuf import: Fix transitive imports across subdirectories ([#943](https://github.com/datacontract/datacontract-cli/issues/943))
- Protobuf export now works without error ([#951](https://github.com/datacontract/datacontract-cli/issues/951))
- lint: YAML date values (e.g., `2022-01-15`) are now kept as strings instead of being converted to datetime objects, fixing ODCS schema validation
- export: field annotation now matches to number/numeric/decimal types
- Excel: Server port is now correctly parsed as integer instead of string for all server types
- Excel: Remove invalid `table` and `view` fields from custom server import
- Fixed DuckDB DDL generation to use `JSON` type instead of invalid empty `STRUCT()` for objects without defined properties ([#940](https://github.com/datacontract/datacontract-cli/issues/940))

### Deprecated

- The `breaking`, `changelog`, and `diff` commands are now deprecated and will be removed in a future version ([#925](https://github.com/datacontract/datacontract-cli/issues/925))

## 0.10.40 — 2025-11-25 {#v0-10-40}

### Added

- Support for ODCS v3.1.0

## 0.10.39 — 2025-11-20 {#v0-10-39}

### Added

- Oracle DB: Client Directory for Connection Mode 'Thick' can now be specified in the `DATACONTRACT_ORACLE_CLIENT_DIR` environment variable ([#949](https://github.com/datacontract/datacontract-cli/issues/949))

### Fixed

- Import composite primary keys from open data contract spec

## 0.10.38 — 2025-11-11 {#v0-10-38}

### Added

- Support for Oracle Database (>= 19C)

### Fixed

- Athena: Now correctly uses the (optional) AWS session token specified in the `DATACONTRACT_S3_SESSION_TOKEN' environment variable when testing contracts ([#934](https://github.com/datacontract/datacontract-cli/issues/934))

## 0.10.37 — 2025-11-03 {#v0-10-37}

### Added

- import: Support for nested arrays in odcs v3 importer
- lint: ODCS schema is now checked before converting
- --debug flag for all commands

### Fixed

- export: Excel exporter now exports critical data element

## 0.10.36 — 2025-10-17 {#v0-10-36}

### Added

- Support for Data Contract Specification v1.2.1 (Data Quality Metrics)
- Support for decimal testing in spark and databricks ([#902](https://github.com/datacontract/datacontract-cli/issues/902))
- Support for BigQuery Flexible Schema in Data Contract Checks ([#909](https://github.com/datacontract/datacontract-cli/issues/909))

### Changed

- `DataContract().import_from_source()` as an instance method is now deprecated. Use `DataContract.import_from_source()` as a class method instead.

### Fixed

- Export to DQX: Correct DQX format for global-level quality check of data contract export. ([#877](https://github.com/datacontract/datacontract-cli/issues/877))
- Import the table tags from a open data contract spec v3 ([#895](https://github.com/datacontract/datacontract-cli/issues/895))
- dbt export: Enhanced model-level primaryKey support with automatic test generation for single and multiple column primary keys ([#898](https://github.com/datacontract/datacontract-cli/issues/898))
- ODCS: field discarded when no logicalType defined  ([#891](https://github.com/datacontract/datacontract-cli/issues/891))
 
### Removed

- Removed specific linters, as the linters did not support ODCS ([#913](https://github.com/datacontract/datacontract-cli/issues/913))

## 0.10.35 — 2025-08-25 {#v0-10-35}

### Added

- Export to DQX : datacontract export --format dqx ([#846](https://github.com/datacontract/datacontract-cli/issues/846))
- API `/test` endpoint now supports `publish_url` parameter to publish test results to a URL. ([#853](https://github.com/datacontract/datacontract-cli/issues/853))
- The Spark importer and exporter now also exports the description of columns via the additional metadata of StructFields ([#868](https://github.com/datacontract/datacontract-cli/issues/868))

### Fixed

- Improved regex for extracting Azure storage account names from URLs with containerName@storageAccountName format ([#848](https://github.com/datacontract/datacontract-cli/issues/848))
- JSON Schema Check: Add globbing support for local JSON files
- Fixed server section rendering for markdown exporter

## 0.10.34 — 2025-08-06 {#v0-10-34}

### Added

- `datacontract test` now supports HTTP APIs.
- `datacontract test` now supports Athena.

### Fixed

- Avro Importer: Optional and required enum types are now supported ([#804](https://github.com/datacontract/datacontract-cli/issues/804))

## 0.10.33 — 2025-07-29 {#v0-10-33}

### Added
- Export to Excel: Convert ODCS YAML to Excel https://github.com/datacontract/open-data-contract-standard-excel-template ([#742](https://github.com/datacontract/datacontract-cli/issues/742))
- Extra properties in Markdown export. ([#842](https://github.com/datacontract/datacontract-cli/issues/842))

## 0.10.32 — 2025-07-28 {#v0-10-32}

### Added
- Import from Excel: Support the new quality sheet

### Fixed
- JUnit Test Report: Fixed incorrect syntax on handling warning test report. ([#833](https://github.com/datacontract/datacontract-cli/issues/833))

## 0.10.31 — 2025-07-18 {#v0-10-31}

### Added
- Added support for Variant with Spark exporter, data_contract.test(), and import as source unity catalog ([#792](https://github.com/datacontract/datacontract-cli/issues/792))

## 0.10.30 — 2025-07-15 {#v0-10-30}

### Fixed
- Excel Import should return ODCS YAML ([#829](https://github.com/datacontract/datacontract-cli/issues/829))
- Excel Import: Missing server section when the server included a schema property ([#823](https://github.com/datacontract/datacontract-cli/issues/823))

### Changed
- Use `&#x2007;` instead of `&numsp;` for tab in Markdown export.

## 0.10.29 — 2025-07-06 {#v0-10-29}

### Added
- Support for Data Contract Specification v1.2.0
- `datacontract import --format json`: Import from JSON files

### Changed
- `datacontract api [OPTIONS]`: Added option to pass extra arguments for `uvicorn.run()`

### Fixed
- `pytest tests\test_api.py`: Fixed an issue where special characters were not read correctly from file.
- `datacontract export --format mermaid`: Fixed an issue where the `mermaid` export did not handle references correctly

## 0.10.28 — 2025-06-05 {#v0-10-28}

### Added
- Much better ODCS support
    - Import anything to ODCS via the `import --spec odcs` flag
    - Export to HTML with an ODCS native template via `export --format html`
    - Export to Mermaid with an ODCS native mapping via `export --format mermaid`
- The databricks `unity` importer now supports more than a single table. You can use `--unity-table-full-name` multiple times to import multiple tables. And it will automatically add a server with the catalog and schema name.

### Changed
- `datacontract catalog [OPTIONS]`: Added version to contract cards in `index.html` of the catalog (enabled search by version)
- The type mapping of the `unity` importer no uses the native databricks types instead of relying on spark types. This allows for better type mapping and more accurate data contracts.

### Fixed

## 0.10.27 — 2025-05-22 {#v0-10-27}

### Added

- `datacontract export --format mermaid` Export
  to [Mermaid](https://mermaid-js.github.io/mermaid/#/) ([#767](https://github.com/datacontract/datacontract-cli/issues/767), [#725](https://github.com/datacontract/datacontract-cli/issues/725))

### Changed

- `datacontract export --format html`: Adding the mermaid figure to the html export
- `datacontract export --format odcs`: Export physical type to ODCS if the physical type is
  configured in config object
- `datacontract import --format spark`: Added support for spark importer table level comments ([#761](https://github.com/datacontract/datacontract-cli/issues/761))
- `datacontract import` respects `--owner` and `--id` flags ([#753](https://github.com/datacontract/datacontract-cli/issues/753))

### Fixed

- `datacontract export --format sodacl`: Fix resolving server when using `--server` flag ([#768](https://github.com/datacontract/datacontract-cli/issues/768))
- `datacontract export --format dbt`: Fixed DBT export behaviour of constraints to default to data tests when no model type is specified in the datacontract model

## 0.10.26 — 2025-05-16 {#v0-10-26}

### Changed
- Databricks: Add support for Variant type ([#758](https://github.com/datacontract/datacontract-cli/issues/758))
- `datacontract export --format odcs`: Export physical type if the physical type is configured in
  config object ([#757](https://github.com/datacontract/datacontract-cli/issues/757))
- `datacontract export --format sql` Include datacontract descriptions in the Snowflake sql export (
  [#756](https://github.com/datacontract/datacontract-cli/issues/756))

## 0.10.25 — 2025-05-07 {#v0-10-25}

### Added
- Extracted the DataContractSpecification and the OpenDataContractSpecification in separate pip modules and use them in the CLI.
- `datacontract import --format excel`: Import from Excel
  template https://github.com/datacontract/open-data-contract-standard-excel-template ([#742](https://github.com/datacontract/datacontract-cli/issues/742))

## 0.10.24 — 2025-04-19 {#v0-10-24}

### Added

- `datacontract test` with DuckDB: Deep nesting of json objects in duckdb ([#681](https://github.com/datacontract/datacontract-cli/issues/681))

### Changed

- `datacontract import --format csv` produces more descriptive output. Replaced
  using clevercsv with duckdb for loading and sniffing csv file.
- Updated dependencies

### Fixed

- Fix to handle logicalType format wrt avro mentioned in issue ([#687](https://github.com/datacontract/datacontract-cli/issues/687))
- Fix field type from TIME to DATETIME in BigQuery converter and schema ([#728](https://github.com/datacontract/datacontract-cli/issues/728))
- Fix encoding issues. ([#712](https://github.com/datacontract/datacontract-cli/issues/712))
- ODCS: Fix required in export and added item and fields format ([#724](https://github.com/datacontract/datacontract-cli/issues/724))

### Removed

- Deprecated QualityLinter is now removed

## 0.10.23 — 2025-03-03 {#v0-10-23}

### Added

- `datacontract test --output-format junit --output TEST-datacontract.xml` Export CLI test results
  to a file, in a standard format (e.g. JUnit) to improve CI/CD experience ([#650](https://github.com/datacontract/datacontract-cli/issues/650))

- Added import for `ProtoBuf`
Code for proto to datacontract ([#696](https://github.com/datacontract/datacontract-cli/issues/696))

- `dbt` & `dbt-sources` export formats now support the optional `--server` flag to adapt the DBT column `data_type` to specific SQL dialects
- Duckdb Connections are now configurable, when used as Python library ([#666](https://github.com/datacontract/datacontract-cli/issues/666))
- export to avro format add map type

### Changed

- Changed Docker base image to python:3.11-bullseye
- Relax fastparquet dependency

### Fixed

- Unicode Encode Error when exporting data contract YAML to HTML
  ([#652](https://github.com/datacontract/datacontract-cli/issues/652))
- Fix multiline descriptions in the DBT export functionality
- Incorrectly parsing $ref values in definitions ([#664](https://github.com/datacontract/datacontract-cli/issues/664))
- Better error message when the server configuration is missing in a data contract ([#670](https://github.com/datacontract/datacontract-cli/issues/670))
- Improved default values in ODCS generator to avoid breaking schema validation ([#671](https://github.com/datacontract/datacontract-cli/issues/671))
- Updated ODCS v3 generator to drop the "is" prefix from fields like `isNullable` and `isUnique` ([#669](https://github.com/datacontract/datacontract-cli/issues/669))
- Fix issue when testing databricks server with ODCS format
- avro export fix float format

## 0.10.22 — 2025-02-20 {#v0-10-22}

### Added

- `datacontract test` now also executes tests for service levels freshness and retention ([#407](https://github.com/datacontract/datacontract-cli/issues/407))

### Changed

- `datacontract import --format sql` is now using SqlGlot as importer.
- `datacontract import --format sql --dialect <dialect>` Dialect can now to defined when importing
  SQL.

### Fixed

- Schema type checks fail on nested fields for Databricks spark ([#618](https://github.com/datacontract/datacontract-cli/issues/618))
- Export to Avro add namespace on field as optional configuration ([#631](https://github.com/datacontract/datacontract-cli/issues/631))

### Removed

- `datacontract test --examples`: This option was removed as it was not very popular and top-level
  examples section is deprecated in the Data Contract Specification v1.1.0 ([#628](https://github.com/datacontract/datacontract-cli/issues/628))
- Support for `odcs_v2` ([#645](https://github.com/datacontract/datacontract-cli/issues/645))

## 0.10.21 — 2025-02-06 {#v0-10-21}

### Added

- `datacontract export --format custom`: Export to custom format with Jinja
- `datacontract api` now can be protected with an API key

### Changed

- `datacontract serve` renamed to `datacontract api`

### Fixed

- Fix Error: 'dict object' has no attribute 'model_extra' when trying to use type: string with enum
  values inside an array ([#619](https://github.com/datacontract/datacontract-cli/issues/619))

## 0.10.20 — 2025-01-30 {#v0-10-20}

### Added

- datacontract serve: now has a route for testing data contracts
- datacontract serve: now has a OpenAPI documentation on root

### Changed

- FastAPI endpoint is now moved to extra "web"

### Fixed

- API Keys for Data Mesh Manager are now also applied for on-premise installations

## 0.10.19 — 2025-01-29 {#v0-10-19}

### Added

- datacontract import --format csv
- publish command also supports publishing ODCS format
- Option to separate physical table name for a model via config option ([#270](https://github.com/datacontract/datacontract-cli/issues/270))

### Changed
- JSON Schemas are now bundled with the application ([#598](https://github.com/datacontract/datacontract-cli/issues/598))
- datacontract export --format html: The model title is now shown if it is different to the model
  name ([#585](https://github.com/datacontract/datacontract-cli/issues/585))
- datacontract export --format html: Custom model attributes are now shown ([#585](https://github.com/datacontract/datacontract-cli/issues/585))
- datacontract export --format html: The composite primary key is now shown. ([#591](https://github.com/datacontract/datacontract-cli/issues/591))
- datacontract export --format html: now examples are rendered in the model and definition ([#497](https://github.com/datacontract/datacontract-cli/issues/497))
- datacontract export --format sql: Create arrays and struct for Databricks ([#467](https://github.com/datacontract/datacontract-cli/issues/467))

### Fixed
- datacontract lint: Linter 'Field references existing field' too many values to unpack (expected
  2) ([#586](https://github.com/datacontract/datacontract-cli/issues/586))
- datacontract test (Azure): Error querying delta tables from azure storage. ([#458](https://github.com/datacontract/datacontract-cli/issues/458))
- datacontract export --format data-caterer: Use `fields` instead of `schema`
- datacontract export --format data-caterer: Use `options` instead of `generator.options`
- datacontract export --format data-caterer: Capture array type length option and inner data type
- Fixed schemas/datacontract-1.1.0.init.yaml not included in build and `--template` not resolving file

## 0.10.18 — 2025-01-18 {#v0-10-18}

### Fixed
- Fixed an issue when resolving project's dependencies when all extras are installed.
- Definitions referenced by nested fields are not validated correctly ([#595](https://github.com/datacontract/datacontract-cli/issues/595))
- Replaced deprecated `primary` field with `primaryKey` in exporters, importers, examples, and Jinja templates for backward compatibility. Fixes [#518](https://github.com/your-repo/your-project/issues/518).
- Cannot execute test on column of type record(bigquery) [#597](https://github.com/datacontract/datacontract-cli/issues/597)

## 0.10.17 — 2025-01-16 {#v0-10-17}

### Added
- added export format **markdown**: `datacontract export --format markdown` ([#545](https://github.com/datacontract/datacontract-cli/issues/545))
- When importing in dbt format, add the dbt unique information as a datacontract unique field ([#558](https://github.com/datacontract/datacontract-cli/issues/558))
- When importing in dbt format, add the dbt primary key information as a datacontract primaryKey field ([#562](https://github.com/datacontract/datacontract-cli/issues/562))
- When exporting in dbt format, add the datacontract references field as a dbt relationships test ([#569](https://github.com/datacontract/datacontract-cli/issues/569))
- When importing in dbt format, add the dbt relationships test field as a reference in the data contract ([#570](https://github.com/datacontract/datacontract-cli/issues/570))
- Add serve command on README ([#592](https://github.com/datacontract/datacontract-cli/issues/592))

### Changed
- Primary and example fields have been deprecated in Data Contract Specification v1.1.0 ([#561](https://github.com/datacontract/datacontract-cli/issues/561))
- Define primaryKey and examples for model to follow the changes in datacontract-specification v1.1.0 ([#559](https://github.com/datacontract/datacontract-cli/issues/559))

### Fixed
- SQL Server: cannot escape reserved word on model ([#557](https://github.com/datacontract/datacontract-cli/issues/557))
- Export dbt-staging-sql error on multi models contracts ([#587](https://github.com/datacontract/datacontract-cli/issues/587))

### Removed
- OpenTelemetry publisher, as it was hardly used

## 0.10.16 — 2024-12-19 {#v0-10-16}

### Added
- Support for exporting a Data Contract to an Iceberg schema definition.
- When importing in dbt format, add the dbt `not_null` information as a datacontract `required` field ([#547](https://github.com/datacontract/datacontract-cli/issues/547))

### Changed
- Type conversion when importing contracts into dbt and exporting contracts from dbt ([#534](https://github.com/datacontract/datacontract-cli/issues/534))
- Ensure 'name' is the first column when exporting in dbt format, considering column attributes ([#541](https://github.com/datacontract/datacontract-cli/issues/541))
- Rename dbt's `tests` to `data_tests` ([#548](https://github.com/datacontract/datacontract-cli/issues/548))

### Fixed
- Modify the arguments to narrow down the import target with `--dbt-model` ([#532](https://github.com/datacontract/datacontract-cli/issues/532))
- SodaCL: Prevent `KeyError: 'fail'` from happening when testing with SodaCL
- fix: populate database and schema values for bigquery in exported dbt sources ([#543](https://github.com/datacontract/datacontract-cli/issues/543))
- Fixing the options for importing and exporting to standard output ([#544](https://github.com/datacontract/datacontract-cli/issues/544))
- Fixing the data quality name for model-level and field-level quality tests

## 0.10.15 — 2024-12-02 {#v0-10-15}

### Added
- Support for model import from parquet file metadata.
- Great Expectation export: add optional args ([#496](https://github.com/datacontract/datacontract-cli/issues/496))
  - `suite_name` the name of the expectation suite to export
  - `engine` used to run checks
  - `sql_server_type` to define the type of SQL Server to use when engine is `sql`
- Changelog support for `Info` and `Terms` blocks.
- `datacontract import` now has `--output` option for saving Data Contract to file
- Enhance JSON file validation (local and S3) to return the first error for each JSON object, the max number of total errors can be configured via the environment variable: `DATACONTRACT_MAX_ERRORS`. Furthermore, the primaryKey will be additionally added to the error message.
- fixes issue where records with no fields create an invalid bq schema.

### Changed
- Changelog support for custom extension keys in `Models` and `Fields` blocks.
- `datacontract catalog --files '*.yaml'` now checks also any subfolders for such files.
- Optimize test output table on console if tests fail

### Fixed
- raise valid exception in DataContractSpecification.from_file if file does not exist
- Fix importing JSON Schemas containing deeply nested objects without `required` array
- SodaCL: Only add data quality tests for executable queries

## 0.10.14 — 2024-10-26 {#v0-10-14}

Data Contract CLI now supports the Open Data Contract Standard (ODCS) v3.0.0.

### Added
- `datacontract test` now also supports ODCS v3 data contract format
- `datacontract export --format odcs_v3`: Export to Open Data Contract Standard v3.0.0 ([#460](https://github.com/datacontract/datacontract-cli/issues/460))
- `datacontract test` now also supports ODCS v3 anda Data Contract SQL quality checks on field and model level
- Support for import from Iceberg table definitions.
- Support for decimal logical type on avro export.
- Support for custom Trino types

### Changed
- `datacontract import --format odcs`: Now supports ODSC v3.0.0 files ([#474](https://github.com/datacontract/datacontract-cli/issues/474))
- `datacontract export --format odcs`: Now creates v3.0.0 Open Data Contract Standard files (alias to odcs_v3). Old versions are still available as format `odcs_v2`. ([#460](https://github.com/datacontract/datacontract-cli/issues/460))

### Fixed
- fix timestamp serialization from parquet -> duckdb ([#472](https://github.com/datacontract/datacontract-cli/issues/472))

## 0.10.13 — 2024-09-20 {#v0-10-13}

### Added
- `datacontract export --format data-caterer`: Export to [Data Caterer YAML](https://data.catering/setup/guide/scenario/data-generation/)

### Changed
- `datacontract export --format jsonschema` handle optional and nullable fields ([#409](https://github.com/datacontract/datacontract-cli/issues/409))
- `datacontract import --format unity` handle nested and complex fields ([#420](https://github.com/datacontract/datacontract-cli/issues/420))
- `datacontract import --format spark` handle field descriptions ([#420](https://github.com/datacontract/datacontract-cli/issues/420))
- `datacontract export --format bigquery` handle bigqueryType ([#422](https://github.com/datacontract/datacontract-cli/issues/422))

### Fixed
- use correct float type with bigquery ([#417](https://github.com/datacontract/datacontract-cli/issues/417))
- Support DATACONTRACT_MANAGER_API_KEY
- Some minor bug fixes

## 0.10.12 — 2024-09-08 {#v0-10-12}

### Added
- Support for import of DBML Models ([#379](https://github.com/datacontract/datacontract-cli/issues/379))
- `datacontract export --format sqlalchemy`: Export to [SQLAlchemy ORM models](https://docs.sqlalchemy.org/en/20/orm/quickstart.html) ([#399](https://github.com/datacontract/datacontract-cli/issues/399))
- Support of varchar max length in Glue import ([#351](https://github.com/datacontract/datacontract-cli/issues/351))
- `datacontract publish` now also accepts the `DATACONTRACT_MANAGER_API_KEY` as an environment variable
- Support required fields for Avro schema export ([#390](https://github.com/datacontract/datacontract-cli/issues/390))
- Support data type map in Spark import and export ([#408](https://github.com/datacontract/datacontract-cli/issues/408))
- Support of enum on export to avro
- Support of enum title on avro import

### Changed
- Deltalake is now using DuckDB's native deltalake support ([#258](https://github.com/datacontract/datacontract-cli/issues/258)). Extra deltalake removed.
- When dumping to YAML (import) the alias name is used instead of the pythonic name. ([#373](https://github.com/datacontract/datacontract-cli/issues/373))

### Fixed
- Fix an issue where the datacontract cli fails if installed without any extras ([#400](https://github.com/datacontract/datacontract-cli/issues/400))
- Fix an issue where Glue database without a location creates invalid data contract ([#351](https://github.com/datacontract/datacontract-cli/issues/351))
- Fix bigint -> long data type mapping ([#351](https://github.com/datacontract/datacontract-cli/issues/351))
- Fix an issue where column description for Glue partition key column is ignored ([#351](https://github.com/datacontract/datacontract-cli/issues/351))
- Corrected name of table parameter for bigquery import ([#377](https://github.com/datacontract/datacontract-cli/issues/377))
- Fix a failed to connect to S3 Server ([#384](https://github.com/datacontract/datacontract-cli/issues/384))
- Fix a model bug mismatching with the specification (`definitions.fields`) ([#375](https://github.com/datacontract/datacontract-cli/issues/375))
- Fix array type management in Spark import ([#408](https://github.com/datacontract/datacontract-cli/issues/408))

## 0.10.11 — 2024-08-08 {#v0-10-11}

### Added

- Support data type map in Glue import. ([#340](https://github.com/datacontract/datacontract-cli/issues/340))
- Basic html export for new `keys` and `values` fields
- Support for recognition of 1 to 1 relationships when exporting to DBML
- Added support for arrays in JSON schema import ([#305](https://github.com/datacontract/datacontract-cli/issues/305))

### Changed

- Aligned JSON schema import and export of required properties
- Change dbt importer to be more robust and customizable

### Fixed

- Fix required field handling in JSON schema import
- Fix an issue where the quality and definition `$ref` are not always resolved
- Fix an issue where the JSON schema validation fails for a field with type `string` and format `uuid`
- Fix an issue where common DBML renderers may not be able to parse parts of an exported file

## 0.10.10 — 2024-07-18 {#v0-10-10}

### Added
- Add support for dbt manifest file ([#104](https://github.com/datacontract/datacontract-cli/issues/104))
- Fix import of pyspark for type-checking when pyspark isn't required as a module ([#312](https://github.com/datacontract/datacontract-cli/issues/312))
- Adds support for referencing fields within a definition ([#322](https://github.com/datacontract/datacontract-cli/issues/322))
- Add `map` and `enum` type for Avro schema import ([#311](https://github.com/datacontract/datacontract-cli/issues/311))

### Fixed
- Fix import of pyspark for type-checking when pyspark isn't required as a module ([#312](https://github.com/datacontract/datacontract-cli/issues/312))- `datacontract import --format spark`: Import from Spark tables ([#326](https://github.com/datacontract/datacontract-cli/issues/326))
- Fix an issue where specifying `glue_table` as parameter did not filter the tables and instead returned all tables from `source` database ([#333](https://github.com/datacontract/datacontract-cli/issues/333))

## 0.10.9 — 2024-07-03 {#v0-10-9}

### Added
- Add support for Trino ([#278](https://github.com/datacontract/datacontract-cli/issues/278))
- Spark export: add Spark StructType exporter ([#277](https://github.com/datacontract/datacontract-cli/issues/277))
- add `--schema` option for the `catalog` and `export` command to provide the schema also locally
- Integrate support into the pre-commit workflow. For further details, please refer to the information provided [here](https://github.com/datacontract/datacontract-cli/blob/main/README.md#use-with-pre-commit).
- Improved HTML export, supporting links, tags, and more
- Add support for AWS SESSION_TOKEN ([#309](https://github.com/datacontract/datacontract-cli/issues/309))

### Changed
- Added array management on HTML export ([#299](https://github.com/datacontract/datacontract-cli/issues/299))

### Fixed
- Fix `datacontract import --format jsonschema` when description is missing ([#300](https://github.com/datacontract/datacontract-cli/issues/300))
- Fix `datacontract test` with case-sensitive Postgres table names ([#310](https://github.com/datacontract/datacontract-cli/issues/310))

## 0.10.8 — 2024-06-19 {#v0-10-8}

### Added
- `datacontract serve` start a local web server to provide a REST-API for the commands
- Provide server for sql export for the appropriate schema ([#153](https://github.com/datacontract/datacontract-cli/issues/153))
- Add struct and array management to Glue export ([#271](https://github.com/datacontract/datacontract-cli/issues/271))

### Changed
- Introduced optional dependencies/extras for significantly faster installation times. ([#213](https://github.com/datacontract/datacontract-cli/issues/213))
- Added delta-lake as an additional optional dependency
- support `GOOGLE_APPLICATION_CREDENTIALS` as variable for connecting to bigquery in `datacontract test`
- better support bigqueries `type` attribute, don't assume all imported models are tables
- added initial implementation of an importer from unity catalog (not all data types supported, yet)
- added the importer factory. This refactoring aims to make it easier to create new importers and consequently the growth and maintainability of the project. ([#273](https://github.com/datacontract/datacontract-cli/issues/273))

### Fixed
- `datacontract export --format avro` fixed array structure ([#243](https://github.com/datacontract/datacontract-cli/issues/243))

## 0.10.7 — 2024-05-31 {#v0-10-7}

### Added
- Test data contract against dataframes / temporary views ([#175](https://github.com/datacontract/datacontract-cli/issues/175))

### Fixed
- AVRO export: Logical Types should be nested ([#233](https://github.com/datacontract/datacontract-cli/issues/233))

## 0.10.6 — 2024-05-29 {#v0-10-6}

### Fixed

- Fixed Docker build by removing msodbcsql18 dependency (temporary workaround)

## 0.10.5 — 2024-05-29 {#v0-10-5}

### Added
- Added support for `sqlserver` ([#196](https://github.com/datacontract/datacontract-cli/issues/196))
- `datacontract export --format dbml`: Export to [Database Markup Language (DBML)](https://dbml.dbdiagram.io/home/) ([#135](https://github.com/datacontract/datacontract-cli/issues/135))
- `datacontract export --format avro`: Now supports config map on field level for logicalTypes and default values [Custom Avro Properties](https://github.com/datacontract/datacontract-cli/blob/main/README.md#custom-avro-properties)
- `datacontract import --format avro`: Now supports importing logicalType and default definition on avro files [Custom Avro Properties](https://github.com/datacontract/datacontract-cli/blob/main/README.md#custom-avro-properties)
- Support `config.bigqueryType` for testing BigQuery types
- Added support for selecting specific tables in an AWS Glue `import` through the `glue-table` parameter ([#122](https://github.com/datacontract/datacontract-cli/issues/122))

### Fixed

- Fixed jsonschema export for models with empty object-typed fields ([#218](https://github.com/datacontract/datacontract-cli/issues/218))
- Fixed testing BigQuery tables with BOOL fields
- `datacontract catalog` Show search bar also on mobile

## 0.10.4 — 2024-05-17 {#v0-10-4}

### Added

- `datacontract catalog` Search
- `datacontract publish`: Publish the data contract to the Data Mesh Manager
- `datacontract import --format bigquery`: Import from BigQuery format ([#110](https://github.com/datacontract/datacontract-cli/issues/110))
- `datacontract export --format bigquery`: Export to BigQuery format ([#111](https://github.com/datacontract/datacontract-cli/issues/111))
- `datacontract export --format avro`: Now supports [Avro logical types](https://avro.apache.org/docs/1.11.1/specification/#logical-types) to better model date types. `date`, `timestamp`/`timestamp-tz` and `timestamp-ntz` are now mapped to the appropriate logical types. ([#141](https://github.com/datacontract/datacontract-cli/issues/141))
- `datacontract import --format jsonschema`: Import from JSON schema ([#91](https://github.com/datacontract/datacontract-cli/issues/91))
- `datacontract export --format jsonschema`: Improved export by exporting more additional information
- `datacontract export --format html`: Added support for Service Levels, Definitions, Examples and nested Fields
- `datacontract export --format go`: Export to go types format

## 0.10.3 — 2024-05-05 {#v0-10-3}

### Fixed
- datacontract catalog: Add index.html to manifest

## 0.10.2 — 2024-05-05 {#v0-10-2}

### Added

- Added import glue ([#166](https://github.com/datacontract/datacontract-cli/issues/166))
- Added test support for `azure` ([#146](https://github.com/datacontract/datacontract-cli/issues/146))
- Added support for `delta` tables on S3 ([#24](https://github.com/datacontract/datacontract-cli/issues/24))
- Added new command `datacontract catalog` that generates a data contract catalog with an `index.html` file.
- Added field format information to HTML export

### Fixed
- RDF Export: Fix error if owner is not a URI/URN

## 0.10.1 — 2024-04-19 {#v0-10-1}

### Fixed

- Fixed docker columns

## 0.10.0 — 2024-04-19 {#v0-10-0}

### Added

- Added timestamp when ah HTML export was created

### Fixed

- Fixed export format **html**

## 0.9.9 — 2024-04-18 {#v0-9-9}

### Added

- Added export format **html** ([#15](https://github.com/datacontract/datacontract-cli/issues/15))
- Added descriptions as comments to `datacontract export --format sql` for Databricks dialects
- Added import of arrays in Avro import

## 0.9.8 — 2024-04-01 {#v0-9-8}

### Added

- Added export format **great-expectations**: `datacontract export --format great-expectations`
- Added gRPC support to OpenTelemetry integration for publishing test results
- Added AVRO import support for namespace ([#121](https://github.com/datacontract/datacontract-cli/issues/121))
- Added handling for optional fields in avro import ([#112](https://github.com/datacontract/datacontract-cli/issues/112))
- Added Databricks SQL dialect for `datacontract export --format sql`

### Fixed

- Use `sql_type_converter` to build checks.
- Fixed AVRO import when doc is missing ([#121](https://github.com/datacontract/datacontract-cli/issues/121))

## 0.9.7 — 2024-03-15 {#v0-9-7}

### Added

- Added option publish test results to **OpenTelemetry**: `datacontract test --publish-to-opentelemetry`
- Added export format **protobuf**: `datacontract export --format protobuf`
- Added export format **terraform**: `datacontract export --format terraform` (limitation: only works for AWS S3 right now)
- Added export format **sql**: `datacontract export --format sql`
- Added export format **sql-query**: `datacontract export --format sql-query`
- Added export format **avro-idl**: `datacontract export --format avro-idl`: Generates an Avro IDL file containing records for each model.
- Added new command **changelog**: `datacontract changelog datacontract1.yaml datacontract2.yaml` will now generate a changelog based on the changes in the data contract. This will be useful for keeping track of changes in the data contract over time.
- Added extensive linting on data contracts. `datacontract lint` will now check for a variety of possible errors in the data contract, such as missing descriptions, incorrect references to models or fields, nonsensical constraints, and more.
- Added importer for avro schemas. `datacontract import --format avro` will now import avro schemas into a data contract.

### Fixed

- Fixed a bug where the export to YAML always escaped the unicode characters.

## 0.9.6-2 — 2024-03-04 {#v0-9-6-2}

### Added

- test kafka for avro messages
- added export format **avro**: `datacontract export --format avro`

## 0.9.6 — 2024-03-04 {#v0-9-6}

This is a huge step forward, we now support testing Kafka messages.
We start with JSON messages and avro, and Protobuf will follow.

### Added
- test kafka for JSON messages
- added import format **sql**: `datacontract import --format sql` ([#51](https://github.com/datacontract/datacontract-cli/issues/51))
- added export format **dbt-sources**: `datacontract export --format dbt-sources`
- added export format **dbt-staging-sql**: `datacontract export --format dbt-staging-sql`
- added export format **rdf**: `datacontract export --format rdf` ([#52](https://github.com/datacontract/datacontract-cli/issues/52))
- added command `datacontract breaking` to detect breaking changes in between two data contracts.

## 0.9.5 — 2024-02-22 {#v0-9-5}

### Added
- export to dbt models ([#37](https://github.com/datacontract/datacontract-cli/issues/37)).
- export to ODCS ([#49](https://github.com/datacontract/datacontract-cli/issues/49)).
- test - show a test summary table.
- lint - Support local schema ([#46](https://github.com/datacontract/datacontract-cli/issues/46)).

## 0.9.4 — 2024-02-18 {#v0-9-4}

### Added
- Support for Postgres
- Support for Databricks

## 0.9.3 — 2024-02-10 {#v0-9-3}

### Added
- Support for BigQuery data connection
- Support for multiple models with S3

### Fixed

- Fix Docker images. Disable builds for linux/amd64.

## 0.9.2 — 2024-01-31 {#v0-9-2}

### Added
- Publish to Docker Hub

## [0.9.0] - 2024-01-26 - BREAKING

This is a breaking change (we are still on a 0.x.x version).
The project migrated from Golang to Python.
The Golang version can be found at [cli-go](https://github.com/datacontract/cli-go)

### Added
- `test` Support to directly run tests and connect to data sources defined in servers section.
- `test` generated schema tests from the model definition.
- `test --publish URL` Publish test results to a server URL.
- `export` now exports the data contract so format jsonschema and sodacl.

### Changed
- The `--file` option removed in favor of a direct argument.: Use `datacontract test datacontract.yaml` instead of `datacontract test --file datacontract.yaml`.

### Removed
- `model` is now part of `export`
- `quality` is now part of `export`
- Temporary Removed: `diff` needs to be migrated to Python.
- Temporary Removed: `breaking` needs to be migrated to Python.
- Temporary Removed: `inline` needs to be migrated to Python.

## 0.6.0 {#v0-6-0}
### Added
- Support local json schema in lint command.
- Update to specification 0.9.2.

## 0.5.3 {#v0-5-3}
### Fixed
- Fix format flag bug in model (print) command.

## 0.5.2 {#v0-5-2}
### Changed
- Log to STDOUT.
- Rename `model` command parameter, `type` -> `format`.

## 0.5.1 {#v0-5-1}
### Removed
- Remove `schema` command.

### Fixed
- Fix documentation.
- Security update of x/sys.

## 0.5.0 {#v0-5-0}
### Added
- Adapt Data Contract Specification in version 0.9.2.
- Use `models` section for `diff`/`breaking`.
- Add `model` command.
- Let `inline` print to STDOUT instead of overwriting datacontract file.
- Let `quality` write input from STDIN if present.

## 0.4.0 {#v0-4-0}
### Added
- Basic implementation of `test` command for Soda Core.

### Changed
- Change package structure to allow usage as library.

## 0.3.2 {#v0-3-2}
### Fixed
- Fix field parsing for dbt models, affects stability of `diff`/`breaking`.

## 0.3.1 {#v0-3-1}
### Fixed
- Fix comparing order of contracts in `diff`/`breaking`.

## 0.3.0 {#v0-3-0}
### Added
- Handle non-existent schema specification when using `diff`/`breaking`.
- Resolve local and remote resources such as schema specifications when using "$ref: ..." notation.
- Implement `schema` command: prints your schema.
- Implement `quality` command: prints your quality definitions.
- Implement the `inline` command: resolves all references using the "$ref: ..." notation and writes them to your data contract.

### Changed
- Allow remote and local location for all data contract inputs (`--file`, `--with`).

## 0.2.0 {#v0-2-0}
### Added
- Add `diff` command for dbt schema specification.
- Add `breaking` command for dbt schema specification.

### Changed
- Suggest a fix during `init` when the file already exists.
- Rename `validate` command to `lint`.

### Removed
- Remove `check-compatibility` command.

### Fixed
- Improve usage documentation.

## 0.1.1 {#v0-1-1}
### Added
- Initial release.

---

## Scheduling and CI/CD


Data contracts deliver the most value when they are checked **continuously**, not just once. The recommended practice is to test contracts in CI/CD on every change and, in addition, to run them on a recurring schedule (for example daily) so you detect data drift and quality regressions in production data over time.

The [`ci`](./commands/ci.md) command is purpose-built for this: it wraps [`test`](./commands/test.md) with CI-friendly annotations, a markdown summary, machine-readable output, and exit-code control via `--fail-on`.

## GitHub Actions (on change and scheduled)

The quickest route is the ready-made **[datacontract/datacontract-action](https://github.com/datacontract/datacontract-action/)**. To run the CLI directly:

```yaml
# .github/workflows/datacontract.yml
name: Data Contract CI

on:
  push:
    branches: [main]
  pull_request:
  schedule:
    # Run every day at 06:00 UTC to catch data drift in production
    - cron: "0 6 * * *"

jobs:
  datacontract-ci:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.11"
      - run: pip install datacontract-cli
      # Test one or more data contracts (supports globs, e.g. contracts/*.yaml)
      - run: datacontract ci datacontract.yaml
        env:
          DATACONTRACT_POSTGRES_USERNAME: ${{ secrets.DB_USERNAME }}
          DATACONTRACT_POSTGRES_PASSWORD: ${{ secrets.DB_PASSWORD }}
```

## Azure DevOps

```yaml
# azure-pipelines.yml
trigger:
  branches:
    include:
      - main

schedules:
  - cron: "0 6 * * *"
    displayName: Daily data contract tests
    branches:
      include: [main]
    always: true

pool:
  vmImage: "ubuntu-latest"

steps:
  - task: UsePythonVersion@0
    inputs:
      versionSpec: "3.11"
  - script: pip install datacontract-cli
    displayName: "Install datacontract-cli"
  - script: datacontract ci datacontract.yaml
    displayName: "Run data contract tests"
```

## Plain cron with Docker

If you don't have a CI system, schedule the Docker image with cron:

```cron
# Run every day at 06:00 — /etc/crontab or `crontab -e`
0 6 * * *  docker run --rm -v "/path/to/contracts:/home/datacontract" \
  -e DATACONTRACT_POSTGRES_USERNAME -e DATACONTRACT_POSTGRES_PASSWORD \
  datacontract/cli:latest ci datacontract.yaml
```

## Orchestrators (Airflow, Databricks, …)

Because the CLI is also a [Python library](./python-library.md), you can call it from any orchestrator task:

```python
from datacontract.data_contract import DataContract

def test_orders_contract():
    run = DataContract(data_contract_file="orders.odcs.yaml").test()
    if not run.has_passed():
        raise RuntimeError("Data contract tests failed")
```

Wrap this in an Airflow `PythonOperator`, a Databricks job, a Dagster op, or a Prefect task and schedule it with your orchestrator's native scheduler.

## Controlling failure behavior

Use `--fail-on` to decide when a scheduled run should be marked as failed:

```bash
# Fail the job on errors only (default)
datacontract ci --fail-on error datacontract.yaml

# Also fail on warnings
datacontract ci --fail-on warning datacontract.yaml

# Never fail (e.g. report-only schedules)
datacontract ci --fail-on never datacontract.yaml
```

## Publishing scheduled results

A scheduled run tells you whether the data is compliant *today*. Once several contracts run on a schedule, the next question is usually how they behave *over time*, and across teams — which a CI log can't answer. Publish each run to track results centrally:

```bash
datacontract ci datacontract.yaml --publish https://api.entropy-data.com/api/test-results
```

See [Integrate with Entropy Data](./entropy-data.md).

## Next steps

- Roll this out beyond a single contract: **[Adopting Data Contracts](./best-practices.md)**.
- Share the current state with your team: `datacontract export html` and the [`catalog`](./commands/catalog.md) command.

---

## Comparison with other Tools


Data quality and data contract tooling changed a lot in 2026: Great Expectations and Soda both changed hands or changed licenses, and dbt Labs merged into Fivetran. This page compares the Data Contract CLI with the three tools it is most often evaluated against — **[Great Expectations (GX Core)](https://greatexpectations.io/)**, **[Soda Core](https://www.soda.io/)**, and **[dbt contracts](https://docs.getdbt.com/docs/collaborate/govern/model-contracts)** — so you can pick the right one, or combine them.

All facts below were verified in July 2026, with sources linked at the bottom of the page.

## At a glance

| | Data Contract CLI | GX Core | Soda Core | dbt contracts |
|---|---|---|---|---|
| **License** | MIT | Apache 2.0 | Elastic License 2.0 (source-available) since v4 | Apache 2.0 (dbt Core) |
| **Maintained by** | Entropy Data and the community | Fivetran (steward since May 2026) | Soda (vendor) | dbt Labs, part of Fivetran since June 2026 |
| **Contract format** | [ODCS](./open-data-contract-standard.md), an open standard governed by [Bitol](https://bitol.io/) under the Linux Foundation AI & Data | Expectation Suites (tool-specific JSON/Python) | Soda contract YAML (tool-specific) | dbt model YAML inside a dbt project |
| **Format is vendor-neutral** | Yes | No | No | No |
| **What the contract covers** | Schema, semantics, quality rules, servers, service levels, ownership | Quality expectations | Schema and quality checks | Column names, data types, constraints |
| **When checks run** | On demand, in CI/CD, or scheduled — against data that already exists | After the data is written | After the data is written | At build time (preflight check plus DDL constraints) |
| **Where it runs** | Standalone CLI, Python library, Docker, GitHub Action, REST API | Python library | CLI and Python library | Only inside a dbt run |
| **Data sources** | [18+](./testing/index.md), including Snowflake, Databricks, BigQuery, Kafka, S3, and local files | SQL databases, Spark, pandas | Postgres, Snowflake, BigQuery, Databricks, Redshift, SQL Server, Fabric, Synapse, Athena, DuckDB | Whatever your dbt adapter supports |
| **Contract is shareable with consumers** | Yes — one portable YAML file | No | Within Soda | No — scoped to one dbt project |

## Data Contract CLI

The CLI is MIT-licensed and built natively on the **Open Data Contract Standard**. The contract is a single YAML file that describes structure, semantics, quality rules, servers, and service levels — an artifact you can put in Git, review in a pull request, hand to a consumer in another team, and [export](./exports/index.md) to 25+ downstream formats.

Because the format is an open standard governed by Bitol under the Linux Foundation AI & Data, it does not belong to any vendor — including us. That is the main structural difference to the other three: with GX, Soda, and dbt, the definition lives in a tool-specific format, so the definition and the tool move together.

**Use it when** you want one portable contract that both documents the data for its consumers and is executable against 18+ platforms, and when license and governance risk matter to you.

##  Great Expectations (GX Core)

GX Core is an established Python data quality framework, Apache 2.0-licensed, with a large expectation library. Its ownership changed in 2026: **GX Cloud was acquired by FICO** and stopped being publicly available on June 1, 2026, while **Fivetran became the steward** of the GX Core project and the open-source community.

GX describes expectations about data, not contracts. There is no portable, vendor-neutral artifact you can hand to a consuming team, and no notion of servers, service levels, or ownership.

**Use it when** you need its breadth of expectations or already have suites in production. You can keep using it from a data contract: [`datacontract export --format great-expectations`](./exports/great-expectations.md) turns a contract into a GX suite.

##  Soda Core

Soda Core changed its license with **v4 (January 2026), from Apache 2.0 to the Elastic License 2.0**. It stays source-available and free for internal use, including production, but you may no longer offer it to third parties as a hosted or managed service. Soda Core v4 also replaced the SodaCL checks language with **its own data contract format**, a breaking change with a migration tool.

So Soda now has a data contract format too, but it is Soda's format for Soda's engine, under a source-available license — not an open standard.

**Use it when** you are on the Soda platform and the ELv2 terms fit your use case. The CLI's [SodaCL export](./exports/sodacl.md) targets the checks language of Soda Core v3.

##  dbt contracts

dbt model contracts (`contract: enforced: true`) are the strongest option for **build-time** enforcement: dbt verifies before building that the model's SQL returns the declared columns, and it puts data types and constraints into the DDL, so a violating build fails instead of shipping bad data. dbt Core stays Apache 2.0-licensed under Fivetran, which completed its merger with dbt Labs on June 1, 2026.

The limits are scope, not quality. A dbt contract lives in a dbt project and applies to models built by dbt: it does not describe a Kafka topic, an S3 bucket, or a table someone else's pipeline writes, it carries no servers, SLAs, or ownership, and a consumer outside your project cannot use it as an agreement. Contracts also only cover supported materializations, and they check structure — content is still the job of dbt tests.

**Use it when** producers and consumers live in the same dbt project and build-time failure is what you need.

## They are not mutually exclusive

A data contract is the definition; these tools are execution engines. The CLI can drive the others from one ODCS contract:

- [`datacontract dbt sync`](./dbt.md) merges the contract's schema and tests into an existing dbt project, giving you build-time enforcement from the same contract.
- [`datacontract export --format great-expectations`](./exports/great-expectations.md) generates a GX suite.
- [`datacontract export --format sodacl`](./exports/sodacl.md) generates SodaCL checks.
- [`datacontract export --format dbt`](./exports/dbt-models.md) generates a dbt model schema, [sources](./exports/dbt-sources.md), or [staging SQL](./exports/dbt-staging-sql.md).

A common setup is: define the contract in ODCS, enforce structure at build time with dbt, and run [`datacontract test`](./testing/index.md) in CI/CD and on a schedule against the deployed data.

## Sources

- [An Update for the Great Expectations Community](https://greatexpectations.io/blog/an-update-from-great-expectations/) (May 6, 2026) and [Fivetran to Become Steward of the Great Expectations Open Source Community and GX Core Project](https://www.fivetran.com/press/fivetran-to-become-steward-of-the-great-expectations-open-source-community-and-gx-core-project)
- [Soda Core License Update: Moving to Elastic License 2.0](https://soda.io/blog/soda-core-license-update-moving-to-elastic-license) and [Introducing Soda 4.0](https://soda.io/blog/introducing-soda-4.0)
- [dbt model contracts](https://docs.getdbt.com/docs/collaborate/govern/model-contracts) and [Fivetran + dbt Labs Complete Merger](https://www.fivetran.com/press/fivetran-dbt-labs-complete-merger-to-create-the-data-infrastructure-for-trusted-ai-agents) (June 1, 2026)

---

## Integrate with Entropy Data


[Entropy Data](https://entropy-data.com/) is a commercial platform to manage data contracts. It provides a web UI, access management, and data governance for a data product marketplace based on data contracts.

The Data Contract CLI integrates with Entropy Data through the `--publish` option: it runs the tests and pushes the **full results** to the [Entropy Data API](https://api.entropy-data.com/swagger/index.html), where they are displayed and tracked over time.

## Publish test results

Reference the contract by URL, run the tests against a server, and append `--publish`. Provide your API key as an environment variable.

```bash
export ENTROPY_DATA_API_KEY=xxx

datacontract test https://demo.entropy-data.com/demo279750347121/datacontracts/4df9d6ee-e55d-4088-9598-b635b2fdcbbc/datacontract.yaml \
  --server production \
  --publish https://api.entropy-data.com/api/test-results
```

The same `--publish` option is available on [`ci`](./commands/ci.md) and [`dbt sync`](./commands/dbt.md), so you can report results from CI/CD and scheduled runs — see [Scheduling and CI/CD](./ci-cd.md).

## Publish the contract

Use the [`publish`](./commands/publish.md) command to push a data contract itself to Entropy Data:

```bash
datacontract publish datacontract.yaml
```

## TLS behind a corporate proxy or internal CA

By default the CLI verifies TLS certificates against the bundled CA certificates (`certifi`). In a corporate network with a TLS-inspecting proxy or an internal certificate authority, this can fail with `CERTIFICATE_VERIFY_FAILED: unable to get local issuer certificate`, because the root CA is installed in the operating system's trust store but not in the bundled list.

Use the global `--system-truststore` option to verify against the operating system's trust store (macOS Keychain, Windows certificate store, or the system CA certificates on Linux) instead:

```bash
datacontract --system-truststore publish datacontract.yaml
```

You can also enable it for every invocation with an environment variable:

```bash
export DATACONTRACT_SYSTEM_TRUSTSTORE=1
```

This keeps certificate verification on while trusting the corporate root CA. It works for all commands that make HTTPS requests, not only the Entropy Data integration.