# Sync with dbt

> Sync a data contract's schema and tests into your dbt project with datacontract dbt sync, then run them with datacontract dbt test.

# Sync with dbt

The Data Contract CLI integrates with [dbt](https://www.getdbt.com/) in multiple ways:

- **Sync**: Merge the schema and tests of one or multiple data contracts into an existing dbt
  project (`datacontract dbt sync`)
- **Test**: Run the CLI-generated dbt tests, optionally scoped to a contract (
  `datacontract dbt test`)
- **Export**: Do a one-time export from a data contract into a dbt model schema, a sources YAML, or a staging SQL file (`datacontract export`)
- **Import**: Create a data contract from a dbt manifest file (`datacontract import dbt`)

## `datacontract dbt sync`

`dbt sync` reads one or more ODCS data contracts and applies the specified schema and tests to an
existing dbt project. It updates the model properties files (YAML) in-place and adds singular SQL
tests if needed.

```bash
# Auto-discover every *.odcs.yaml in a dbt project, update models and test
datacontract dbt sync

# Delete columns, tests etc. that are not specified in the contract
datacontract dbt sync --prune

# Specify contract and dbt project directory
datacontract dbt sync orders.odcs.yaml --project-dir ./warehouse

# Several contracts, or a glob
datacontract dbt sync orders.odcs.yaml customers.odcs.yaml
datacontract dbt sync "contracts/*.odcs.yaml"

# Run the generated tests (or as separate command: datacontract dbt test)
datacontract dbt sync --run-tests

# Generate, run, and publish results to an Entropy Data instance 
datacontract dbt sync orders.odcs.yaml --run-tests --publish https://api.entropy-data.com/api/test-results
```

A list of all options can be found at the [`dbt` command reference](./commands/dbt/sync.md).

### What is synced?

All CLI-managed content in the dbt project gets marked with custom meta tags. By default, schemas
or tests that are neither specified in the contract nor generated by the CLI are kept, unless
`--prune` is specified.

| ODCS source | dbt target                                                 |
|---|------------------------------------------------------------|
| `schema` / `property` description | model / column `description`                               |
| `property.logicalType` / `physicalType` | column `data_type`, mapped to the server dialect           |
| `property` | column entry                                               |
| `schema.tags` / `property.tags` | `config.tags`                                              |
| `property.required`, single `primaryKey`, `property.quality.mustBe` | native dbt tests (`not_null`, `unique`, `accepted_values`) |
| composite `primaryKey`, `logicalTypeOptions` bounds (`minLength`, `maxLength`, `pattern`, `minimum`, …), `quality` with a `query` + bound, `schema.quality` `rowCount` | singular SQL tests at `<testsRoot>/datacontract_cli/`      |

### Example

Given this contract for IDs and email addresses in the `orders` schema:

```yaml title="orders-v1.odcs.yaml"
apiVersion: v3.0.0
kind: DataContract
id: orders
version: 1.0.0
status: active
schema:
  - name: orders
    description: Customer orders
    properties:
      - name: order_id
        physicalType: integer
        primaryKey: true
      - name: customer_email
        description: Billing email address
        logicalType: string
        physicalType: text
        required: true
        logicalTypeOptions:
          maxLength: 320
```

Let's assume that a minimal model properties file already exists. This is what it looks like after running
`datacontract dbt sync orders-v1.odcs.yaml` (highlighted lines got added):

```yaml title="models/orders.yml" {5-8,12-35,39-52}
version: 2
models:
  - name: orders
    description: Customer orders
    config:
      meta:
        datacontract_cli:
          contract_id: orders
    columns:
      - name: order_id
        data_type: integer
        data_tests:
          - not_null:
              config:
                meta:
                  datacontract_cli:
                    check: orders__order_id__field_required
                    include_in_tests: true
                    contract_versions:
                      - 1.0.0
                    generated: true
              description: Check that field order_id has no missing values
          - unique:
              config:
                meta:
                  datacontract_cli:
                    check: orders__order_id__field_unique
                    include_in_tests: true
                    contract_versions:
                      - 1.0.0
                    generated: true
              description: Check that field order_id has no duplicate values
        meta:
          datacontract_cli:
            generated: true
      - name: customer_email
        data_type: text
        description: Billing email address
        data_tests:
          - not_null:
              config:
                meta:
                  datacontract_cli:
                    check: orders__customer_email__field_required
                    include_in_tests: true
                    contract_versions:
                      - 1.0.0
                    generated: true
              description: Check that field customer_email has no missing values
        meta:
          datacontract_cli:
            generated: true
```

The generated `config.meta.datacontract_cli` block is how `dbt sync`/`dbt test` recognize and scope managed tests; the per-column `meta.datacontract_cli.generated` marks a column the CLI added.

The `maxLength` bound becomes a self-contained singular SQL test (no `dbt_utils` needed). Its `config()` header carries the same `datacontract_cli` metadata:

```sql title="tests/datacontract_cli/orders/orders__1_0_0__orders__customer_email__length.sql"
-- AUTO-GENERATED by `datacontract dbt sync`. Do not edit.
-- Source contract: orders@1.0.0 (model: orders, check: customer_email__length)
{{ config(meta={"datacontract_cli": {"check": "orders__customer_email__field_length", "contract_versions": ["1.0.0"], "generated": true, "include_in_tests": true, "model": "orders", "field": "customer_email", "description": "Check that field customer_email has a length of at most 320"}}) }}
SELECT *
FROM {{ ref('orders') }}
WHERE "customer_email" IS NOT NULL
  AND (LENGTH("customer_email") > 320)
```

### Versioned models

`dbt sync` works natively with [dbt model versions](https://docs.getdbt.com/docs/mesh/govern/model-versions). To specify multiple versions using data contracts, create one contract file per version. Their filename must include an integer version number with a preceding `v`, e.g. `orders-v01.odcs.yaml`.

If `dbt sync` is fed with a subset of the versioned contracts, the other versions remain unchanged.

## `datacontract dbt test`

Runs the contract-managed tests that `dbt sync` generated, reports the results, and optionally publishes them. Like `dbt sync`, the command accepts multiple contracts.

```bash
# Every contract in the current project
datacontract dbt test

# A single contract and dbt target in a specified project
datacontract dbt test orders.odcs.yaml --project-dir ./warehouse --target dev
```

See the [`dbt` command reference](./commands/dbt/index.md).

## dbt exporters

Convert a contract into dbt artifacts:

- **[`dbt-models`](./exports/dbt-models.md)** — dbt model schema YAML. If a server is selected via `--server`, the dbt `data_types` match the expected data types of that server; otherwise it defaults to `snowflake`.
- **[`dbt-sources`](./exports/dbt-sources.md)** — dbt sources YAML.
- **[`dbt-staging-sql`](./exports/dbt-staging-sql.md)** — a dbt staging SQL file.

```bash
datacontract export dbt-models datacontract.yaml --server snowflake
```

## dbt importer

Generate a contract from a dbt project's `manifest.json`:

```bash
# Import specific tables from a dbt manifest
datacontract import dbt --source manifest.json --model orders --model line_items

# Import all tables
datacontract import dbt --source manifest.json
```