Sync with dbt

The Data Contract CLI integrates with dbt in multiple ways:

Sync: Merge the schema and tests of one or multiple data contracts into an existing dbt project (datacontract dbt sync)
Test: Run the CLI-generated dbt tests, optionally scoped to a contract ( datacontract dbt test)
Export: Do a one-time export from a data contract into a dbt model schema, a sources YAML, or a staging SQL file (datacontract export)
Import: Create a data contract from a dbt manifest file (datacontract import dbt)

`datacontract dbt sync`

dbt sync reads one or more ODCS data contracts and applies the specified schema and tests to an existing dbt project. It updates the model properties files (YAML) in-place and adds singular SQL tests if needed.

# Auto-discover every *.odcs.yaml in a dbt project, update models and test
datacontract dbt sync

# Delete columns, tests etc. that are not specified in the contract
datacontract dbt sync --prune

# Specify contract and dbt project directory
datacontract dbt sync orders.odcs.yaml --project-dir ./warehouse

# Several contracts, or a glob
datacontract dbt sync orders.odcs.yaml customers.odcs.yaml
datacontract dbt sync "contracts/*.odcs.yaml"

# Run the generated tests (or as separate command: datacontract dbt test)
datacontract dbt sync --run-tests

# Generate, run, and publish results to an Entropy Data instance 
datacontract dbt sync orders.odcs.yaml --run-tests --publish https://api.entropy-data.com/api/test-results

A list of all options can be found at the dbt command reference.

What is synced?

All CLI-managed content in the dbt project gets marked with custom meta tags. By default, schemas or tests that are neither specified in the contract nor generated by the CLI are kept, unless --prune is specified.

ODCS source	dbt target
`schema` / `property` description	model / column `description`
`property.logicalType` / `physicalType`	column `data_type`, mapped to the server dialect
`property`	column entry
`schema.tags` / `property.tags`	`config.tags`
`property.required`, single `primaryKey`, `property.quality.mustBe`	native dbt tests (`not_null`, `unique`, `accepted_values`)
composite `primaryKey`, `logicalTypeOptions` bounds (`minLength`, `maxLength`, `pattern`, `minimum`, …), `quality` with a `query` + bound, `schema.quality` `rowCount`	singular SQL tests at `<testsRoot>/datacontract_cli/`

Example

Given this contract for IDs and email addresses in the orders schema:

orders-v1.odcs.yaml
apiVersion: v3.0.0
kind: DataContract
id: orders
version: 1.0.0
status: active
schema:
  - name: orders
    description: Customer orders
    properties:
      - name: order_id
        physicalType: integer
        primaryKey: true
      - name: customer_email
        description: Billing email address
        logicalType: string
        physicalType: text
        required: true
        logicalTypeOptions:
          maxLength: 320

Let's assume that a minimal model properties file already exists. This is what it looks like after running datacontract dbt sync orders-v1.odcs.yaml (highlighted lines got added):

models/orders.yml
version: 2
models:
  - name: orders
    description: Customer orders
    config:
      meta:
        datacontract_cli:
          contract_id: orders
    columns:
      - name: order_id
        data_type: integer
        data_tests:
          - not_null:
              config:
                meta:
                  datacontract_cli:
                    check: orders__order_id__field_required
                    include_in_tests: true
                    contract_versions:
                      - 1.0.0
                    generated: true
              description: Check that field order_id has no missing values
          - unique:
              config:
                meta:
                  datacontract_cli:
                    check: orders__order_id__field_unique
                    include_in_tests: true
                    contract_versions:
                      - 1.0.0
                    generated: true
              description: Check that field order_id has no duplicate values
        meta:
          datacontract_cli:
            generated: true
      - name: customer_email
        data_type: text
        description: Billing email address
        data_tests:
          - not_null:
              config:
                meta:
                  datacontract_cli:
                    check: orders__customer_email__field_required
                    include_in_tests: true
                    contract_versions:
                      - 1.0.0
                    generated: true
              description: Check that field customer_email has no missing values
        meta:
          datacontract_cli:
            generated: true

The generated config.meta.datacontract_cli block is how dbt sync/dbt test recognize and scope managed tests; the per-column meta.datacontract_cli.generated marks a column the CLI added.

The maxLength bound becomes a self-contained singular SQL test (no dbt_utils needed). Its config() header carries the same datacontract_cli metadata:

tests/datacontract_cli/orders/orders__1_0_0__orders__customer_email__length.sql
-- AUTO-GENERATED by `datacontract dbt sync`. Do not edit.
-- Source contract: orders@1.0.0 (model: orders, check: customer_email__length)
{{ config(meta={"datacontract_cli": {"check": "orders__customer_email__field_length", "contract_versions": ["1.0.0"], "generated": true, "include_in_tests": true, "model": "orders", "field": "customer_email", "description": "Check that field customer_email has a length of at most 320"}}) }}
SELECT *
FROM {{ ref('orders') }}
WHERE "customer_email" IS NOT NULL
  AND (LENGTH("customer_email") > 320)

Versioned models

dbt sync works natively with dbt model versions. To specify multiple versions using data contracts, create one contract file per version. Their filename must include an integer version number with a preceding v, e.g. orders-v01.odcs.yaml.

If dbt sync is fed with a subset of the versioned contracts, the other versions remain unchanged.

`datacontract dbt test`

Runs the contract-managed tests that dbt sync generated, reports the results, and optionally publishes them. Like dbt sync, the command accepts multiple contracts.

# Every contract in the current project
datacontract dbt test

# A single contract and dbt target in a specified project
datacontract dbt test orders.odcs.yaml --project-dir ./warehouse --target dev

See the dbt command reference.

dbt exporters

Convert a contract into dbt artifacts:

dbt-models — dbt model schema YAML. If a server is selected via --server, the dbt data_types match the expected data types of that server; otherwise it defaults to snowflake.
dbt-sources — dbt sources YAML.
dbt-staging-sql — a dbt staging SQL file.

datacontract export dbt-models datacontract.yaml --server snowflake

dbt importer

Generate a contract from a dbt project's manifest.json:

# Import specific tables from a dbt manifest
datacontract import dbt --source manifest.json --model orders --model line_items

# Import all tables
datacontract import dbt --source manifest.json

datacontract dbt sync​

What is synced?​

Example​

Versioned models​

datacontract dbt test​

dbt exporters​

dbt importer​