Connect your Data

To test a data contract, the CLI connects to the data source defined in the contract's servers block. The connection details (host, catalog, location, …) live in the contract; credentials are provided as environment variables.

servers:
  production:
    type: postgres   # selects the connection engine
    host: localhost
    port: 5432
    database: postgres
    schema: public

Environment variables are also loaded from a .env file in the current working directory (or the nearest parent directory containing one). Already-set environment variables take precedence over values from .env.

# .env
DATACONTRACT_POSTGRES_USERNAME=postgres
DATACONTRACT_POSTGRES_PASSWORD=postgres

Supported connections

Amazon AthenaAthena over data in S3Amazon RedshiftRedshift (Postgres wire protocol)Amazon S3CSV, JSON, Delta, Parquet on S3 / S3-compatible storageApache ImpalaImpalaAzure Blob / ADLSFiles on Azure Blob storage or ADLS Gen2DatabricksUnity Catalog / Hive metastore (warehouse or notebook)Google BigQueryBigQuery tables and viewsGoogle Cloud StorageFiles on GCS via S3 interoperabilityHTTP APIJSON HTTP APIs (GET only)KafkaKafka topics (experimental)Local filesLocal Parquet, JSON, CSV, or Delta filesMicrosoft SQL ServerSQL Server, Azure SQL, Synapse, FabricMySQLMySQL / MariaDBOracleOracle DatabasePostgresPostgres and Postgres-compatible (e.g. RisingWave)SnowflakeSnowflakeSpark DataFrameIn-memory Spark DataFrames (programmatic)TrinoTrino (basic, JWT, OAuth2)

tip

Missing a source? Open an issue on GitHub.

Each connection requires the matching optional dependency (extra), or install everything with datacontract-cli[all].