Amazon S3

:::info Required extra This connection requires the s3 and duckdb extras. See Installation. :::

Test data stored in S3 buckets or any S3-compatible endpoint, in CSV, JSON, Delta, or Parquet format.

Server (JSON)

servers:
  - server: production
    type: s3
    endpointUrl: https://minio.example.com # not needed with AWS S3
    location: s3://bucket-name/path/*/*.json
    format: json
    delimiter: new_line # new_line, array, or none

Server (Delta)

servers:
  - server: production
    type: s3
    endpointUrl: https://minio.example.com # not needed with AWS S3
    location: s3://bucket-name/path/table.delta # folder with parquet files and _delta_log
    format: delta

Environment variables

Variable	Example	Description
DATACONTRACT_S3_REGION	eu-central-1	Region of the S3 bucket
DATACONTRACT_S3_ACCESS_KEY_ID	AKIAXV5Q5Q...	AWS Access Key ID
DATACONTRACT_S3_SECRET_ACCESS_KEY	93S7LRrJ...	AWS Secret Access Key
DATACONTRACT_S3_SESSION_TOKEN	AQoDYXdzEJr...	AWS temporary session token (optional)