Amazon S3 Reference

Authentication options and data type handling for S3 connections.

Server

JSON

servers:
  - server: production
    type: s3
    endpointUrl: https://minio.example.com # not needed with AWS S3
    location: s3://bucket-name/path/*/*.json
    format: json
    delimiter: new_line # new_line, array, or none

Delta

servers:
  - server: production
    type: s3
    endpointUrl: https://minio.example.com # not needed with AWS S3
    location: s3://bucket-name/path/table.delta # folder with parquet files and _delta_log
    format: delta

Authentication

An existing AWS session is used when no key is set — aws sso login, AWS_PROFILE, EC2/ECS/EKS instance roles, or GitHub OIDC in CI. With nothing resolvable the objects are read unsigned, which is what public buckets need.

The variables below override that:

Variable	Example	Description
DATACONTRACT_S3_REGION	eu-central-1	Region of the S3 bucket
DATACONTRACT_S3_ACCESS_KEY_ID	AKIAXV5Q5Q...	AWS Access Key ID
DATACONTRACT_S3_SECRET_ACCESS_KEY	93S7LRrJ...	AWS Secret Access Key
DATACONTRACT_S3_SESSION_TOKEN	AQoDYXdzEJr...	AWS temporary session token (optional)

If DATACONTRACT_S3_ACCESS_KEY_ID is not set, no credentials are configured — public buckets and ambient credentials still work. For S3-compatible storage (MinIO), set endpointUrl in the servers block; the CLI then uses path-style addressing.

Data types

Files on S3 are read with DuckDB. Type handling depends on the format in the servers block:

format	How types are handled
csv	The file is read as the contract's types — no type checks are generated; a value that can't be coerced surfaces as a read error.
json	Same as CSV, plus every record is validated against a JSON Schema derived from the contract's logicalTypes (with format options like date-time, uuid).
parquet	The file is read as the contract's types, like CSV, so checking its types is not supported yet, and a value that can't be cast surfaces as a read error.
delta	Column types come from the Delta table; the contract's logicalType is checked by category.

physicalType is never checked against file sources — declare logicalType (and logicalTypeOptions for value constraints). See Local files for how CSV/JSON imports infer types.

Logical type mapping

When no physicalType is declared, the CLI derives the DuckDB column type from the logicalType when reading files. This table is generated from the converters in the CLI's code:

logicalType	CSV read type	Parquet read type
string	VARCHAR	VARCHAR
integer	BIGINT	INTEGER
number	DOUBLE	DECIMAL
boolean	BOOLEAN	BOOLEAN
date	DATE	DATE
timestamp	TIMESTAMP	TIMESTAMP WITH TIME ZONE
time	TIME	TIME
object	VARCHAR	STRUCT()
array	VARCHAR	VARCHAR[]
map	VARCHAR	MAP(VARCHAR, VARCHAR)
vector	VARCHAR	FLOAT[]