Skip to main content

Amazon S3 Reference

Authentication options and data type handling for S3 connections.

Authentication

VariableExampleDescription
DATACONTRACT_S3_REGIONeu-central-1Region of the S3 bucket
DATACONTRACT_S3_ACCESS_KEY_IDAKIAXV5Q5Q...AWS Access Key ID
DATACONTRACT_S3_SECRET_ACCESS_KEY93S7LRrJ...AWS Secret Access Key
DATACONTRACT_S3_SESSION_TOKENAQoDYXdzEJr...AWS temporary session token (optional)

If DATACONTRACT_S3_ACCESS_KEY_ID is not set, no credentials are configured — public buckets and ambient credentials still work. For S3-compatible storage (MinIO), set endpointUrl in the servers block; the CLI then uses path-style addressing.

Data types

Files on S3 are read with DuckDB. Type handling depends on the format in the servers block:

formatHow types are handled
csvThe file is read as the contract's types — no type checks are generated; a value that can't be coerced surfaces as a read error.
jsonSame as CSV, plus every record is validated against a JSON Schema derived from the contract's logicalTypes (with format options like date-time, uuid).
parquetColumn types come from the Parquet file; the contract's logicalType is checked by category (Check that field x has type y).
deltaColumn types come from the Delta table; the contract's logicalType is checked by category.

physicalType is never checked against file sources — declare logicalType (and logicalTypeOptions for value constraints). See Local files for how CSV/JSON imports infer types.

Logical type mapping

When no physicalType is declared, the CLI derives the DuckDB column type from the logicalType when reading files. This table is generated from the converters in the CLI's code:

logicalTypeCSV read typeParquet read type
stringVARCHARVARCHAR
integerBIGINTINTEGER
numberDOUBLEDECIMAL
booleanBOOLEANBOOLEAN
dateDATEDATE
timestampTIMESTAMPTIMESTAMP WITH TIME ZONE
timeTIMETIME
objectVARCHARSTRUCT()
arrayVARCHARVARCHAR[]