Skip to main content

Azure Blob / ADLS Reference

Authentication options and data type handling for Azure storage connections.

Authentication

Authentication uses an Azure Service Principal (App Registration) with a secret:

VariableExampleDescription
DATACONTRACT_AZURE_TENANT_ID79f5b80f-...The Azure Tenant ID
DATACONTRACT_AZURE_CLIENT_ID3cf7ce49-...The Application/Client ID of the app registration
DATACONTRACT_AZURE_CLIENT_SECRETyZK8Q~GWO1M...The client secret value

The service principal needs the Storage Blob Data Reader role. The metadata-check path alternatively accepts DATACONTRACT_AZURE_CONNECTION_STRING or DATACONTRACT_AZURE_STORAGE_ACCOUNT_KEY.

Supported location URL formats (in the servers block):

  • https://<account>.blob.core.windows.net/<container>/<prefix>
  • abfss://<container>@<account>.dfs.core.windows.net/<prefix>
  • azure://<container>@<account>.blob.core.windows.net/<prefix>
  • wasbs://<container>@<account>.blob.core.windows.net/<prefix>

Data types

Files on Azure storage are read with DuckDB. Type handling depends on the format in the servers block:

formatHow types are handled
csvThe file is read as the contract's types — no type checks are generated; a value that can't be coerced surfaces as a read error.
jsonSame as CSV, plus every record is validated against a JSON Schema derived from the contract's logicalTypes.
parquetColumn types come from the Parquet file; the contract's logicalType is checked by category.
deltaColumn types come from the Delta table; the contract's logicalType is checked by category.

physicalType is never checked against file sources. Schema objects with logicalType: blob / physicalType: file switch to metadata checks against blob properties instead of reading file contents.

Logical type mapping

When no physicalType is declared, the CLI derives the DuckDB column type from the logicalType when reading files. This table is generated from the converters in the CLI's code:

logicalTypeCSV read typeParquet read type
stringVARCHARVARCHAR
integerBIGINTINTEGER
numberDOUBLEDECIMAL
booleanBOOLEANBOOLEAN
dateDATEDATE
timestampTIMESTAMPTIMESTAMP WITH TIME ZONE
timeTIMETIME
objectVARCHARSTRUCT()
arrayVARCHARVARCHAR[]