Skip to main content

Import: Iceberg

Creates a data contract from an Apache Iceberg table schema.

Import from a REST catalog​

datacontract import iceberg \
  --catalog-url https://polaris.example.com/api/catalog \
  --catalog main --namespace sales --table orders \
  --output datacontract.yaml

The result includes a server ready for datacontract test. --catalog defaults to default; a qualified --table sales.orders also works without --namespace. See testing Iceberg for authentication and Amazon S3 Tables for the AWS endpoint, SSO, and required warehouse ARN.

Import from a schema file​

A schema-file import does not add a server. Add one before testing live data.

Given this orders.json:

{
  "type": "struct",
  "fields": [
    { "id": 1, "name": "order_id", "type": "string", "required": true },
    { "id": 2, "name": "order_timestamp", "type": "timestamptz", "required": true },
    { "id": 3, "name": "customer_id", "type": "string", "required": true },
    { "id": 4, "name": "order_total", "type": "long", "required": true },
    { "id": 5, "name": "status", "type": "string", "required": true }
  ],
  "schema-id": 0,
  "identifier-field-ids": [1]
}

Run:

datacontract import iceberg --source orders.json

to produce the data contract:

version: 1.0.0
kind: DataContract
apiVersion: v3.2.0
id: my-data-contract
name: My Data Contract
status: draft
schema:
- name: iceberg_table
  physicalType: table
  logicalType: object
  physicalName: iceberg_table
  properties:
  - name: order_id
    physicalType: string
    customProperties:
    - property: icebergFieldId
      value: 1
    primaryKey: true
    primaryKeyPosition: 1
    logicalType: string
    required: true
  - name: order_timestamp
    physicalType: timestamptz
    customProperties:
    - property: icebergFieldId
      value: 2
    logicalType: timestamp
    required: true
  - name: customer_id
    physicalType: string
    customProperties:
    - property: icebergFieldId
      value: 3
    logicalType: string
# …

All options: datacontract import iceberg.