Skip to main content

Apache Impala

Run checks against an Apache Impala cluster.

1. Install​

uv tool install --python python3.11 --upgrade 'datacontract-cli[impala]'

See Installation for pip, pipx, and Docker.

2. Set credentials​

Create a .env file in your working directory (or export the variables):

# .env
DATACONTRACT_IMPALA_USERNAME=analytics_user
DATACONTRACT_IMPALA_PASSWORD=mysecretpassword

3. Create a contract from your tables​

Get the DDL of a table (SHOW CREATE TABLE orders; in impala-shell or Hue), save it to a file, and import it. Impala DDL is Hive-compatible, so use the spark dialect:

datacontract import sql --source orders.sql --dialect spark --output datacontract.yaml

The SQL import can't know your connection details, so it writes a servers block with placeholder values. Open datacontract.yaml and fill in your cluster:

servers:
  - server: impala
    type: impala
    host: my-impala-host
    port: 443
    database: my_database # optional default database

4. Test the actual data​

datacontract test datacontract.yaml
🟢 data contract is valid. Run 24 checks. Took 6.3 seconds.

5. Let it catch a violation​

The contract becomes valuable when it detects drift. Tighten an expectation — for example, add a quality rule to a schema in datacontract.yaml:

schema:
  - name: orders
    # ...
    quality:
      - type: sql
        description: No order has a negative total
        query: SELECT COUNT(*) FROM orders WHERE order_total < 0
        mustBe: 0

Run datacontract test datacontract.yaml again: every violation is listed as an error, and the command exits with code 1 — ready for CI/CD scheduling so you catch drift before your consumers do.

Reference​

All authentication options (SSL, transport, auth mechanism) and the data type mappings: Impala Reference.

Troubleshooting​

  • Connection errors — the defaults assume LDAP auth over SSL and HTTP transport (port 443, e.g. behind a load balancer). For a plain binary-protocol cluster, set DATACONTRACT_IMPALA_USE_HTTP_TRANSPORT=false and use port 21050.
  • AuthorizationException — the user lacks SELECT on the table or the Ranger/Sentry policy doesn't cover the database in the servers block.