DELTA-LAKE

Databricks ACID Parquet table format.

Definition

Delta Lake uses an append-only _delta_log/*.json transaction log, with snapshots, schema enforcement, generated columns. Delta Sharing protocol for cross-org sharing. UniForm enables cross-format reading (Delta ↔ Iceberg ↔ Hudi). Spec v3 published 2024.

Origin

Created by Databricks April 2019 ; donated to Linux Foundation October 2019 ; protocol v3 in 2024.

Example in context

Comcast uses Delta Lake on Databricks to process 1 trillion events/day with audit-purpose time-travel.

Related terms

• Iceberg — open competitor.