Delta Lake

Delta Lake is an open-source storage framework that enables building a format-agnostic Lakehouse architecture, compatible with compute engines including Spark, PrestoDB, Flink, Trino, Hive, Snowflake, Google BigQuery, Athena, Redshift, and Databricks. It provides ACID transactions, scalable metadata handling, and time travel for petabyte-scale data workloads. With Delta Universal Format (UniForm), Delta tables can be read by Iceberg and Hudi clients, enabling broad interoperability. Backed by contributions from over 190 developers across 70+ organizations, it is battle-tested in over 10,000 production environments.

• ACID Transactions — Protects data with serializability, the strongest level of isolation, ensuring data integrity across concurrent operations.
• Scalable Metadata — Handles petabyte-scale tables with billions of partitions and files with ease, enabling large-scale analytics.
• Time Travel — Access or revert to earlier versions of data for audits, rollbacks, or reproducibility of experiments.
• Open Source — Community-driven with open standards, open protocol, and open discussions under the Linux Foundation Projects umbrella.
• Unified Batch/Streaming — Supports exactly-once semantics for ingestion, backfill, and interactive queries in a single framework.
• Schema Evolution & Enforcement — Prevents bad data from causing corruption by enforcing and evolving schemas automatically.
• Audit History — Logs all change details to provide a full audit trail for compliance and debugging.
• DML Operations — Provides SQL, Scala/Java, and Python APIs to merge, update, and delete datasets.
• Delta UniForm — Enables reading Delta tables with Iceberg and Hudi clients for universal lakehouse interoperability.
• Platform Agnostic — Works with any query engine on any cloud, on-premises, or locally without vendor lock-in.

To get started, visit the Getting Started guide on delta.io, install the Delta Lake library for your preferred compute engine (e.g., via Maven or pip), and begin creating Delta tables with your existing Spark or Python workflows.