A lakehouse is a data lake with a transactional table format layered on top of it, giving cheap object storage the schema enforcement, ACID transactions, and time travel that used to require a separate data warehouse. Instead of landing raw files in a lake and then copying curated subsets into a warehouse for analytics, a lakehouse lets both raw data engineering and structured SQL analytics query the same underlying files. It is not a new storage medium — it is a metadata and transaction layer (Delta Lake, Apache Iceberg, or Apache Hudi) that makes existing lake storage behave enough like a warehouse to skip maintaining both.
The core idea
Object storage like S3 or GCS has always been cheap and scalable, but plain files on it have no transactional guarantees: two concurrent writers can corrupt a table, a failed job can leave partial data behind, and there is no way to see what a table looked like yesterday. Open table formats solve this by writing a transaction log alongside the data files. Every write appends an entry to that log; readers use the log to know exactly which files make up a consistent snapshot of the table at any point in time.
# Delta Lake: an ACID write is just a commit to the transaction log
df.write.format("delta").mode("append").save("s3://lakehouse/orders")
# Time travel: query the table as it existed at a previous version
spark.read.format("delta").option("versionAsOf", 12).load("s3://lakehouse/orders")
This is what lets a lakehouse offer schema enforcement (reject a write that does not match the expected columns), upserts and deletes (needed for GDPR-style data removal, which plain append-only lake files cannot do cleanly), and concurrent writers without corruption — all without moving the data into a different storage system.
Delta Lake vs Iceberg vs Hudi
| Format |
Origin |
Strength |
Common pairing |
| Delta Lake |
Databricks |
Deep Spark integration, strong tooling maturity |
Databricks, Spark-heavy stacks |
| Apache Iceberg |
Netflix, now Apache |
Broad engine support (Trino, Flink, Spark, Snowflake) |
Multi-engine, vendor-neutral stacks |
| Apache Hudi |
Uber, now Apache |
Strong incremental upsert performance |
High-frequency change-data-capture pipelines |
All three solve the same core problem — transactions and schema on lake storage — and the practical choice usually comes down to which query engines your team already runs, since engine support varies between formats.
What a lakehouse changes operationally
A lakehouse collapses a pipeline that used to look like: land raw data in a lake, run a scheduled job to clean and copy a subset into a warehouse, then query the warehouse. Instead: land raw data directly into lakehouse tables, and both the data science team (querying with Spark) and the BI team (querying with a SQL engine like Trino or Databricks SQL) read the same tables. This removes the duplicated storage and the lag introduced by a batch copy step, and it means a single governance and access-control layer (like Unity Catalog or AWS Lake Formation) covers both audiences instead of two separate systems.
It does not remove the need for good data pipeline orchestration — you still need scheduled jobs to compact small files, manage partitions, and run the transformations that turn raw landed data into clean, queryable tables.
Common mistakes
- Treating the table format as a performance guarantee. Delta, Iceberg, and Hudi provide correctness and transactions, not automatic speed. Small-file problems and poor partitioning still make queries slow; regular compaction jobs are part of running a lakehouse, not optional maintenance.
- Picking a table format based on hype instead of engine support. If your BI team's query engine only has mature support for one format, that constraint should drive the choice more than benchmarks you found online.
- Skipping a real orchestration layer. A lakehouse still needs scheduled compaction, vacuuming of old file versions, and transformation jobs — running these ad hoc instead of through an orchestrator leads to silently stale or bloated tables.
- Assuming a lakehouse eliminates the need for ELT logic. Raw landed data still needs cleaning and modeling before it is analyst-ready; the lakehouse changes where that data lives, not whether transformation work is needed. See ETL vs ELT.
FAQ
Is a lakehouse just a data lake with a new name?
No. The defining feature is the transaction log from an open table format, which gives a plain lake ACID transactions, schema enforcement, and time travel it did not have before.
Do I need Databricks to run a lakehouse?
No. Delta Lake, Iceberg, and Hudi are open source and readable by many engines (Spark, Trino, Flink, Snowflake, and others). Databricks popularized the term and Delta Lake, but the pattern is not vendor-locked.
Does a lakehouse replace a data warehouse entirely?
For many teams, yes — especially once query engines caught up on lake-native performance. Some teams still keep a small warehouse for the fastest, most latency-sensitive BI dashboards, but it is no longer the default assumption.
Which table format should I pick?
Match it to your query engines. Iceberg has the broadest multi-engine support, Delta Lake has the deepest Spark and Databricks integration, and Hudi leads on high-frequency upsert workloads like change data capture.
Where to go next
Start with data lake vs data warehouse for the pattern this replaces, ETL vs ELT explained for how transformation still fits in, and data pipeline orchestration tools for scheduling the jobs a lakehouse still needs.