{"post":{"slug":"columnar-storage-explained-2026","title":"Columnar Storage Explained: Why Analytics Databases Are Fast","excerpt":"Storing a table by column instead of by row changes which queries are cheap. It is the single biggest reason an analytics query that crawls in Postgres flies in a warehouse.","category":"code","date":"August 14, 2026","author":"ByteLedger Team","content":"<span class=\"dropcap\">A</span> query that takes four minutes on your production Postgres finishes in under a second on a warehouse holding the same data. The hardware is comparable. The query is identical. The difference is which direction the data is written to disk.\n\nRow-oriented storage keeps each record's fields adjacent — everything about customer 4,192 sits together. Columnar storage keeps each field's values adjacent — every customer's signup date sits together. Both are reasonable. They are optimised for opposite questions.\n\n## What changed in 2026\n\n- **Open table formats displaced proprietary ones.** Columnar files plus an open metadata layer became the default way to store analytical data, which decoupled storage from whichever engine queries it.\n- **Row and column stopped being an either/or.** Transactional databases gained columnar extensions and analytical engines gained faster point lookups, narrowing a gap that used to force an architectural choice.\n- **Vectorised execution became the expected companion.** Columnar layout enables processing values in batches through CPU vector instructions, and engines that do not exploit that leave most of the benefit unclaimed.\n- **Object storage became the default substrate.** Columnar files on cheap object storage, queried directly, replaced a great deal of dedicated warehouse infrastructure.\n\n## Row vs columnar\n\n| | Row store | Columnar store |\n| --- | --- | --- |\n| Layout | All fields of a record together | All values of a field together |\n| Fast at | Fetching or writing whole records | Scanning a few columns over many rows |\n| Slow at | Wide scans over few columns | Single-row reads and updates |\n| Compression | Modest — mixed types adjacent | Strong — uniform types adjacent |\n| Typical use | Application databases | Analytics, reporting, warehouses |\n| Write pattern | Frequent small updates | Bulk append |\n\nThe intuition worth holding: a query's cost is roughly what it reads, not what it returns. `SELECT AVG(amount) FROM orders` over a 200-column table reads one column in a columnar store and all 200 in a row store. That is not a tuning difference, it is a different amount of work.\n\n## Why compression works so much better\n\nThis is the part that surprises people, and it compounds with the first benefit rather than merely adding to it.\n\nIn a row store, adjacent bytes are a name, then a timestamp, then a boolean, then a float. Mixed types, mixed ranges, little repetition — general-purpose compression has little to exploit.\n\nIn a columnar store, adjacent bytes are ten thousand consecutive timestamps, or ten thousand values from a set of five country codes. Now specialised encodings apply. Run-length encoding collapses repeated values. Dictionary encoding replaces repeated strings with small integers. Delta encoding stores differences between sorted numbers rather than the numbers. A sorted timestamp column can compress by an order of magnitude or more.\n\nThe effect compounds because compressed data is less data to read from disk *and* less to move into CPU cache. Many engines operate directly on the encoded representation — filtering dictionary-encoded values by comparing integers rather than strings — so decompression is often partially avoided rather than merely amortised.\n\nSort order matters enormously here and is the most underused lever available. Sorting by a low-cardinality column before writing dramatically improves both compression and the effectiveness of row-group skipping.\n\n## Where it falls down\n\nFetching one complete row means reading from every column's storage — the exact inverse of the analytical case, and columnar layout is genuinely bad at it. Updating one field of one row can require rewriting a whole block, since columnar files are typically immutable.\n\nThis is why columnar storage did not simply replace row storage. An application serving user profiles does point lookups and small updates constantly; that is a row store's home ground. The industry answer has been to run both — a transactional row store for the application, columnar storage for analytics, and a pipeline moving data between them. [Data lake vs data warehouse](/blog/data-lake-vs-data-warehouse-2026) covers how those pieces usually fit together.\n\nModern formats hedge by organising data into row groups with columnar layout inside each. Statistics per group — minimum, maximum, null counts — let a query skip entire groups without reading them, which recovers some point-lookup ability. It is a genuine improvement and still not a substitute for an index on a transactional table.\n\n## Common mistakes\n\n- **Using it for transactional workloads.** Frequent single-row updates against columnar storage is the wrong tool applied with conviction.\n- **`SELECT *` in an analytical query.** It discards the entire benefit by asking for every column.\n- **Ignoring sort order at write time.** Unsorted data compresses worse and skips fewer row groups. This is often the largest single win available.\n- **Very small files.** Metadata overhead and lost compression context. Small files are the most common cause of a slow query layer over object storage.\n- **Row groups sized badly.** Too small loses compression, too large defeats skipping.\n- **Expecting an index to fix it.** Columnar engines rely on scanning efficiently and skipping blocks, not on the index structures a row store uses.\n\n## FAQ\n\n**Is Parquet a database?**\nNo — it is a file format. Engines read and write it; the format itself stores columnar data with per-row-group statistics. Storing data in an open format rather than an engine's internal one is what lets several query engines read the same files.\n\n**Can Postgres do columnar?**\nThrough extensions and adjacent tooling, yes, and it can be a good middle path when your analytical volume does not justify separate infrastructure. Beyond a certain scale a purpose-built engine wins clearly. [Postgres 18 features](/blog/postgres-18-features-2026) covers what the core has been adding.\n\n**Does columnar storage help with joins?**\nSomewhat — reading fewer columns means less data to join. But join strategy, data distribution, and shuffle cost usually dominate, and none of those are storage-layout questions.\n\n**How does this relate to a lakehouse?**\nA lakehouse is essentially columnar files on object storage plus a metadata layer providing transactions and schema evolution. Columnar storage is the substrate — [lakehouse architecture explained](/blog/lakehouse-architecture-explained-2026) covers what gets built on top.\n\n## Where to go next\n\nFor where columnar storage sits in a broader data platform, read [data lake vs data warehouse](/blog/data-lake-vs-data-warehouse-2026) and [lakehouse architecture explained](/blog/lakehouse-architecture-explained-2026). For the row-store side and what it optimises instead, [database indexing explained](/blog/database-indexing-explained-2026).","hue":316,"metaTitle":"Columnar Storage Explained: Fast Analytics Queries","metaDescription":"How columnar storage makes analytical queries fast, why compression works better on columns, and when a row store is still the right choice for your workload.","keywords":["columnar storage","column oriented database","row vs columnar storage","olap vs oltp storage","parquet format","analytics database performance","column compression","data warehouse storage 2026"],"tldr":["**Columnar layout reads only the columns you asked for** — a five-column query against a 200-column table touches a fortieth of the data.","**Compression works far better on columns** — values of one type and similar range sit together, so encoding schemes get dramatic ratios.","**Single-row operations are the weakness** — fetching or updating one complete row means touching every column file.","**Most engines are hybrid now** — row groups with columnar layout inside, which is why the row-versus-column framing is less absolute than it was.","**Skip:** columnar storage for transactional workloads. Point lookups and frequent single-row updates are exactly what it is bad at."],"readTime":"5 min read","wordCount":1057},"related":[{"slug":"duckdb-vs-postgres-2026","title":"DuckDB vs Postgres in 2026: When Embedded OLAP Wins","excerpt":"DuckDB and Postgres solve different problems: one is a columnar engine for fast scans, the other a row-oriented database for concurrent transactions. Here is when embedded OLAP actually beats a Postgres query.","category":"code","date":"August 1, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":306},{"slug":"toast-storage-explained-2026","title":"TOAST Storage: Where Large Values Actually Live","excerpt":"A row containing a large text or JSON value does not store it inline. It goes to separate storage, which explains why SELECT * can be dramatically slower than selecting columns.","category":"code","date":"February 9, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":309},{"slug":"time-series-databases-2026","title":"Time-Series Databases: When a Normal Table Stops Coping","excerpt":"Metrics, events, and sensor readings share a shape ordinary databases handle badly at volume. Purpose-built stores exploit that shape, and a partitioned relational table gets you most of the way.","category":"code","date":"January 13, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":318}],"sameCategory":[{"slug":"bazel-build-explained-2026","title":"Bazel Explained: What Hermetic Builds Actually Cost","excerpt":"Bazel delivers reproducible, precisely cached builds and demands that you describe everything. Here is what you get, what it costs, and who should say no.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":145},{"slug":"graphql-federation-2026","title":"GraphQL Federation: One Graph, Many Teams","excerpt":"Federation composes independently-owned subgraphs into a single API. Here is what it solves, the operational cost it adds, and when a simpler gateway is enough.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":300},{"slug":"hermetic-builds-explained-2026","title":"Hermetic Builds Explained: Same Inputs, Same Output","excerpt":"A hermetic build depends only on declared inputs. Here is what that property gives you, the common leaks that break it, and how to move toward it incrementally.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":130},{"slug":"monorepo-tooling-compared-2026","title":"Monorepo Tooling Compared: What You Actually Need","excerpt":"Monorepo tools solve task orchestration and caching, not repository layout. Here is what each tier provides and how to tell when you have outgrown the simple option.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":150}],"newer":{"slug":"cloud-gaming-latency-2026","title":"Cloud Gaming Latency: Where the Milliseconds Go","excerpt":"Cloud gaming adds delay at several points and the network is only one of them. Here is the full latency budget and which games tolerate it.","category":"tech","date":"August 14, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":236},"older":{"slug":"context-rot-explained-2026","title":"Context Rot Explained: Why Long Prompts Get Worse","excerpt":"A million-token window does not mean a million usable tokens. Here is what context rot is, where accuracy actually falls off, and how to keep long prompts sharp.","category":"ai","date":"August 14, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":218}}