{"post":{"slug":"toast-storage-explained-2026","title":"TOAST Storage: Where Large Values Actually Live","excerpt":"A row containing a large text or JSON value does not store it inline. It goes to separate storage, which explains why SELECT * can be dramatically slower than selecting columns.","category":"code","date":"February 9, 2026","author":"ByteLedger Team","content":"<span class=\"dropcap\">A</span> database stores rows in fixed-size pages. A row must fit in a page, which puts a hard ceiling on how large a single row can be — and a column holding a long document or a large JSON structure would exceed it easily.\n\nThe mechanism that resolves this moves oversized values out of the main table into separate storage, leaving a pointer behind. It is transparent, and it produces performance behaviour that looks arbitrary until you know it exists.\n\n## What changed in 2026\n\n- **JSON columns made it common.** Storing documents in relational tables put large values into far more schemas — see [JSON columns](/blog/json-columns-explained-2026).\n- **Compression options expanded.** Choice of compression algorithm per column became more widely available.\n- **The `SELECT *` cost became better known.** The relationship between column selection and out-of-line fetches entered general awareness.\n- **Storage strategy tuning stayed niche.** Per-column strategy settings remained available and rarely adjusted.\n\n## What happens to a large value\n\nWhen a row would exceed the page limit, the database works through options in order:\n\n**Compress the value.** Frequently enough to bring the row under the limit, in which case nothing is moved. Text and JSON compress well.\n\n**Move it out of line.** If still too large, the value is split into chunks and stored in a separate associated table, with the main row keeping a reference.\n\n**Both.** Compress and then move, for very large values.\n\nThe result is that a table with a large text column has two physical structures: the main table holding ordinary columns and pointers, and a secondary store holding the large values.\n\n## Why column selection matters so much\n\nThis is the practical consequence and the reason to care.\n\nA query that does not reference the large column reads only the main table. Those pages are small, so many rows fit per page, and scans are fast — the large values are simply not touched.\n\nA query that *does* reference it must additionally fetch the value from the secondary store, which is an extra read per row.\n\n| Query | Reads | Relative cost |\n| --- | --- | --- |\n| `SELECT id, name` | Main table only | Low |\n| `SELECT id, document` | Main table plus out-of-line fetch per row | High |\n| `SELECT *` | Everything, always | Highest |\n| `WHERE document @> ...` | Must fetch to evaluate | High |\n\nSo `SELECT *` on a table with large columns is not slightly more expensive than selecting what you need — it can be an order of magnitude worse, because every row triggers an additional fetch of data you are discarding.\n\nThis is the single most actionable thing about the mechanism: **name your columns**. It is advice people follow for style reasons and the performance reason is larger than the style one here.\n\nIt also explains an observation that otherwise looks strange: adding a large column to a table does not slow down existing queries that ignore it. The data is not in the pages those queries read.\n\n## Updates rewrite the whole value\n\nOut-of-line values are not updated in place. Changing one character in a large document writes an entirely new copy, with the old one remaining until cleanup reclaims it — see [MVCC explained](/blog/mvcc-explained-2026).\n\nOn a table where large values are updated frequently, that produces substantial write volume and bloat. A JSON document updated on every request, with one field changing, rewrites the whole document every time.\n\nWhere that pattern appears, splitting the frequently-updated field into its own small column — or a separate table — avoids rewriting the large value repeatedly. Generated columns do not help here, since they derive from the document rather than replacing it.\n\n## Common mistakes\n\n- **`SELECT *` on tables with large columns.** The dominant avoidable cost.\n- **Frequent small updates to large values.** Rewrites everything each time.\n- **Filtering on a large column's contents without an index.** Fetches every value to evaluate.\n- **Assuming column count affects scan speed uniformly.** Untouched large columns are free.\n- **Not measuring the secondary storage size.** It can dwarf the main table.\n- **Storing large values that are never queried.** Consider whether they belong in the database at all.\n\n## FAQ\n\n**Does this apply to all databases?**\nThe specific mechanism and naming vary by engine; the general behaviour — large values stored separately with a pointer — is common. Check the details for yours.\n\n**Can I control it?**\nMost engines expose a per-column storage strategy, letting you prefer or avoid compression and out-of-line storage. Rarely worth adjusting unless you have measured a specific problem.\n\n**How do I see how much space it uses?**\nEngines expose the size of the associated storage separately from the main table. Worth checking on tables with large columns; the ratio is frequently surprising.\n\n**Should large files go in the database at all?**\nFrequently not. Object storage with a reference in the database is usually better for genuinely large binary content, and keeps your database smaller and backups faster.\n\n## Where to go next\n\nFor the column type this most affects, read [JSON columns](/blog/json-columns-explained-2026). For the version churn updates produce, [MVCC explained](/blog/mvcc-explained-2026), and for indexing inside stored documents, [GIN indexes](/blog/gin-indexes-explained-2026).","hue":309,"metaTitle":"TOAST Storage: Where Large Values Actually Live","metaDescription":"How databases move oversized column values to separate storage, why SELECT * becomes expensive, and how compression and updates interact with it.","keywords":["toast storage","out of line storage","large value columns","select star performance","column compression database","postgres toast","row size limits","database internals 2026"],"tldr":["**Large values move out of the main table** — the row keeps a pointer, not the data.","**Queries not touching them never pay for them** — which is why column selection matters so much.","**Values are compressed before being moved** — sometimes avoiding the move entirely.","**Updating a large value rewrites it** — even if you changed one character.","**Skip:** SELECT * on tables with large columns. The cost difference is not marginal."],"readTime":"4 min read","wordCount":875},"related":[{"slug":"columnar-storage-explained-2026","title":"Columnar Storage Explained: Why Analytics Databases Are Fast","excerpt":"Storing a table by column instead of by row changes which queries are cheap. It is the single biggest reason an analytics query that crawls in Postgres flies in a warehouse.","category":"code","date":"August 14, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":316},{"slug":"generated-columns-explained-2026","title":"Generated Columns: Let the Database Compute It","excerpt":"A value derived from other columns can be maintained by the database itself, indexed like any other column, and never fall out of sync with its inputs.","category":"code","date":"August 15, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":304},{"slug":"enum-vs-lookup-table-2026","title":"Enum vs Lookup Table: Where Should Your Fixed Values Live?","excerpt":"A status column with five possible values can be a database enum, a lookup table, or a check constraint. The right answer depends almost entirely on who needs to change the list.","category":"code","date":"January 9, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":315}],"sameCategory":[{"slug":"bazel-build-explained-2026","title":"Bazel Explained: What Hermetic Builds Actually Cost","excerpt":"Bazel delivers reproducible, precisely cached builds and demands that you describe everything. Here is what you get, what it costs, and who should say no.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":145},{"slug":"graphql-federation-2026","title":"GraphQL Federation: One Graph, Many Teams","excerpt":"Federation composes independently-owned subgraphs into a single API. Here is what it solves, the operational cost it adds, and when a simpler gateway is enough.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":300},{"slug":"hermetic-builds-explained-2026","title":"Hermetic Builds Explained: Same Inputs, Same Output","excerpt":"A hermetic build depends only on declared inputs. Here is what that property gives you, the common leaks that break it, and how to move toward it incrementally.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":130},{"slug":"monorepo-tooling-compared-2026","title":"Monorepo Tooling Compared: What You Actually Need","excerpt":"Monorepo tools solve task orchestration and caching, not repository layout. Here is what each tier provides and how to tell when you have outgrown the simple option.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":150}],"newer":{"slug":"sovereign-gold-bonds-2026","title":"Sovereign Gold Bonds 2026: tranches, pricing, and how SGB compares to gold ETFs and physical gold","excerpt":"Sovereign Gold Bonds in 2026 — the new issuance status, the pricing math, and the honest comparison with gold ETFs and physical gold.","category":"finance","date":"February 9, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":314},"older":{"slug":"zero-based-budgeting-2026","title":"Zero-based budgeting 2026: YNAB-style envelopes vs apps, when ZBB is worth it","excerpt":"Zero-based budgeting in 2026 — YNAB methodology, the envelope approach, and whether ZBB is worth the effort vs simpler 50/30/20.","category":"finance","date":"February 9, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":286}}