{"post":{"slug":"bulk-insert-performance-2026","title":"Bulk Insert Performance: Loading Millions of Rows","excerpt":"Inserting a million rows one statement at a time takes hours. The same data loaded properly takes minutes, and the difference is almost entirely round trips and index maintenance.","category":"code","date":"January 2, 2026","author":"ByteLedger Team","content":"<span class=\"dropcap\">A</span> data import inserts one row per statement. Each one is a network round trip, a parse, a plan, an execution, and an index update. A million of those takes hours, and the database is barely working — it is waiting on the network.\n\nThe same load, done properly, finishes in minutes.\n\n## What changed in 2026\n\n- **Bulk copy paths became better known.** Using the dedicated bulk-load interface rather than batched inserts spread as standard practice.\n- **Staging table patterns matured.** Loading raw and then transforming inside the database became the default ETL shape.\n- **Index management during load got documented.** The drop-and-rebuild decision became better understood.\n- **Unlogged and minimally-logged options got attention.** Reducing write-ahead log volume during initial loads became a recognised technique.\n\n## Where the time goes\n\n| Cost | Row-by-row | Batched | Bulk copy |\n| --- | --- | --- | --- |\n| Network round trips | One per row | One per batch | One stream |\n| Statement parsing | Per row | Per batch | None |\n| Index maintenance | Per row | Per row | Per row |\n| Transaction overhead | Per row, if autocommit | Per batch | Per batch |\n| Log volume | High | High | Lower in some modes |\n\nRound trips dominate at the start. A batch of a thousand rows in one statement eliminates 999 round trips, and that single change frequently accounts for most of the available improvement.\n\nIndex maintenance becomes the limiting factor once round trips are handled, because every index on the table is updated for every row regardless of how the rows arrive.\n\n## Batch the statements and the commits\n\nTwo separate decisions people conflate.\n\n**Statement batching** puts many rows in one insert. Reduces round trips and parsing.\n\n**Transaction batching** commits every N rows rather than per row or once at the end.\n\nBoth matter. Autocommit per row means a durable write per row, which is the other reason row-by-row loading is slow — see [write-ahead logging](/blog/write-ahead-logging-explained-2026).\n\nThe opposite extreme — one transaction for the entire load — has its own problems: a long-running transaction blocking cleanup, an enormous rollback if it fails at 90%, and no progress visibility — see [MVCC explained](/blog/mvcc-explained-2026).\n\nCommitting every few thousand rows is the usual balance. Failures lose one batch rather than everything, and transactions stay short.\n\n## Index management\n\nFor a large load into an existing table, index maintenance per row can exceed the insert cost.\n\n**Dropping indexes before the load and rebuilding after** is frequently much faster, because building an index in bulk is far more efficient than maintaining it incrementally.\n\nThat is only safe when the table is not serving queries during the load, since it is unindexed meanwhile. It also means the rebuild is a substantial operation of its own, so the saving must exceed it — generally true for a load that is large relative to existing table size.\n\nFor loading into an empty table, create indexes *after* loading rather than before. Same reasoning, no downside.\n\nConstraints follow similar logic, and dropping them requires confidence the data is clean — see [check constraints](/blog/check-constraints-explained-2026).\n\n## Look for per-row work\n\nBefore optimising the insert path, check what fires per row.\n\n**Triggers** are the usual culprit. A trigger doing anything non-trivial per row multiplies across a million rows, and it is frequently the dominant cost while everyone optimises the insert.\n\n**Foreign key checks** require a lookup per row on the referenced table, which needs an index on the referencing column or each check scans.\n\n**Generated columns** are computed per row, which is cheap individually and not free at scale.\n\nThe pattern that avoids all of this: **load raw into a staging table with no indexes, constraints, or triggers, then transform and move into the real table with set-based statements.** The database processes the transformation as a bulk operation rather than row by row, and it is usually dramatically faster than doing the work during ingestion — see [COPY bulk loading](/blog/copy-bulk-loading-2026).\n\n## Common mistakes\n\n- **Row-by-row inserts.** Round trips dominate.\n- **Autocommit per row.** A durable write per row.\n- **One transaction for the whole load.** Long transaction, huge rollback risk.\n- **Maintaining indexes during a large load.** Drop and rebuild instead.\n- **Ignoring triggers.** Frequently the real cost.\n- **Missing indexes on foreign key columns.** Every check scans.\n- **Transforming during ingestion.** Stage raw, transform in bulk.\n\n## FAQ\n\n**What batch size should I use?**\nLarge enough to amortise round trips, small enough that a failure does not lose much and transactions stay short. Thousands rather than tens or millions; measure on your setup.\n\n**Should I disable constraints?**\nOnly with confidence the data is clean, and re-enabling requires validating existing rows anyway. Frequently not worth it unless the load is very large.\n\n**What about parallel loading?**\nEffective when the target supports concurrent writes without contention. Parallel loads into the same indexes can contend, so measure rather than assuming linear improvement.\n\n**Does this apply to updates too?**\nThe same principles — batch, watch triggers, mind transaction size. Updates additionally create dead row versions, so a large update has cleanup consequences a load does not.\n\n## Where to go next\n\nFor the dedicated bulk path, read [COPY bulk loading](/blog/copy-bulk-loading-2026). For the transaction-size considerations, [MVCC explained](/blog/mvcc-explained-2026), and for index rebuild costs, [index bloat](/blog/index-bloat-explained-2026).","hue":308,"metaTitle":"Bulk Insert Performance: Loading Millions of Rows","metaDescription":"Why row-by-row insertion is slow, how batching and index management change bulk load times, and when to drop indexes before loading.","keywords":["bulk insert performance","batch insert","data loading speed","index rebuild after load","transaction batching","etl performance","database import","database operations 2026"],"tldr":["**Round trips dominate row-by-row insertion** — batching is the single largest win available.","**Index maintenance is the second cost** — dropping and rebuilding beats maintaining during a large load.","**One enormous transaction is its own problem** — batch the commits too.","**A bulk copy path beats even batched inserts** — where your database offers one.","**Skip:** optimising the insert when the real cost is a trigger firing per row."],"readTime":"4 min read","wordCount":894},"related":[{"slug":"copy-bulk-loading-2026","title":"Bulk Copy Loading: The Fast Path Into a Table","excerpt":"Databases have a dedicated interface for loading data that bypasses statement parsing entirely. It is dramatically faster than inserts and considerably less forgiving.","category":"code","date":"January 8, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":302},{"slug":"n-plus-one-query-problem-2026","title":"The N+1 Query Problem Explained in 2026","excerpt":"The N+1 query problem quietly turns one query into hundreds by firing a fetch per row inside a loop. Here is how to spot it, and the fixes that actually work.","category":"code","date":"July 2, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":280},{"slug":"bloat-monitoring-2026","title":"Bloat Monitoring: Measuring Space You Cannot See","excerpt":"A table twice the size of its live data reads twice as many pages for every scan. Bloat is invisible in ordinary metrics and degrades performance continuously until someone measures it.","category":"code","date":"January 7, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":320}],"sameCategory":[{"slug":"bazel-build-explained-2026","title":"Bazel Explained: What Hermetic Builds Actually Cost","excerpt":"Bazel delivers reproducible, precisely cached builds and demands that you describe everything. Here is what you get, what it costs, and who should say no.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":145},{"slug":"graphql-federation-2026","title":"GraphQL Federation: One Graph, Many Teams","excerpt":"Federation composes independently-owned subgraphs into a single API. Here is what it solves, the operational cost it adds, and when a simpler gateway is enough.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":300},{"slug":"hermetic-builds-explained-2026","title":"Hermetic Builds Explained: Same Inputs, Same Output","excerpt":"A hermetic build depends only on declared inputs. Here is what that property gives you, the common leaks that break it, and how to move toward it incrementally.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":130},{"slug":"monorepo-tooling-compared-2026","title":"Monorepo Tooling Compared: What You Actually Need","excerpt":"Monorepo tools solve task orchestration and caching, not repository layout. Here is what each tier provides and how to tell when you have outgrown the simple option.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":150}],"newer":{"slug":"best-ai-email-assistants-for-gmail-2026","title":"Best AI email assistants for Gmail in 2026: Superhuman vs Shortwave vs Notion Mail vs Gemini","excerpt":"We compared the AI email tools that actually save you 30+ minutes a day vs the ones that just summarise emails you'd skim anyway. Honest verdicts by inbox volume, work style, and whether you actually want to reply faster.","category":"ai","date":"January 2, 2026","author":"ByteLedger Team","readTime":"11 min read","hue":195},"older":{"slug":"deferrable-constraints-explained-2026","title":"Deferrable Constraints: Checking at Commit Instead of Immediately","excerpt":"Swapping two unique values fails because the intermediate state violates the constraint. Deferring the check to commit time lets the transaction pass through invalid states legally.","category":"code","date":"January 2, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":320}}