{"post":{"slug":"what-is-a-flaky-test-2026","title":"What is a flaky test in 2026? The test that cannot be trusted","excerpt":"A flaky test passes and fails on the same code without anything actually changing, and every one left unfixed teaches your team to ignore red builds. Here is how to find and fix them in 2026.","category":"code","date":"July 17, 2026","author":"ByteLedger Team","content":"<span class=\"dropcap\">A</span> flaky test is a test that passes sometimes and fails other times against the exact same code, with nothing meaningful changed between runs. It is not testing a real bug — it is testing something incidental: a race condition, a shared fixture, a network call that occasionally times out. The test itself is not lying about a code change; it is revealing that its own setup is unreliable.\n\n## What changed in 2026\n\n- **Flaky test detection became a built-in CI feature** at major providers, automatically tracking pass/fail history per test and flagging inconsistent ones without any extra tooling setup.\n- **Automatic quarantine workflows matured** — a detected flaky test gets pulled out of the required-to-pass set and filed as a ticket automatically, instead of silently blocking unrelated pull requests.\n- **Retry-until-green configurations came under scrutiny**, after teams noticed blanket retries were hiding hundreds of genuinely flaky tests instead of surfacing them for a fix.\n- **Root-cause categories became standardized in tooling reports** — timing, isolation, environment, and concurrency are now common labels, making it faster to triage a new flake.\n\n## The usual causes of flakiness\n\n| Cause | What is happening | Typical fix |\n| --- | --- | --- |\n| Timing / race conditions | Test asserts before an async action finishes | Await the actual condition, not a fixed delay |\n| Shared state | Tests run in parallel and collide on the same data | Isolate fixtures per test, avoid shared globals |\n| Real network calls | An external service is slow, rate-limited, or down | Replace with a mocked or local endpoint |\n| Test order dependency | A test relies on state left by a previous test | Make every test independent and self-contained |\n| Environment differences | Passes locally, fails in CI, or the reverse | Match environments; pin versions and locale/timezone |\n\nReal hardware-level nondeterminism is rare in application test suites. Almost every flaky test traces back to one of these five categories.\n\n## Finding the actual cause\n\n1. **Re-run the failing test in isolation**, many times, before touching anything else. If it never fails alone, the cause is likely shared state or ordering.\n2. **Check for unmocked network or filesystem calls.** A test hitting a real, uncontrolled dependency is one of the most common sources — see [what an API mock server is in 2026](/blog/what-is-an-api-mock-server-2026) for how to remove that dependency entirely.\n3. **Look for fixed sleeps and timeouts** (`sleep(500)`) instead of waiting on an actual condition or event.\n4. **Compare local and CI environments** for differences in timezone, locale, parallelism, or available resources.\n\n## How to handle a flaky test without losing trust in CI\n\nQuarantine it immediately: remove it from the required, blocking test set so it stops failing unrelated pull requests, but keep it running and visible so it is not forgotten. File a ticket the same day. A quarantined test with no ticket and no owner just becomes permanently ignored, which defeats the purpose of having it at all.\n\n## Common mistakes\n\n**Re-running a failed CI job until it goes green.** This makes the symptom disappear without fixing the cause, and it can hide a very real, occasional production bug behind \"it is just flaky.\"\n\n**Deleting the test instead of fixing the cause.** Sometimes justified if the test has no real value, but often the underlying issue — a race condition, an unmocked dependency — will surface again somewhere else.\n\n**Ignoring flakiness because \"it is probably nothing.\"** Left unaddressed, a handful of flaky tests teaches an entire team to distrust red builds, which is far more expensive than the original bug the test was meant to catch.\n\n## FAQ\n\n**Is a flaky test the same as a broken test?**\nNo. A broken test fails consistently and points at a real problem. A flaky test is inconsistent on identical code, which usually points at the test itself rather than the code under test.\n\n**Should flaky tests be deleted?**\nOnly as a last resort, after investigation shows the test has little value or the flakiness cannot reasonably be fixed. Deleting without investigating just removes coverage without removing the underlying risk.\n\n**How do I stop a flaky test from blocking my team?**\nQuarantine it out of the required checks immediately, file a ticket, and keep it visible in reporting so it gets fixed rather than forgotten.\n\n**Can retries ever be an acceptable fix?**\nA single automatic retry for genuinely external, rate-limited services can be reasonable as a stopgap, but it should not be the default answer for internal test flakiness.\n\n## Where to go next\n\n- [What is regression testing in 2026](/blog/what-is-regression-testing-2026)\n- [What is a smoke test in 2026](/blog/what-is-a-smoke-test-2026)\n- [What is an API mock server in 2026](/blog/what-is-an-api-mock-server-2026)","hue":280,"metaTitle":"What Is a Flaky Test in 2026? Causes and Fixes","metaDescription":"What is a flaky test? A 2026 guide to why tests flake, the most common causes, how to quarantine them safely, and how to fix the root problems.","keywords":["flaky test 2026","what is a flaky test","flaky test causes","how to fix flaky tests","test flakiness ci cd","quarantine flaky tests","non deterministic test failures","flaky test detection tools","reduce test flakiness","unreliable test suite"],"tldr":["**A flaky test gives different results on identical code** — no change, no fix, just an inconsistent pass or fail from run to run.","**Timing, shared state, and network calls cause most flakiness** — real nondeterminism is rare; unmanaged side effects are the usual culprit.","**One flaky test trains a team to ignore red builds** — the real cost is not the test itself, it is the trust it quietly destroys.","**Quarantine, then fix, do not just re-run and move on** — re-running until green hides the problem instead of solving it.","**Skip:** deleting a flaky test instead of fixing it, unless it is genuinely testing nothing of value — deletion should be a last resort."],"readTime":"4 min read","wordCount":796},"related":[{"slug":"flaky-tests-how-to-fix-2026","title":"How to Fix Flaky Tests in 2026: A Diagnostic Playbook","excerpt":"Flaky tests erode trust in CI faster than any single bug, teaching engineers to ignore red builds. This 2026 playbook shows how to isolate the real cause instead of retrying until green.","category":"code","date":"August 2, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":312},{"slug":"unit-vs-integration-testing-2026","title":"Unit vs Integration Testing Explained 2026: Where Each Pays Off","excerpt":"Unit tests check one piece in isolation; integration tests check pieces working together. Here is the 2026 guide to the difference, the right balance, and how to avoid slow, brittle test suites.","category":"code","date":"July 1, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":150},{"slug":"what-is-mutation-testing-2026","title":"What is mutation testing in 2026? Testing your tests","excerpt":"Mutation testing checks whether your tests would actually catch a bug by deliberately breaking your code and seeing if anything fails. Here is how it works and when it is worth the extra CI time in 2026.","category":"code","date":"July 23, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":150}],"sameCategory":[{"slug":"bazel-build-explained-2026","title":"Bazel Explained: What Hermetic Builds Actually Cost","excerpt":"Bazel delivers reproducible, precisely cached builds and demands that you describe everything. Here is what you get, what it costs, and who should say no.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":145},{"slug":"graphql-federation-2026","title":"GraphQL Federation: One Graph, Many Teams","excerpt":"Federation composes independently-owned subgraphs into a single API. Here is what it solves, the operational cost it adds, and when a simpler gateway is enough.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":300},{"slug":"hermetic-builds-explained-2026","title":"Hermetic Builds Explained: Same Inputs, Same Output","excerpt":"A hermetic build depends only on declared inputs. Here is what that property gives you, the common leaks that break it, and how to move toward it incrementally.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":130},{"slug":"monorepo-tooling-compared-2026","title":"Monorepo Tooling Compared: What You Actually Need","excerpt":"Monorepo tools solve task orchestration and caching, not repository layout. Here is what each tier provides and how to tell when you have outgrown the simple option.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":150}],"newer":{"slug":"what-is-a-financial-moat-2026","title":"What Is a Financial Moat? A 2026 Investor Guide","excerpt":"A clear look at financial moats in 2026 — the main types of durable competitive advantage, how to spot one, and why moats erode over time.","category":"finance","date":"July 17, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":320},"older":{"slug":"what-is-a-formatter-vs-linter-2026","title":"Formatter vs linter in 2026: what each one actually does","excerpt":"A formatter rewrites layout; a linter judges the code for risk. Here is what each tool actually checks in 2026, why most projects need both, and how to stop them from fighting each other.","category":"code","date":"July 17, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":310}}