The backup job has succeeded every night for two years. The monitoring is green. Nobody has ever restored one.
That is not a backup strategy; it is a file-writing strategy with a strong assumption attached. The failure modes are numerous and all silent: the backup excludes a schema added last year, archives stopped being retained after a configuration change, the encryption key is not stored anywhere recoverable, or the file is complete and corrupt.
Every one of those reports success, and every one is discovered during an incident.
What changed in 2026
- Automated verification became standard practice. Scheduled restore-and-validate pipelines moved from advanced to expected.
- Managed platforms added verification features. Some services began offering automated restore testing.
- Recovery time entered service objectives. Measuring restore duration became part of defining reliability targets.
- Ransomware raised the stakes. Immutable backups and verified recoverability became a mainstream concern rather than a specialist one.
What a successful job does not prove
| Job reports |
Actually verified |
| Backup completed |
A file was written |
| No errors |
The process did not crash |
| Expected file size |
Roughly the right amount of data |
| Checksum matches |
The file is intact as written |
| — |
The file restores |
| — |
The restored database works |
| — |
It contains what you need |
| — |
How long it takes |
Everything below the line requires actually restoring. Nothing above it does.
The gap that catches people most often is coverage: a backup that has been running successfully for two years may not include the tables, schemas, or extensions added since it was configured.
Automate it
Verification requiring someone to remember does not happen. The only reliable approach is a scheduled pipeline.
The shape: take the latest backup, restore it to a temporary instance, run validation checks, record how long it took, tear the instance down, and alert on failure.
Frequency depends on how much change your system sees. Weekly is a reasonable default; daily for systems where the backup configuration changes often. The important property is that it runs without anyone deciding to run it.
Cost is modest — a temporary instance for the duration of a restore — and small against the cost of discovering a bad backup during an outage.
Validate the data, not just the restore
A restore completing tells you the file was readable. It does not tell you the data is right.
Useful checks after restore:
Row counts against expectations for key tables. A restored database with an empty orders table completed successfully and is useless.
Recent data present. Query for something written shortly before the backup. This catches archives that stopped being retained.
Schema completeness. Every expected table, index, constraint, and extension. Catches coverage drift.
Application-level smoke test. Point a test instance of your application at the restored database and exercise a few paths. Catches problems no data check would.
Duration recorded. Recovery time is a number you should know in advance, and it changes as data grows.
That last one deserves emphasis. Teams frequently have a recovery time objective and no measurement of actual recovery time. The gap is discovered when it matters.
Common mistakes
- Treating job success as verification. It proves a file exists.
- Manual verification. It stops happening.
- Checking only that the restore completed. Not that the data is correct.
- Not measuring duration. Recovery time unknown until an incident.
- Backups on the same infrastructure as the database. One failure takes both.
- Encryption keys stored only in the system being backed up. Unrecoverable backups.
- No test of point-in-time recovery specifically. Restoring the latest backup is not the same operation — see point-in-time recovery.
FAQ
How often should I verify?
Weekly as a baseline, more often if your schema or backup configuration changes frequently. The key property is automation rather than the exact interval.
Can I verify without a full restore?
Partially — file integrity checks catch corruption. They do not catch coverage gaps, missing archives, or unusable output. Only a restore proves a restore.
What about very large databases?
Restore cost is real at scale, and so is the risk. Options include verifying a subset regularly and the whole thing periodically, or restoring to cheaper storage. Not verifying because it is expensive is the wrong conclusion.
Should I test the runbook too?
Yes — a verified backup with an unpracticed recovery procedure still produces a slow, error-prone incident. That is what a restore drill is for — see restore drills.
Where to go next
For practising the human side, read restore drills. For the recovery mechanism itself, point-in-time recovery, and for testing failover separately, failover testing.