Skip to content
The phrase “we lost the backups” has gotten more expensive every year. Here are three incidents from the last 24 months, then the playbook for what to do when it’s your turn.
When the backup is the disaster
May 2024: UniSuper and Google Cloud. A Google Cloud provisioning misconfiguration caused deletion of UniSuper’s GCVE Private Cloud environment across two zones. Same-provider redundancy was not enough; recovery relied on Google Cloud Storage backups and third-party backup software, including backups held with another provider. UniSuper members lost access for more than a week, with all member-facing services restored by May 15.
February 2024: Change Healthcare. Ransomware operators sat inside Change Healthcare’s network for nine days, claimed to have exfiltrated about 6 TB of data, and then deployed ransomware that made core systems inaccessible. The breach ultimately affected about 192.7 million people. UnitedHealth paid a $22M ransom; the data was not reliably contained, and RansomHub later ran a follow-up extortion campaign.
July 2024: CrowdStrike. A faulty Falcon Sensor update crashed 8.5 million Windows machines worldwide. Recovery often required manual remediation or recovery-environment access. BitLocker-encrypted endpoints sometimes required 48-digit recovery keys, creating another bottleneck where key-management systems or directory services were unavailable.
Three different causes. Same lesson: recovery fails when the recovery path depends on the same control plane, identity system, region, tenant, or endpoint fleet that just failed.
The cost when backups don’t recover
For mid-size and large enterprises, one hour of unplanned downtime costs more than $300,000 in 90%+ of cases (source: ITIC’s 2024 survey). In large enterprises, downtime can run from hundreds of thousands to millions of dollars per hour, with finance, healthcare, aviation, and payments among the sectors where losses escalate fastest.
Infrascale’s 2025 report, based on AI-driven audience profiling of 71,353 U.S. technology leaders, found that 67.7% of businesses reported significant data loss in the prior 12 months.
Prolonged data loss can become existential, especially for firms with thin cash reserves, regulatory exposure, or customer-facing transaction systems.
📊 Wondering how much downtime might cost your org? Use our Downtime Cost Calculator to find out.
What to do in the first 24 hours
1. Own it publicly, fast
Post-mortems published within hours hold up better than corporate non-statements. GitLab’s 2017 post-mortem is still cited as a gold standard because they were transparent about exactly what failed and why. Customers forgive mistakes faster than they forgive cover-ups.
2. Check every unofficial copy
Before you assume the data is gone, look in:
- Developer laptops and IDE caches
- Git repositories, especially fixture, seed, and migration data
- Staging and QA environments
- BI tools that pulled extracts (Looker, Tableau, Power BI)
- Vendor exports sent over email
- CDN logs and analytics events for recent records
- Local copies on shared drives
You’re rarely as wiped as the panic suggests.
3. Pull cloud-native snapshots
Pull cloud-native snapshots and version history, but do not assume they survived. Their value depends on whether they sit behind different credentials, account boundaries, retention locks, or immutable policies.
4. Bring in incident response
If ransomware is in play, a forensics firm can sometimes recover data the attackers thought they’d destroyed. But it’s better not to rely on this. Have immutable backups in a separate account or even a separate cloud so your data is untouchable by ransomware.
5. Reverse-engineer the recent past
Customer-facing apps cache recent records in more places than you’d expect. Search logs, email confirmations, payment processor records, third-party APIs you’ve integrated with. It won’t replace a real backup, but it can reconstruct the last 24-48 hours while you rebuild.
Build a DR plan that actually runs
DR plans that fail in production almost always have one thing in common: they were never tested with the production team, in production conditions.
A disaster recovery plan that works includes:
- An on-call list that reflects the current org chart, not last year’s. Tested for reachability monthly.
- A comms plan with pre-approved customer language, regulatory disclosure timelines, and a named exec spokesperson.
- A current network map and asset inventory. CMDB drift kills more recoveries than people admit.
- Defined RPO and RTO per system tier, not one number for everything. Your billing database is not your internal wiki.
- Quarterly restore tests with timed dry runs. If you’ve never done this, your published RTO is fiction.
The new prevention baseline (cloud version)
The old 3-2-1 rule (3 copies, 2 media types, 1 offsite) has become 3-2-1-1-0. The extra ‘1’ is one immutable or air-gapped copy. The ‘0’ is zero errors verified by automated restore testing.
In practice:
- Immutable storage for at least one backup tier. AWS S3 Object Lock, Azure Immutable Blob.
- Cross-account or cross-tenant vaulting. If your backups live in the same AWS account or Azure subscription as production, ransomware can delete both with one set of credentials.
- MFA on every backup admin role. Most ransomware backup-deletion happens through compromised admin accounts.
- Automated restore verification, not just successful-job alerts. A green backup job does not prove the data restores. Test the restore.
- Documented retention by data class. Separate operational recovery targets from regulatory retention. HIPAA, PCI DSS, NIS2, DORA, and sector rules may require logs, records, audit evidence, or transaction data to be retained longer than your operational backup cycle.
How N2W fits in
N2W is policy-driven. You decide what to back up, when, where to store it, and how long to retain it. That includes routing recovery copies to a separate AWS account or Azure subscription and placing them under compliance-lock immutability, so they cannot be altered, deleted, or retention-shortened before expiry (not by ransomware, not by backup admins, not by the account root user). And, for even tighter security: N2W is deployed in your own cloud account, not as an external SaaS control plane.
If you’d rather not be one configuration mistake away from a UniSuper-style recovery scramble, start a free N2W trial.
