Your customer rollout corrupts downstream records, and reverting the software cannot undo them. How would you choose the recovery plan?

Example Answer

I'd stop the affected writing path and preserve operation identifiers, versions and source evidence. Reverting the software may contain the defect, but it cannot tell us the correct state of records already changed. I'd identify the affected scope before choosing a repair.

I'd compare targeted compensation, restore and replay with the customer data owner. A restore can erase legitimate later work; replay needs trustworthy history and ordering; compensation needs evidence for each correction. I'd choose the smallest repair we can validate while protecting valid changes made after the incident.

I'd test it on an isolated copy or permitted sample, obtain the required approval and reconcile the result in the customer system. Records whose intended state is unknown would remain unresolved, with an agreed restricted or manual workflow. I'd report recovered scope and remaining uncertainty separately.

Make it your own

Use the actual authoritative history and recovery permissions. No restore, replay or compensation method guarantees correctness without checking legitimate changes and unresolved records.

Why this works

Tests a recovery choice for persistent data damage, separating software mitigation from validated business-state repair.

Interviewer follow-up

Would you simply restore the latest backup?

Only after checking its coverage, restore point and the legitimate work we would lose. I'd compare that loss with a targeted repair and establish how later valid changes would be preserved or replayed safely. If we cannot reconcile them, I'd state the risk to the authorized owners. A backup provides a recovery input; it does not prove the restored business state is correct.

Pressure test follow-up

The next dispatch window begins before full repair can finish. A subset of orders is independently reconciled. Would you resume dispatch?

I'd consider resuming only that verified subset if the downstream workflow can reliably exclude unresolved orders and the business owner approves the limited operation. I'd confirm that the recent change is contained and monitor the resumed path. If dispatch cannot enforce the boundary, I'd keep it paused or use an approved manual alternative. I'd make the lost capacity and next repair milestone explicit rather than treat partial reconciliation as complete recovery.

What this tests

The candidate must decide whether a verified partial recovery can support business continuity without allowing unresolved data back into the workflow.

Assessment criteria

These are practice criteria for this scenario, not an employer's scoring rubric.

• Strong: Contains the writer, establishes affected history and validates repair while protecting legitimate later work and isolating unknown records.
• Adequate: Distinguishes code rollback from data repair and compares restore, replay or compensation.
• Weak: Restores blindly or resumes a mixed correct/unknown record set without an enforceable boundary.

A tempting weak answer

"I'd restore the latest backup and reopen the workflow."

Why it fails: The backup can omit valid later changes and does not independently prove the business state is correct.

Other defensible approaches

Targeted compensating updates may preserve legitimate later work when operation history is trustworthy. Restore followed by verified replay can be better for broad damage when that history and recovery path are available. A reconciled subset may support limited continuity only when unresolved records can be reliably excluded. Choose from the evidence rather than from a favored recovery tool.

References

• Google SRE incident response