Write a release memo from test results, a defect report and a log.

Question

Fictional candidate I42 contains a read-only stock audit screen and a new reservation path. Product values either a full or audit-only release. The release manager decides scope. Access correctness and nonnegative stock are nonwaivable. There is no schema migration. Test results on I42: audit functionality 30/30; audit access 10/10; new reservations 58/60; legacy reservations with the new-path flag off 20/20. These are distinct cases without retries. The two failures reproduce one defect: with stock=1, two concurrent requests both succeed (3/3 trials). Expected: one success and one out-of-stock response. The same interleaving passes on the legacy path. Log: R1 reads stock=1; R2 reads stock=1; R1 commits A, stock=0; R2 commits B, stock=-1. The code-level cause is unconfirmed. Configuration evidence: independently controlled server-side flags; clients cannot override them. With the new-path flag off, every reservation request, including direct API calls, reaches the legacy implementation. The 20 legacy checks used that configuration on I42. Audit uses the existing read API and makes no writes. Its access review, performance and audit-only configuration checks passed. On-call is staffed. Runbook: flag off audit for access violations or confirmed audit mismatches. Negative stock triggers incident response and suspension of affected reservations. Write the release recommendation and its evidence.

Accepted Answer

Example Answer

I recommend releasing only the read-only audit screen on I42, with the new reservation path disabled server-side. I would not approve the new reservation path: its confirmed race violates a nonwaivable inventory invariant. The two failed cases represent a consequential defect, even though the overall pass rate is high.

The supplied routing and concurrency evidence supports the narrower configuration. It demonstrates that all reservation requests use the tested legacy path when the new flag is off, while the audit screen has independent access, performance and configuration evidence. This is a specific verified scope reduction, not hiding the new UI and hoping the API is safe.

The release memo would attach the configuration and unresolved defect, name the release manager and operational owner, and make later activation a separate decision after repair and regression evidence. If the configuration no longer matches the tested routing or a mandatory gate changes, I would pause that narrower release too.

Release decision memo

Decision requested: Approve I42 with the read-only audit screen enabled and the new reservation path disabled server-side. Keep the new path blocked from activation.

Evidence: 118 of 120 required cases passed: approximately 98.33%. Audit functional/access cases passed; two new-path cases failed the same confirmed invariant. All 20 flag-off legacy cases passed, including the supplied concurrent interleaving and denied access. Required audit-only access, performance and configuration checks passed. Test counts do not replace risk assessment.

Scope	Decision and owner	Evidence or condition
Read-only audit	Release manager may approve; on-call operates	Passed independent checks; retain exact tested flags/routing and immediate flag-off triggers for access or audit mismatch.
New reservation path	Do not activate; engineering owns repair	Confirmed overselling and negative available stock. Fix and fresh concurrency/regression evidence are required; Product cannot waive the invariant.
Legacy reservation path	Continue under verified flag-off routing	All reservation requests, including direct API calls, use the tested legacy implementation. No schema change couples activation to this release.
Incident or configuration drift	On-call and release manager act	Any negative stock triggers incident response and suspension of affected reservations; reconcile impacted records. Changed routing or failed mandatory evidence suspends approval.

Follow-through: Keep both failed cases visible with a deferred feature disposition. Verify the active server configuration after deployment; later enablement requires a separate recorded decision. Turning a flag off prevents future exposure only after it takes effect and does not undo already incorrect reservations.

What the interviewer is assessing

Core checks (1 point each):

Rejects the new reservation path on its invariant failure.
Derives the audit-only option from actual artifact/routing evidence.
Keeps failed cases visible and names decision authority and incident actions.

Stronger (+1 after all core checks): Interprets 118/120 correctly without letting the percentage override defect severity, avoids declaring a code-level root cause from logs alone, and requires fresh concurrent-operation evidence before the new path returns. Deferring the whole candidate is defensible if tied to an actual deficiency in the stated configuration control, not a general claim that every known defect blocks every independent feature.

Acceptable alternatives: Another scope decision needs a stated evidence or business reason; no blanket zero-defect rule is supplied.

Red flags: Approves both features because 98% pass, treats a client-side hide as sufficient, says rollback repairs existing negative stock automatically, or silently marks the two failures passing because the feature is disabled.

Tempting answer that misses the point

Weak answer: Release all features because 118 of 120 tests passed.

Why it fails: Two failures violate a nonwaivable inventory invariant; aggregate pass rate cannot authorize that path.

Follow-up

Does the observed log prove which code change will fix the race?

Example response: “It proves that the tested operation allowed two successful reservations to consume one unit and violate the invariant. It suggests a concurrency-control problem, but not the exact defect or remedy. Engineering should inspect the transaction and storage semantics, implement an appropriate atomic or serialized decision, and test adversarial interleavings, conflicts, retries and stock conservation. I would validate behavior rather than prescribe a fix from one log alone.”

Pressure test

Changed constraint: Immediately before deployment, engineering reveals that the shared reservation library also changed, and the 20 flag-off legacy cases ran against the previous library rather than the packaged I42 artifact.

Adaptive response: “I would defer I42 deployment. The evidence supporting the safe legacy route no longer applies to the artifact we would ship, so the original narrow-release rationale is incomplete. Engineering can produce a candidate without that library change or test the actual packaged candidate against the required legacy concurrency and access gates. I would keep the last approved release operating while the missing evidence is resolved.”