A customer wants an AI agent to approve refunds even though the policy is a fixed set of rules. What would you propose for the first pilot?

Example Answer

I’d clarify the refund policy with its owner and establish which system contains the authoritative purchase and eligibility facts. For the first pilot, I’d implement those fixed rules as a testable eligibility check, keep exceptions with the customer’s existing approval process and show the operator why a request qualifies or needs review. That gives us a clear deliverable before introducing model uncertainty.

If customers submit unstructured messages, AI might help draft a summary or extract a proposed order reference. I’d verify that information against the trusted record before using it. A model’s interpretation would not override refund limits, user permissions or required approval, and the pilot could remain read-only while the customer validates the decisions.

I’d test approved policy examples and compare handling time and incorrect decisions with the current process. We’d decide separately whether automated execution is justified and supported by the payment system. I’d explain that I’m trying to improve their workflow; if rules alone solve the bottleneck, a working, auditable implementation is a better first result than an agent choosing its own policy.

Make it your own

Choose a concrete first pilot and separate extraction, eligibility and refund execution. Treat policy and authorization as enforced customer rules rather than model judgment.

Why this works

Assesses whether the candidate can challenge an AI request constructively and ship a simpler customer deliverable with explicit action boundaries.

Interviewer follow-up

The sponsor insists that the demo must use AI. What would you offer?

I’d offer a bounded AI component if it addresses a real friction point, such as turning a message into a reviewable summary. I’d show the rule-based eligibility result beside it and demonstrate that incorrect extracted details cannot authorize a refund. I’d agree that scope with the policy owner. If the demand is for the model to bypass those controls, I’d explain why that cannot be the pilot.

Assessment criteria

These are practice criteria for this scenario, not an employer's scoring rubric.

• Strong: Separates deterministic eligibility, approved decision authority and any useful AI extraction step; tests exceptions and missing facts.
• Adequate: Checks policy and proposes a bounded rules-first pilot.
• Weak: Lets model output override the customer's fixed eligibility policy.

A tempting weak answer

"I'd ask the model to interpret the refund policy and approve eligible requests."

Why it fails: A fluent interpretation is not an enforced rule or delegated authority to approve a refund.

References

• Google: Rules of Machine Learning
• OWASP: Excessive Agency