Which primary metric would you choose for an AI support assistant?

Instruction: The product goal is to resolve routine account-support requests correctly with less user effort. Choose a primary metric and guardrails. Explain why a shorter session could be a better outcome than more engagement.

Context: Choose a product-goal metric with a clear denominator and distinguish successful completion from interaction volume.

Updated

Example answer

I would use verified successful resolution per eligible support request, with a declared follow-up window. A resolution needs evidence that the request was correctly completed and did not require a repeat contact for the same unresolved problem; a user leaving the chat alone is not proof of success. Define eligibility and exclusions before comparing versions.

For this product, a shorter successful session can be better than a long conversation. More messages may mean confusion or repeated failed attempts. Track time/effort to successful resolution, repeat-contact rate, wrong-action rate, access to human help and satisfaction alongside the primary metric. Report important request and user slices, and include cost and latency guardrails.

Engagement can help diagnose adoption and workflow use, but it is not a universal measure of value. A creative companion or learning tool may have a different primary outcome. Choose the metric from the user's goal and validate its relationship to that goal rather than maximize whatever is easiest to count.

Related Questions