How would you measure answer quality when there is no single exact correct output?

Example Answer

The way I'd approach it in an interview is this: When there is no exact target string, I judge the answer against the job it was supposed to do. That usually means a rubric around dimensions like factual support, completeness, actionability, tone, policy compliance, and whether the response moved the workflow forward.

I try to make those dimensions concrete enough that different reviewers can apply them consistently. For example, instead of asking whether an answer is "good," I ask whether it answered the user’s request, whether material claims were supported, whether it respected product boundaries, and whether it chose the right fallback when information was missing.

I also like pairwise comparisons for ambiguous outputs. Often it is easier and more stable to ask which of two answers better serves the task than to assign one absolute score. That tends to produce cleaner signals when multiple answers could all be acceptable.

Common Poor Answer

A weak answer is saying you would use BLEU, ROUGE, or exact-match style metrics for open-ended outputs. Those usually miss what makes the answer useful in product context.