How do you assess the potential of generative AI in your product's context?

Example answer

I would start with the user task and decide whether generation is actually needed. Drafting an editable reply, summarizing supplied documents and predicting next month's demand are different problems. A generative model may help produce language, but fluent output does not establish factual correctness or better forecasts than a task-specific statistical model.

Consider an illustrative support-reply assistant. Compare the current manual workflow, approved templates and a model that drafts from permission-checked source material. The draft should cite or expose supporting evidence and preserve an explicit route to correction or human handling. Retrieval can provide relevant material; it does not guarantee that the model uses it correctly.

Before a pilot, define an evaluation set with routine requests, missing or conflicting sources, sensitive content and attempts to redirect the assistant. Score unsupported claims, omitted important facts, appropriate abstention, edit effort and total completion time. Include review time and model/retrieval costs in the business case. A high acceptance rate alone can hide people accepting incorrect drafts.

Use only data authorized for the purpose, review provider retention and training terms, and obtain appropriate advice on privacy, rights and disclosure obligations. Check for harmful or exclusionary outputs with affected users and domain reviewers. Model access does not authorize disclosure or an external action; those permissions need separate enforcement.

I would expand only if the workflow meets its quality and risk criteria at a useful cost. If reliable verification is unavailable for a consequential output, limit the use or choose another approach. The result should be a measured decision about this task, not a general claim that generative AI improves every product.