What is a practical LLM use case, and how would you evaluate it?
Instruction: Explain your approach, give a practical example, and describe its limits.
Updated
Example Answer
One practical use case is helping support staff find an answer in a large policy library. I would retrieve the documents the staff member is allowed to access, then ask the model to draft a short answer with references to the relevant passages. The staff member can inspect the evidence before using the draft.
I would evaluate correct policy selection, unsupported statements, response time, and the amount of review needed. The system should show when evidence is missing or conflicting. I would compare it with the existing search workflow before claiming that it saves time; a polished demo does not establish that the benefit holds on real cases.
Concrete example
Include two policies with different effective dates and a question that neither document answers. Measure whether the system selects the current source and recognizes the missing evidence.
Follow-up to practice
When would better search alone be enough to solve this problem?