When would you choose search, retrieval-augmented generation, or fine-tuning?

Instruction: Recommend an initial approach for each need, explain the differences to a business sponsor, and identify what evidence would justify added complexity.

Context:

A fictional service business has three needs: employees must locate an exact policy clause; advisers must combine information from several approved policies that change weekly; and an existing assistant repeatedly produces the wrong response format despite clear instructions and examples. Policy access varies by employee. The sponsor proposes fine-tuning one model on every document.

Updated

Example Answer

I would separate these needs before choosing a technology. For finding an exact clause, I would start with permission-aware search and clear links. Generating prose may add little value to that task.

For combining current policies, I would test retrieval-augmented generation, or RAG: retrieve relevant, permitted passages and give them to a model as evidence for its response. I would require useful source references, current versions, and a way to decline when the evidence is insufficient. Retrieval can miss information, and a model can still misinterpret what it receives.

For the recurring format problem, I would first try a clearer output schema, validation, and representative examples. Fine-tuning may be worth testing if those simpler approaches fail and we have enough good examples of the desired behavior. It requires its own held-out evaluation and maintenance plan.

I would not use fine-tuning as the primary mechanism for weekly policy updates or employee permissions. Those need dependable content and authorization controls. I would compare each proposed solution with the current workflow on correctness, effort, access protection, and operating cost.

What the interviewer is assessing

Essential: Maps exact-clause lookup to search, current multi-document synthesis to retrieval plus generation, and the format problem first to a schema and validation; explains each choice in business terms.

Stronger: Explains that permission checks and content freshness remain separate controls. Accept a simpler search-based answer for the second need if it makes advisers do the synthesis and measures that tradeoff; fine-tuning needs a demonstrated behavior gap and independent evaluation.

Red flags: Says retrieval guarantees truth, uses fine-tuning as document authorization, or adds all three approaches without identifying what each improves.

Follow-up

Could RAG and fine-tuning be used together?

Example response: “Yes. Retrieval could supply current permitted evidence while a fine-tuned model follows a specialized response style. I would add the second component only if a measured problem remained and the improvement justified its cost and maintenance.”

Related Questions