What makes retrieved context useful instead of just longer?

Example Answer

The way I'd think about it is this: Useful context is context that helps answer the specific question with enough support to trust the result. Longer context is not automatically better. If half the window is boilerplate, duplicates, or loosely related material, the model has more ways to get distracted while still sounding confident.

I look for three things. First, the evidence has to match the user’s intent, not just share keywords. Second, it has to be sufficient, meaning it contains enough detail to support the answer without forcing the model to invent missing steps. Third, it should be attributable to a source so the system can cite it and engineers can debug it.

Adding chunks can increase processing cost and introduce distracting or conflicting evidence, but its effect on latency and faithfulness depends on the model, serving setup and task. I would compare context sizes on representative queries, measure both outcomes, and keep the evidence needed to answer rather than optimize for either the shortest or longest prompt.

Common Poor Answer

A weak answer is, "More context is always better because the model can ignore what it does not need." In practice, noisy context often makes the answer less faithful and harder to debug.