Why do tests matter differently for coding agents than for chatbots?

Example Answer

The way I'd think about it is this: For chatbots, tests mostly evaluate behavior. For coding agents, tests are part of the execution loop itself. They are both a validation mechanism and a navigation signal telling the agent whether its patch actually respects the repo’s expected behavior.

They also matter because coding agents can make changes with real blast radius. Fluent explanations are irrelevant if the patch quietly breaks adjacent logic. Tests turn "looks plausible" into something closer to "survives contact with the codebase."

I still treat tests as incomplete evidence, but they are far more operationally central for coding agents than for ordinary chat systems.

What matters in an interview is not only knowing the definition, but being able to connect it back to how it changes modeling, evaluation, or deployment decisions in practice.

Common Poor Answer

A weak answer is saying tests matter more simply because code is technical. The deeper reason is that tests are part of the execution and validation loop.