Instruction: Describe how you would observe the three main operating dimensions of an LLM product together.
Context: Assesses whether the candidate can design a practical architecture and explain the main tradeoffs. Describe how you would observe the three main operating dimensions of an LLM product together.
Official answer available
Preview the opening of the answer, then unlock the full walkthrough.
I would instrument the workflow so cost, latency, and quality can all be sliced by request class, model route, customer segment, and release version. Those three dimensions should be inspectable together because optimizing one often moves the others.
I want stage-level latency, per-request and...
easy
easy
easy
easy
easy
easy