Instruction: Describe how you would make model routing maintainable as the product changes.
Context: Assesses whether the candidate can design a practical architecture and explain the main tradeoffs. Describe how you would make model routing maintainable as the product changes.
Official answer available
Preview the opening of the answer, then unlock the full walkthrough.
I would keep the routing layer versioned, testable, and observable. Routing rules should be represented in a way engineers can diff, benchmark, and roll back without spelunking through prompt text.
I also want the layer to support shadow evaluation and partial rollout so...
easy
easy
easy
easy
easy
easy