Instruction: Discuss the strategies LLMs use to deal with words that they have not encountered before.
Context: This question tests the candidate's knowledge of the mechanisms LLMs employ to maintain performance even when faced with novel input.
Official answer available
Preview the opening of the answer, then unlock the full walkthrough.
The way I'd approach it in an interview is this: Modern LLMs typically handle unknown words through subword or byte-level tokenization rather than treating them as one impossible token. That means a rare term, misspelling, or new word can often be decomposed...
easy
medium
medium
medium
hard
hard