How do you implement a transformer model from scratch for a specific NLP task?
Instruction: Discuss the key components and steps involved in implementing a transformer model, including attention mechanisms, positional encoding, and layer normalization.
Context: This question assesses the candidate's deep understanding of the transformer architecture and their ability to implement it from the ground up for a specific NLP application.
Updated
Official answer available
Read the opening below, then unlock the full answer and practical guidance.
I would start by scoping the task, tokenization, architecture size, training objective, and data pipeline before writing layers...