Instruction: Explain the concept of catastrophic forgetting and discuss strategies to mitigate its effects during the Transfer Learning process.
Context: The question tests the candidate's knowledge of a critical issue in Transfer Learning and their ability to implement strategies to prevent it, ensuring the robustness of the transferred model.
Official answer available
Preview the opening of the answer, then unlock the full walkthrough.
I would approach catastrophic forgetting by controlling how aggressively the pretrained model is updated. That can mean freezing early layers, using smaller learning rates for transferred weights, replaying source-like examples, or applying regularization methods that preserve...