Instruction: Explain what an activation function is and its purpose within a neural network.
Context: This question evaluates the candidate's knowledge of neural networks, focusing on the critical role of activation functions.
Official answer available
Preview the opening of the answer, then unlock the full walkthrough.
The way I'd explain it in an interview is this: The activation function introduces nonlinearity into the network. Without it, stacking layers would still behave like one big linear transformation, which would make deep networks far less expressive than people assume.
In practice, the activation function determines...
easy
easy
hard
hard