Why do gradients vanish in deep networks?
Interview insights are lockedSee how often this is asked and which companies asked it with Buncho Pro.Why do gradients vanish in deep networks?
- Show mathematically why the gradient for early layers shrinks.
- Estimate the gradient scale for a 20-layer sigmoid network.
- List the techniques that fix the problem and explain how each helps.