Why do we scale dot-product attention by the square root of d_k?
Interview insights are lockedSee how often this is asked and which companies asked it with Buncho Pro.Why do we scale the dot-product attention by in the Transformer architecture?
Why do we scale the dot-product attention by in the Transformer architecture?
Give the mathematical argument, not just the intuition.
Locked until you attempt this problem
Log in and submit an answer to get personalised AI feedback.
Log in and attempt this problem to read how other learners answered it.