Skip to content

Problems Blog Pricing

Search problems

Loading…

Practice for AI, ML and GenAI engineering interviews. Answer out loud or in writing, then learn from the reference solution.

Start practicing

Practice

All problems
Easy warm-ups
Hard challenges
System design
Blog

Your space

Dashboard
Saved problems
Privacy
Terms

Account

Create free account
Log in

© 2026 Buncho. Built for engineers preparing for AI roles.

Back to top ↑

Loading…

Explain scaled dot-product attention | Buncho

00:00:00

249 of 565

Problems / Generative AI and LLMs

Explain scaled dot-product attention

MediumGenerative AI and LLMsConceptual12 min Save for later

Interview insights are lockedSee how often this is asked and which companies asked it with Buncho Pro.

Explain scaled dot-product attention.

Write the formula and explain what queries, keys and values represent.
Why is the score divided by sqrt(d_k)?
Implement it in PyTorch with an optional causal mask, and state its time and memory complexity.

Hints

Related questions

Why do gradients vanish in deep networks?Medium

AI feedback

Locked until you attempt this problem

Log in and submit an answer to get personalised AI feedback.

Community solutions

Log in and attempt this problem to read how other learners answered it.

Answer

Write your answer here. Your answer is private unless you choose to share it.

You need to log in / sign up to submit an answer and get AI feedback.