Scaled Dot-Product Attention with Masking

Asked at Meta · Coding Questions · Difficulty: Medium · Last asked Jun 30, 2026

Implement the scaled dot-product attention used in Transformer layers, using plain Python lists instead of a tensor library so that every shape is explicit. Each query compares itself with every key, turns the scores into weights with a softmax, and returns the weighted average of the value rows. An optional mask removes some keys from a query's view.

Loading the problem…