Multi-Head Self-Attention in NumPy (Batched)

Asked at TikTok · Coding Questions · Difficulty: Medium · Last asked Nov 2, 2026

Attention lets every position in a sequence read from every other position, and a transformer layer runs several such readers (heads) side by side. Implement the forward pass of batched multi-head self-attention with NumPy: one function that takes a batch of sequences and four weight matrices and returns a new sequence of the same shape.

Loading the problem…