Implement Scaled Dot-Product Attention
Asked at Google · Deep Learning & Architectures · Difficulty: Medium
Implement the forward pass of one scaled dot-product self-attention head with NumPy. Each token looks at every token that is allowed, takes a weighted average of their value vectors, and the weights come from how well its query matches their keys.
Loading the problem…
More Google interview questions
- LLM conceptual questions
- Implement a Nested Tic-Tac-Toe Game Engine
- ML Fundamentals Deep Dive (AI/ML & MLE Roles)
- ML System Design: Recsys, Chatbot, Image Classifier
- Meeting Rooms: Minimum Rooms to Hold All Meetings
- Longest-Match Greedy Tokenization with Dictionary ID Replacement