Design a CI Job Scheduling System with CPU and GPU Workers

Asked at LinkedIn · System Design for ML · Difficulty: Hard · Last asked Nov 10, 2026

Design a continuous-integration scheduling system. Source-control events (webhooks for pushes, pull requests and merges) trigger pipelines made of jobs; each job must run on a suitable worker, either a CPU worker or a GPU worker. At peak load there are more jobs than workers, and the system must stay fair, predictable and cost-aware. Assume an agent model like the ones used by hosted build services: long-lived agents on your own machines pull work from named queues.

Loading the problem…