A GPU kernel is the code that runs on the GPU when you call an operation like torch.matmul, as thousands of copies at once.