The invention discloses a multi-GPU scheduling apparatus, a distributed computing system and a multi-GPU scheduling method, and relates to the field of cloud computing. The multi-GPU scheduling apparatus comprises a request receiving module used for receiving GPU usage requests sent by users, a request message queue module used for adding the GPU usage requests to a request message queue, and a request allocation module used for allocating GPU resources in distributed GPU nodes to the GPU usage requests in the request message queue to enable a GPU to process the GPU usage requests. The GPU usage requests are received in a unified way and added to the queue, and then the GPU usage requests in the queue are allocated with the GPU resources in the distributed GPU nodes and processed, so that universal scheduling of cross-machine multiple GPUs is realized and GPU application developers do not need to pay attention to the problem of GPU co-scheduling.