A method for controlling the speed of a work vehicle towing an implement that is movable between a working position, in which ground engaging tools of the implement are configured to perform a field operation, and a transport position, in which the ground engaging tools are raised relative to the ground. The method may include monitoring, with a computing device, an implement weight supported by the implement while the implement is in the transport position. The method may further include comparing, with the computing device, the implement weight to a predetermined threshold weight. Additionally, the method may include, when the implement weight differs from the predetermined threshold weight, adjusting, with the computing device, a maximum speed limit for the work vehicle.