A remote control device is a remote control device configured to control one or more mobile objects via a network, which includes a receiver configured to receive mobile object information including a first state quantity of a state quantity of the mobile object and surrounding information around the mobile object, a trajectory generation unit configured to generate a target trajectory of the mobile object on the basis of the surrounding information, a mobile object estimation unit configured to estimate transmission latency of the network, a gain setting unit configured to set a control gain on the basis of the transmission latency, a control amount calculation unit configured to calculate a control amount for causing the mobile object to follow the target trajectory on the basis of the mobile object information and the control gain, and a transmitter configured to transmit the control amount to the mobile object.