Systems and techniques are described for implementing autonomous control of earth-moving vehicles, including to automatically determine and control movement around a site having potential obstacles. For example, the systems/techniques may determine and implement autonomous operations of excavator vehicle(s) (e.g., obtain/integrate data from sensors of multiple types, and use it to determine and control movement of the excavator vehicle around a site), including in some situations to implement coordinated actions of multiple excavator vehicles and/or of an excavator vehicle with one or more other types of earth-moving vehicles. The described techniques may further include determining current location and positioning of the excavator vehicle on the site, determining a target destination location and/or route of the excavator vehicle, identifying and classifying obstacles (if any) along a desired route or otherwise between current and destination locations, and implementing actions to address any such obstacles.