In one aspect, an example method includes (i) obtaining video that depicts an object across multiple frames of the video; (ii) detecting the object within the obtained video and determining object characteristic data associated with the detected object; (iii) determining user profile data associated with a viewer of the video; (iv) using at least the determined object characteristic data and the determined user profile data as a basis to select a replacement object from among a set of multiple candidate replacement objects; (v) replacing the detected object with the selected replacement object to generate video that is a modified version of the obtained video; and (vi) outputting for presentation the generated video.