A computer system generates augmented training datasets to train neural network models. The computer system receives an initial training dataset comprising images for training a neural network model, and generates an augmented training dataset by modifying images from the first training dataset. The computer system identifies a representation of a target object against a background from the initial training dataset and extracts a portion of the image displaying the target object. The computer system generates samples for including in the augmented training dataset based on the image. For example, new images may be obtained by performing transformations on the portion of the image displaying the target object and/or by overlaying the transformed portion of the image over a different background. The modified images are included in the augmented training dataset used for training the neural network model to recognize the target object.