An online system receives content items from a third party content provider. For each content item, the online system inputs an image into a neural network and extracts a feature vector from a hidden layer of the neural network. The online system compresses each feature vector by assigning a label to each feature value representing whether the feature value was above a threshold value. The online system identifies a set of content items that the user has interacted with and determines a user feature vector by aggregating feature vectors of the set of content items. For a new set of content items, the online system compares the compressed feature vectors of the content item with the user feature vector. The online system selects one or more of the new content items based on the comparison and sends the selected content items to the user.