Similar faces may be determined within images based on human perception of facial similarity. The user may provide an image including a query face to which the user wishes to find faces that are similar. Similar faces may be determined based on similarity information. Similarity information may be generated from information related to a human perception of facial similarity. Images that include faces determined to be similar, based on the similarity information, may be provided to the user as search result images. The user then may provide feedback to indicate the user's perception of similarity between the query face and the search result images.