Methods and systems for post-processing video to identify interests based on clustered user interactions are provided. A client application program running on a set-top box is provided to enable a user to interactively select a point of interest within video content being played via the set-top box. The selected points of interest of multiple users is aggregated, and clusters of the points of interest are determined. Objects associated with the clusters are determined by analyzing video segments containing the clustered points of interest. Messages related to the objects are created and sent to the relevant users that indicated selection of points of interest which correlate to the objects.