A method for filtering a notification being provided from an application to a user of a social networking system based on an action of the user is described. The method starts by receiving the notification from the application to the user. If the notification is a first notification sent from the application to the user, the notification is displayed to the user with (i) a prompting message that includes a selectable opt in image, and (ii) a selectable opt out image. The prompting message includes a request to opt in to receiving notifications from the application. If the notification is not the first notification sent from the application to the user, the method determines whether to display the notification to the user by determining whether the user previously opted in to notifications from the application and whether click-through rate (CTR) of notifications from the application is greater than CTR threshold.