Abstract:
A group of social networking system users are associated with a holdout group for an advertisement. Users in the holdout group are not presented with the advertisement. When the advertisement is selected for presentation to a user, the social networking system presents the advertisement to the user if the user is not in the holdout group. However, if the user is in the holdout group, alternative content is presented to the user. If a user performs a conversion event associated with the advertisement via a client device, the social networking system determines a fee for an advertiser if the advertisement was presented to the user. The fee may be adjusted based on differences between conversion events by users in the holdout group for the advertisement and by users not in the holdout group.