An online system receives one or more aggregation rules associated with a third party system that determine online system users' eligibility for receiving content items provided by the third party system using the online system users' behavior. When an online system user performs actions related to content provided by the third party system via a client device, events describing the actions are generated and communicated from the client device to the online system. The online system evaluates a user's eligibility when a user performs an action by applying an aggregation rule to an event corresponding to the user's action and other events associated with the user corresponding to the user's previous actions. The online system may re-evaluate a user's eligibility at a later time if the user is determined to be ineligible. The online system provides content items to eligible users for presentation.