CUMULATIVE INCREMENTALITY SCORES FOR EVALUATING THE PERFORMANCE OF MACHINE LEARNING MODELS

    公开(公告)号:US20230385886A1

    公开(公告)日:2023-11-30

    申请号:US17752800

    申请日:2022-05-24

    CPC classification number: G06Q30/0601 G06N5/022

    Abstract: An online concierge system uses a cumulative incrementality score to evaluate the performance of incrementality models used by the online concierge system to identify users for treatment. The online concierge system applies an incrementality model to a set of examples to generate predicted incrementality scores for the examples. The online concierge system ranks the examples based on the predicted incrementality scores for the examples and groups the examples based on their rankings. The online concierge system iteratively computes cumulative incrementality scores for each grouping based on the examples of each grouping, and computes a final cumulative incrementality score for the incrementality model based on each of the cumulative incrementality scores.

    AUTOMATED POLICY FUNCTION ADJUSTMENT USING REINFORCEMENT LEARNING ALGORITHM

    公开(公告)号:US20230298080A1

    公开(公告)日:2023-09-21

    申请号:US18108916

    申请日:2023-02-13

    CPC classification number: G06Q30/0617 G06N3/092

    Abstract: An online system may receive, from a content provider, a content presentation campaign that includes one or more objectives. The online system may define a set of one or more policy functions that automatically controls the content presentation campaign. A policy function may control one or more criteria in bidding content slots. The online system may monitor a realized outcome of the content presentation campaign. The online system may apply a reinforcement learning algorithm in adjusting the set of policy functions. The reinforcement learning algorithm adjusts one or more parameters in the set of policy functions to reduce a difference between the realized outcome and the desired outcome set by the content provider. The online system generates an adjusted set of policy functions and uses the adjusted set of policy functions in bidding content slots to present one or more content items provided by the content provider.

    SELECTING METHODS TO ALLOCATE SHOPPERS FOR ORDER FULFILLMENT IN GEOGRAPHIC REGIONS BY AN ONLINE CONCIERGE SYSTEM BASED ON MACHINE-LEARNED EFFICIENCIES FOR DIFFERENT ALLOCATION METHODS

    公开(公告)号:US20230196442A1

    公开(公告)日:2023-06-22

    申请号:US17556936

    申请日:2021-12-20

    CPC classification number: G06Q30/0635

    Abstract: An online concierge system allocates shoppers to different geographic regions at different times to fulfill orders received from users. The online concierge system uses different methods for adjusting allocation of shoppers to geographic regions, such as obtaining new shoppers or providing incentives to additional shoppers, based on estimated numbers of orders identifying different geographic regions. To account for costs to the online concierge system for allocating shoppers to geographic regions, the online concierge system trains multiple machine learned models to predict different efficiency metrics for methods for adjusting shopper allocation. Discrete samples are obtained from each efficiency metric, and samples that do not satisfy one or more constraints removed. From the remaining samples, a combination of samples for different methods for adjusting shopper allocation is selected to optimize a value to the online concierge system based on one or more criteria.

Patent Agency Ranking