公开(公告)号：US20180218287A1

公开(公告)日：2018-08-02

申请号：US15421438

申请日：2017-02-01

Applicant: Facebook, Inc.

Inventor： Zhuang Wang ,  Robert Oliver Burns Zeldin ,  Sushma Nagesh Bannur ,  Rami Mahdi ,  Rubinder Singh Sethi ,  Shyamsundar Rajaram ,  Leon R. Cho

IPC: G06N99/00 ,  G06Q30/02 ,  G06N5/04

CPC classification number: G06N20/00 ,  G06N5/04 ,  G06Q30/0201

Abstract: An online system receives content items, for example, from content providers and sends the content items to users. The online system uses machine-learning models for predicting whether a user is likely to interact with a content item. The online system uses stored user interactions to measure the model performance to determine whether the model can be used online. The online system determines a baseline model using stored user interactions. The online system determines whether the machine-learning model performs better than the baseline model or worse for each content provider. The online system determines whether to approve the model for online use based on an aggregate normalized performance metric, for example, a metric representing the fraction of content providers for which the model performs better than the baseline. If the online system determines to reject the model, the online system retrains the model.