公开(公告)号：US10951503B1

公开(公告)日：2021-03-16

申请号：US15494328

申请日：2017-04-21

Applicant: Amazon Technologies, Inc.

Inventor： Rui Chen ,  Siwei Jia ,  Tae Yeon Kim ,  Mengyun Lv ,  Serguei B. Stepaniants

IPC: G06F9/44 ,  G06Q30/02 ,  H04L29/08 ,  H04L12/26 ,  H04L12/24 ,  G06F11/36 ,  G06F30/20 ,  G05B23/02

Abstract: Technologies are disclosed for determining validity of data obtained from an A/B experiment, where the experiment evaluates the desirability of a potential change at a website. The experiment is run for a period of time and based upon an expected allocation of users into the A group (control group) and the B group (e.g., treatment group), along with an actual number of users directed into the two groups, it is determined if a minimal detectable error of allocation has been exceeded and, if it has, the data is deemed to be invalid. If not, the data is deemed to be valid.

公开(公告)号：US10185970B1

公开(公告)日：2019-01-22

申请号：US15494318

申请日：2017-04-21

Applicant: Amazon Technologies, Inc.

Inventor： Rui Chen ,  Jeff Brooks ,  Siwei Jia ,  Mia Kilborn ,  Zhen Liu ,  Serguei B. Stepaniants

IPC: G06F15/16 ,  G06Q30/02

Abstract: Technologies are disclosed for determining a runtime length for an A/B experiment, where the experiment evaluates the desirability of a potential change at a website. The experiment is run for an initial period of time and based upon initial data from the initial period of time, an equation is iteratively solved until a minimal amount of time is determined that that indicates when a statistically significant change will be observed in the data. The experiment is then run for the minimal amount of time.