公开(公告)号：US20240281836A1

公开(公告)日：2024-08-22

申请号：US18110620

申请日：2023-02-16

申请人： Adobe Inc.

发明人： Ritwik Sinha ,  Ziao Liu ,  Robert Sebastian Mares ,  Oana Catalina Persoiu-Focsa ,  Moumita Sinha ,  Ivan Andrus ,  David Arbour ,  Akash Maharaj ,  Prithvi Bhutani

IPC分类号： G06Q30/0203 ,  G06F17/18

CPC分类号： G06Q30/0203 ,  G06F17/18

摘要： Certain aspects and features of this disclosure relate to providing anytime-valid confidence sequences for multiple messaging treatments in an experiment. A process controls and/or corrects statistical error when multiple messaging treatments are being evaluated together. Messages can be stored, formatted, and transmitted from a communication server or other computing system. In one example, each test message from among multiple test messages is sent to an independent group of recipients over some period of time. An analytics application programmatically evaluates a metric related to message responses over time and determines a difference in the metric for each of several unique messages as compared to a baseline message. The analytics application also determines a confidence value and can display the changing confidence value in sequence over time along with the current difference, or lift, while maintaining the accuracy of the values.