A/B testing using sequential hypothesis

    公开(公告)号:US11593667B2

    公开(公告)日:2023-02-28

    申请号:US17077911

    申请日:2020-10-22

    申请人: Optimizely, Inc.

    发明人: Michael Lindon

    摘要: A method of executing an A/B test includes, during execution of the A/B test, determining, by a processing device using a sequential frequentist test, that a sample ratio mismatch has occurred, wherein the sample ratio mismatch is determined before the A/B test ends the execution. The method further includes, in response to the determining, ending the execution of the A/B test before a previously scheduled end of the A/B test.

    Systems and methods for website optimization

    公开(公告)号:US10902196B1

    公开(公告)日:2021-01-26

    申请号:US14460142

    申请日:2014-08-14

    申请人: Optimizely, Inc.

    摘要: Webpages are optimizing through the use of experiments that compare the responses of viewers that are either presented with the original webpage or a variant thereof. One or more variants are first defined through the use of a browser-based editor application that initially examines the webpage for a snippet of code. The snippet can be added to the webpage, if missing, and the webpage returned to the editor application. Changes made to the webpage to define a variant are saved in variation code. When the webpage is later requested by multiple viewers, in each instance the viewer receives the webpage with the snippet, the snippet instructs the browser to download a file, and the instructions of the file determine whether the viewer will see the variant or the original webpage. Tracking viewer responses to the webpage and the variant allow a statistical basis for comparison to be developed.

    Customizable decision service
    3.
    发明授权

    公开(公告)号:US11663117B2

    公开(公告)日:2023-05-30

    申请号:US17140907

    申请日:2021-01-04

    申请人: Optimizely, Inc.

    摘要: A method of executing an A/B testing service comprises deploying an instance of the A/B testing service between a client device and an A/B testing server and receiving by the A/B testing service from the A/B testing server a software development kit (SDK) key for a client device, the SDK key indicating an A/B testing environment to be associated with the instance. The method further comprises, in response to receiving the SDK key: storing data associated with the A/B testing environment in memory of the client device; providing a HyperText Transfer Protocol (HTTP) application programming interface (API) associated with the instance of the A/B testing service to one or more applications associated with the A/B testing environment; and executing, by a processing device, commands between the client device and the one or more applications on behalf of the A/B testing environment.

    Dynamic content recommendation for responsive websites

    公开(公告)号:US11449671B2

    公开(公告)日:2022-09-20

    申请号:US16777120

    申请日:2020-01-30

    申请人: Optimizely, Inc.

    发明人: Dan Siroker

    摘要: A method includes detecting a first plurality of user interactions with first dynamic content of a website. The method further includes detecting a second plurality of user interactions with second dynamic content of a website. The method further includes providing a graphical user interface (GUI) that illustrates the first and second pluralities of user interactions to a client device. The method further includes determining a correlation between the first and second plurality of user interactions and a conversion metric. The method further includes determining, by a processing device, that the first dynamic content has a higher conversion metric than the second dynamic content. The method further includes sending a first variation recommendation for the first dynamic content to the client device.

    CUSTOMIZABLE DECISION SERVICE
    5.
    发明申请

    公开(公告)号:US20220214964A1

    公开(公告)日:2022-07-07

    申请号:US17140907

    申请日:2021-01-04

    申请人: Optimizely, Inc.

    IPC分类号: G06F11/36 G06F9/54 H04L29/08

    摘要: A method of executing an A/B testing service comprises deploying an instance of the A/B testing service between a client device and an A/B testing server and receiving by the A/B testing service from the A/B testing server a software development kit (SDK) key for a client device, the SDK key indicating an A/B testing environment to be associated with the instance. The method further comprises, in response to receiving the SDK key: storing data associated with the A/B testing environment in memory of the client device; providing a HyperText Transfer Protocol (HTTP) application programming interface (API) associated with the instance of the A/B testing service to one or more applications associated with the A/B testing environment; and executing, by a processing device, commands between the client device and the one or more applications on behalf of the A/B testing environment.

    Software development kit driven features and events

    公开(公告)号:US11321418B2

    公开(公告)日:2022-05-03

    申请号:US16660537

    申请日:2019-10-22

    申请人: Optimizely, Inc.

    摘要: A method includes providing, to a third-party content provider, code identifying a feature associated with digital content of the third-party content provider, wherein the identification comprises a feature variable placeholder associated with the feature. The method further includes receiving, by an experimentation platform, an indication of a first event type associated with the digital content, wherein the first event type identifies a first trackable action associated with the feature. The method further includes determining, by a processing device of the experimentation platform, that the first event type does not exist in a configuration associated with the feature; The method further includes, in response to the determining, generating the first event type in the configuration, generating and providing a graphical representation of the first event type for display in a user interface of the experimentation platform, and tracking the first event type on the experimentation platform.

    Systems and methods for website optimization

    公开(公告)号:US10747942B1

    公开(公告)日:2020-08-18

    申请号:US14460116

    申请日:2014-08-14

    申请人: Optimizely, Inc.

    摘要: Webpages are optimizing through the use of experiments that compare the responses of viewers that are either presented with the original webpage or a variant thereof. One or more variants are first defined through the use of a browser-based editor application that initially examines the webpage for a snippet of code. The snippet can be added to the webpage, if missing, and the webpage returned to the editor application. Changes made to the webpage to define a variant are saved in variation code. When the webpage is later requested by multiple viewers, in each instance the viewer receives the webpage with the snippet, the snippet instructs the browser to download a file, and the instructions of the file determine whether the viewer will see the variant or the original webpage. Tracking viewer responses to the webpage and the variant allow a statistical basis for comparison to be developed.

    Systems and methods for website optimization
    8.
    发明授权
    Systems and methods for website optimization 有权
    网站优化的系统和方法

    公开(公告)号:US08839093B1

    公开(公告)日:2014-09-16

    申请号:US13345211

    申请日:2012-01-06

    IPC分类号: G06F17/20

    CPC分类号: G06F17/3089

    摘要: Webpages are optimizing through the use of experiments that compare the responses of viewers that are either presented with the original webpage or a variant thereof. One or more variants are first defined through the use of a browser-based editor application that initially examines the webpage for a snippet of code. The snippet can be added to the webpage, if missing, and the webpage returned to the editor application. Changes made to the webpage to define a variant are saved in variation code. When the webpage is later requested by multiple viewers, in each instance the viewer receives the webpage with the snippet, the snippet instructs the browser to download a file, and the instructions of the file determine whether the viewer will see the variant or the original webpage. Tracking viewer responses to the webpage and the variant allow a statistical basis for comparison to be developed.

    摘要翻译: 网页正在通过使用实验进行优化,实验比较了与原始网页或其变体呈现的观看者的响应。 首先通过使用基于浏览器的编辑器应用程序来定义一个或多个变体,该应用程序最初检查网页以获取代码段。 该片段可以添加到网页,如果缺少,并且网页返回到编辑器应用程序。 对网页进行的定义变体的更改将保存在变体代码中。 当网页随后被多个观众请求时,在每个实例中,观众利用该片段接收网页,片段指示浏览器下载文件,并且该文件的指令确定观众是否看到该变体或原始网页 。 跟踪查看器对网页和变体的响应允许开发用于比较的统计基础。

    Experiment system integration service

    公开(公告)号:US11593452B1

    公开(公告)日:2023-02-28

    申请号:US16568554

    申请日:2019-09-12

    申请人: Optimizely, Inc.

    摘要: A method comprises receiving a first outbound request, from an internal user account of an internal platform, indicating a first action to be performed by a first third-party user account of a first third-party platform. In response to authenticating the first outbound request, the method further comprises sending an application programming interface (API) request to the first third-party platform to perform the first action on the first third-party platform on behalf of the internal user account. The method further comprises receiving a first inbound request, from the first third-party user account, indicating a second action to be performed on behalf of the internal user account on the internal platform. In response to authenticating the first inbound request, the method further comprises sending an internal request to the internal platform to perform the second action on the internal platform on behalf of the first third-party user account.

    Server-side configuration variables in feature testing

    公开(公告)号:US11539806B2

    公开(公告)日:2022-12-27

    申请号:US17403501

    申请日:2021-08-16

    申请人: Optimizely, Inc.

    摘要: A method includes receiving an identification of a feature associated with digital content of a third-party content provider, wherein the identification comprises a feature variable placeholder associated with the feature. The method further includes receiving a configuration of a feature flag associated with the feature. The method further includes determining, by a processing device of an experimentation system, a plurality of feature variable values corresponding to the feature variable placeholder. The method further includes configuring, by the processing device, one or more rules on the experimentation system to determine: when and to whom the feature is to be deployed, based on the feature flag; and which of the plurality of feature variable values is to be deployed when and to whom.