公开(公告)号：US10713594B2

公开(公告)日：2020-07-14

申请号：US15407147

申请日：2017-01-16

Applicant: salesforce.com, inc.

Inventor： Kit Pang Szeto ,  Simon Chan

IPC: G06N20/00 ,  G06F11/14 ,  G06Q30/02

Abstract: In accordance with disclosed embodiments, there are provided systems, methods, and apparatuses for implementing machine learning model training and deployment with a rollback mechanism within a computing environment. For example, an exemplary machine learning platform includes means for receiving training data as input at the machine learning platform, in which the training data includes a multiple transactions, each of the transactions specifying a plurality of features upon which to make a prediction and a label representing a correct answer for the plurality of features according to each respective transaction; specifying a model to be trained by the machine learning platform using the training data, in which the model includes a plurality of algorithms and source code; generating a new predictive engine variant by training the model to algorithmically arrive upon the label representing the correct answer as provided with the training data based on the plurality of features for each of the multiple transactions; versioning the new predictive engine variant based at least on the time the new predictive engine variant was generated a version of the source code utilized within the model and the training data received as input; deploying the new predictive engine variant into a production environment to replace a prior version of the predictive engine variant; and rolling back the new predictive engine variant from the production environment to a specified version which is less than a version of the new predictive engine variant. Other related embodiments are disclosed.

公开(公告)号：US10469584B2

公开(公告)日：2019-11-05

申请号：US15721575

申请日：2017-09-29

Applicant: salesforce.com, inc.

Inventor： Ka Hou Chan ,  Karl Ryszard Skucha ,  Kit Pang Szeto ,  Emmanual Felipe Oliveira ,  Jean-Marc Soumet ,  Simon Chan ,  Matvey Tovbin

IPC: G06F15/16 ,  H04L29/08 ,  G06F3/06 ,  G06F16/27 ,  G06F16/182 ,  G06F16/25 ,  G06F16/2453

Abstract: Techniques and architectures for data modeling and management. Data modeling services are provided to agents within multiple different operating environments of a computing environment having at least one database stored on one or more physical memory devices communicatively coupled with one or more hardware processors the one or physical memory devices. Building and versioning of data modeling projects is coordinated and data utilized for the data modeling projects with the one or more hardware processors.

公开(公告)号：US20190146830A1

公开(公告)日：2019-05-16

申请号：US15809752

申请日：2017-11-10

Applicant: salesforce.com, Inc.

Inventor： Thomas Gerber ,  Kit Pang Szeto ,  Vitaly Gordon ,  Ji Oh Yoo ,  Shaun Senecal ,  Gregory Rice ,  Ka Hou Chan

IPC: G06F9/46 ,  G06N99/00

Abstract: Methods, systems, and devices for multi-tenant workflow processing are described. In some cases, a cloud platform may utilize a set of pre-defined batch processes (e.g., workflow templates) and tenant-specific configurations for instantiating and executing tenant-specific batch processes for each tenant of a user. As such, the cloud platform may utilize common data process workflows for each tenant, where a configuration specifies tenant-specific information for the common data process workflows. The workflow templates may include a set of job definitions (e.g., actions for a server to execute) and a schedule defining the frequency for running the templates for a specific project. The configurations may indicate a tenant to execute the workflow templates for, and may include tenant-specific information to override default template information. The cloud platform or a designated server or server cluster may instantiate and execute workflows based on one or more combinations of configurations and indicated workflow templates.

公开(公告)号：US10192172B2

公开(公告)日：2019-01-29

申请号：US15404052

申请日：2017-01-11

Applicant: salesforce.com, inc.

Inventor： Ka Hou Chan ,  Simon Chan ,  Kit Pang Szeto ,  Yue Kwen Justin Yip

IPC: G06N99/00 ,  G06Q30/02 ,  G06F17/30 ,  G06N7/00 ,  G06N5/02

Abstract: In accordance with disclosed embodiments, there are provided systems, methods, and apparatuses for implementing predictive engine evaluation and replay of engine performance. An exemplary system may include, for example: means selecting a first set of one or more algorithms for a machine learning model; tuning a first group of predictive engine parameters for the machine learning model; training the machine learning model with one or more sources of data using the selected first set of one or more algorithms and the first group of tuned predictive engine parameters to generate a first predictive engine variant from the trained machine learning model; selecting a second set of one or more algorithms for a machine learning model which are different than the first set; tuning a second group of predictive engine parameters for the machine learning model which are different than the first group; training the machine learning model with the one or more sources of data using the selected second set of one or more algorithms and the second group of tuned predictive engine parameters to generate a second predictive engine variant from the trained machine learning model; performing multiple experiments using the first and second predictive engine variants; comparing results from the multiple experiments; and deploying either the first predictive engine variant or the second predictive engine variant based on the comparison of the results of the multiple experiments. Other related embodiments are disclosed.

公开(公告)号：US20180097880A1

公开(公告)日：2018-04-05

申请号：US15721575

申请日：2017-09-29

Applicant: salesforce.com, inc.

Inventor： Ka Hou Chan ,  Karl Ryszard Skucha ,  Kit Pang Szeto ,  Emmanual Felipe Oliveira ,  Jean-Marc Soumet ,  Simon Chan ,  Matvey Tovbin

IPC: H04L29/08 ,  G06F17/30 ,  G06F3/06

CPC classification number: H04L67/1097 ,  G06F3/067 ,  G06F16/183 ,  G06F16/25 ,  G06F16/27 ,  H04L67/10

Abstract: Techniques and architectures for data modeling and management. Data modeling services are provided to agents within multiple different operating environments of a computing environment having at least one database stored on one or more physical memory devices communicatively coupled with one or more hardware processors the one or physical memory devices. Building and versioning of data modeling projects is coordinated and data utilized for the data modeling projects with the one or more hardware processors.

公开(公告)号：US20180096028A1

公开(公告)日：2018-04-05

申请号：US15713069

申请日：2017-09-22

Applicant: salesforce.com, inc.

Inventor： Chalenge Masekera ,  Simon Chan ,  Kit Pang Szeto

IPC: G06F17/30 ,  G06F21/62

CPC classification number: G06F16/2453 ,  G06F16/2462 ,  G06F21/6227

Abstract: In accordance with embodiments, there are provided mechanisms and methods for facilitating a framework for management of machine learning models for tenants in an on-demand services environment according to one embodiment. In one embodiment and by way of example, a method comprises determining, by a model management server computing device (“management device”), business criteria for a tenant in a multi-tenant environment, where the business criteria are based on business preferences of the tenant. The method may further include building, by the management device, multiple models dedicated to the tenant based on the business criteria such that each model is trained and fitted to perform one or more combinations of processes based on one or more integrations of the business criteria. The method may further include dynamically selecting, by the management device, a model from the multiple models to perform a combination of processes involving an integration of two or more criterion of the business criteria as requested by the tenant.