公开(公告)号：US11615315B2

公开(公告)日：2023-03-28

申请号：US17654194

申请日：2022-03-09

Applicant: D5AI LLC

Inventor： James K. Baker

IPC: G06N3/08 ,  G06K9/62 ,  G06N3/04 ,  G06N3/084 ,  G06N3/082

Abstract: A machine learning system includes a coach machine learning system that uses machine learning to help a student machine learning system learn its system. By monitoring the student learning system, the coach machine learning system can learn (through machine learning techniques) “hyperparameters” for the student learning system that control the machine learning process for the student learning system. The machine learning coach could also determine structural modifications for the student learning system architecture. The learning coach can also control data flow to the student learning system.

公开(公告)号：US11562246B2

公开(公告)日：2023-01-24

申请号：US17664898

申请日：2022-05-25

Applicant: D5AI LLC

Inventor： James K. Baker

IPC: G06N3/08 ,  G06N3/04

Abstract: Methods and computer systems improve a trained base deep neural network by structurally changing the base deep neural network to create an updated deep neural network, such that the updated deep neural network has no degradation in performance relative to the base deep neural network on the training data. The updated deep neural network is subsequently training. Also, an asynchronous agent for use in a machine learning system comprises a second machine learning system ML2 that is to be trained to perform some machine learning task. The asynchronous agent further comprises a learning coach LC and an optional data selector machine learning system DS. The purpose of the data selection machine learning system DS is to make the second stage machine learning system ML2 more efficient in its learning (by selecting a set of training data that is smaller but sufficient) and/or more effective (by selecting a set of training data that is focused on an important task). The learning coach LC is a machine learning system that assists the learning of the DS and ML2. Multiple asynchronous agents could also be in communication with each others, each trained and grown asynchronously under the guidance of their respective learning coaches to perform different tasks.

公开(公告)号：US20220391707A1

公开(公告)日：2022-12-08

申请号：US17654187

申请日：2022-03-09

Applicant: D5AI LLC

Inventor： James K. Baker

IPC: G06N3/08 ,  G06K9/62 ,  G06N3/04

Abstract: A machine learning system includes a coach machine learning system that uses machine learning to help a student machine learning system learn its system. By monitoring the student learning system, the coach machine learning system can learn (through machine learning techniques) “hyperparameters” for the student learning system that control the machine learning process for the student learning system. The machine learning coach could also determine structural modifications for the student learning system architecture. The learning coach can also control data flow to the student learning system.

公开(公告)号：US20220335305A1

公开(公告)日：2022-10-20

申请号：US17810778

申请日：2022-07-05

Applicant: D5AI LLC

Inventor： James K. BAKER

IPC: G06N3/08 ,  G06N3/04 ,  G06N20/00 ,  G06N7/00 ,  G06K9/62 ,  G06N3/063

Abstract: Computer systems and methods cooperatively train multiple generators and a classifier. Cooperative training includes: training, through machine learning, the multiple generators such that each generator is trained according to a first objective to output examples of a designated classification category; training, through machine learning, the classifier to determine, for each generated by the multiple generators, which of the multiple generators generated the example; and back-propagating partial derivatives of an error cost function from the classifier to the multiple generators.

公开(公告)号：US11410050B2

公开(公告)日：2022-08-09

申请号：US16901608

申请日：2020-06-15

Applicant: D5AI LLC

Inventor： James K. Baker

IPC: G06N3/04 ,  G06N3/08 ,  G06N20/00 ,  G06N7/00 ,  G06K9/62 ,  G06N3/063 ,  G06F17/18 ,  G06F12/0815

Abstract: Various systems and methods are described herein for improving the aggressive development of machine learning systems. In machine learning, there is always a trade-off between allowing a machine learning system to learn as much as it can from training data and overfitting on the training data. This trade-off is important because overfitting usually causes performance on new data to be worse. However, various systems and methods can be utilized to separate the process of detailed learning and knowledge acquisition and the process of imposing restrictions and smoothing estimates, thereby allowing machine learning systems to aggressively learn from training data, while mitigating the effects of overfitting on the training data.

公开(公告)号：US11222288B2

公开(公告)日：2022-01-11

申请号：US17268660

申请日：2019-08-12

Applicant: D5AI LLC

Inventor： James K. Baker

IPC: G06N20/20 ,  G06N3/04 ,  G06N3/063 ,  G06N3/08

Abstract: A computer-implemented method of training an ensemble machine learning system comprising a plurality of ensemble members. The method includes selecting a shared objective and an objective for each of the ensemble members. The method further includes training each of the ensemble members according to each objective on a training data set, connecting an output of each of the ensemble members to a joint optimization machine learning system to form a consolidated machine learning system, and training the consolidated machine learning system according to the shared objective and the objective for each of the ensemble members on the training data set. The ensemble members can be the same or different types of machine learning systems. Further, the joint optimization machine learning system can be the same or a different type of machine learning system than the ensemble members.

公开(公告)号：US11195097B2

公开(公告)日：2021-12-07

申请号：US16609130

申请日：2019-07-02

Applicant: D5AI LLC

Inventor： James K. Baker ,  Bradley J. Baker

IPC: G06N3/08 ,  G06N3/04

Abstract: Computer-implemented systems and methods build ensembles for deep learning through parallel data splitting by creating and training an ensemble of up to 2n ensemble members based on a single base network and a selection of n network elements. The ensemble members are created by the “blasting” process, in which training data are selected for each of the up to 2n ensemble members such that each of the ensemble members trains with updates in a different direction from each of the other ensemble members. The ensemble members may also be trained with joint optimization.

公开(公告)号：US20210342683A1

公开(公告)日：2021-11-04

申请号：US16620177

申请日：2019-08-08

Applicant: D5AI LLC

Inventor： James K. BAKER

IPC: G06N3/08 ,  G06N3/04

Abstract: Systems and methods for analyzing a first machine learning system via a second machine learning system. The first machine learning system comprising a first objective function. The method includes connecting the first machine learning system to an input of the second machine learning system, which includes a second objective function for analyzing an internal characteristic of the first machine learning system. The method further includes providing a data item to the first machine learning system, collecting internal characteristic data from the first machine learning system associated with the internal characteristic, computing partial derivatives of the first objective function through the first machine learning system with respect to the data item, and computing partial derivatives of the second objective function through both the second machine learning system and the first machine learning system with respect to the collected internal characteristic data.

公开(公告)号：US11087217B2

公开(公告)日：2021-08-10

申请号：US16923234

申请日：2020-07-08

Applicant: D5AI LLC

Inventor： James K. Baker ,  Bradley J. Baker

IPC: G06N3/08 ,  G06N20/20 ,  G06N3/04 ,  H04L29/08 ,  G06K9/62 ,  G06N20/00 ,  G06F16/901 ,  G06N5/04

Abstract: A system and method for controlling a nodal network. The method includes estimating an effect on the objective caused by the existence or non-existence of a direct connection between a pair of nodes and changing a structure of the nodal network based at least in part on the estimate of the effect. A nodal network includes a strict partially ordered set, a weighted directed acyclic graph, an artificial neural network, and/or a layered feed-forward neural network.

公开(公告)号：US11003982B2

公开(公告)日：2021-05-11

申请号：US16620214

申请日：2018-06-15

Applicant: D5AI LLC

Inventor： James K. Baker

IPC: G06N3/063 ,  G06N3/04 ,  G06N3/08 ,  G06T1/20

Abstract: Computer-based systems and methods guide the learning of features in middle layers of a deep neural network. The guidance can be provided by aligning sets of nodes or entire layers in a network being trained with sets of nodes in a reference system. This guidance facilitates the trained network to more efficiently learn features learned by the reference system using fewer parameters and with faster training. The guidance also enables training of a new system with a deeper network, i.e., more layers, which tend to perform better than shallow networks. Also, with fewer parameters, the new network has fewer tendencies to overfit the training data.