公开(公告)号：US20230142528A1

公开(公告)日：2023-05-11

申请号：US18147313

申请日：2022-12-28

申请人： D5AI LLC

发明人： James K. Baker

IPC分类号： G06N3/082 ,  G06N3/044 ,  G06N3/084

CPC分类号： G06N3/082 ,  G06N3/044 ,  G06N3/084

摘要： Computer systems and methods modify a base deep neural network (DNN). The method comprises replacing the target node of the base DNN with a compound node to thereby create a modified base DNN. The compound node comprises at least first and second nodes. The first node is trained to detect target node patterns in inputs to the first node and the second node is trained to detect an absence of the target node patterns in inputs to the second node, and the first and second nodes are trained to be non-complementary. Replacing the target node with the compound node comprises: connecting the first node to the upper sub-network of the base DNN, such that the first node has a weighted connection for each of the one or more base connection that the target node had to the upper sub-network in the base DNN; and connecting the second node to the upper sub-network of the base deep DNN, such that the second node has a weighted connection for each of the one or more base connection that the target node had to the upper sub-network in the base deep DNN. After replacing the target node, the modified base DNN is trained.

公开(公告)号：US11610130B2

公开(公告)日：2023-03-21

申请号：US17654187

申请日：2022-03-09

申请人： D5AI LLC

发明人： James K. Baker

IPC分类号： G06N3/08 ,  G06K9/62 ,  G06N3/04 ,  G06N3/084 ,  G06N3/082

摘要： A machine learning system includes a coach machine learning system that uses machine learning to help a student machine learning system learn its system. By monitoring the student learning system, the coach machine learning system can learn (through machine learning techniques) “hyperparameters” for the student learning system that control the machine learning process for the student learning system. The machine learning coach could also determine structural modifications for the student learning system architecture. The learning coach can also control data flow to the student learning system.

公开(公告)号：US20220383111A1

公开(公告)日：2022-12-01

申请号：US17753727

申请日：2020-09-09

申请人： D5AI LLC

发明人： James K. Baker

IPC分类号： G06N3/08

摘要： Machine-learning computer system breaks a neural network into a plurality of modules and tracks the training process module-by-module and datum-by-datum, recording auxiliary information during one iteration of the training process for retrieval during a later iteration. Based on this auxiliary information, the computer system can make decisions that can greatly reduce the amount of computation required by the training process. The auxiliary information allows the computer system to diagnose and fix problems that occur during the training process on a module-by-module and/or datum-by-datum basis.

公开(公告)号：US20220335296A1

公开(公告)日：2022-10-20

申请号：US17753061

申请日：2020-07-28

申请人： D5AI LLC

发明人： James K. Baker

IPC分类号： G06N3/08 ,  G06N3/04

摘要： Computer systems and computer-implemented methods modify a machine learning network, such as a deep neural network, to introduce judgment to the network. A “combining” node is added to the network, to thereby generate a modified network, where activation of the combining node is based, at least in part, on output from a subject node of the network. The computer system then trains the modified network by, for each training data item in a set of training data, performing forward and back propagation computations through the modified network, where the backward propagation computation through the modified network comprises computing estimated partial derivatives of an error function of an objective for the network, except that the combining node selectively blocks back-propagation of estimated partial derivatives to the subject node, even though activation of the combining node is based on the activation of the subject node.

公开(公告)号：US11461655B2

公开(公告)日：2022-10-04

申请号：US16767966

申请日：2019-01-28

申请人： D5AI LLC

发明人： James K. Baker ,  Bradley J. Baker

IPC分类号： G06N3/08 ,  G06N20/20 ,  G06K9/62 ,  G06N5/04 ,  G06N3/04 ,  H04L67/142 ,  G06N20/00 ,  G06F16/901

摘要： A system and method for controlling a nodal network. The method includes estimating an effect on the objective caused by the existence or non-existence of a direct connection between a pair of nodes and changing a structure of the nodal network based at least in part on the estimate of the effect. A nodal network includes a strict partially ordered set, a weighted directed acyclic graph, an artificial neural network, and/or a layered feed-forward neural network.

公开(公告)号：US11074502B2

公开(公告)日：2021-07-27

申请号：US16620052

申请日：2019-08-12

申请人： D5AI LLC

发明人： James K. Baker

IPC分类号： G06N3/08 ,  G06N3/04 ,  G06N3/063

摘要： A computer system uses a pool of predefined functions and pre-trained networks to accelerate the process of building a large neural network or building a combination of (i) an ensemble of other machine learning systems with (ii) a deep neural network. Copies of a predefined function node or network may be placed in multiple locations in a network being built. In building a neural network using a pool of predefined networks, the computer system only needs to decide the relative location of each copy of a predefined network or function. The location may be determined by (i) the connections to a predefined network from source nodes and (ii) the connections from a predefined network to nodes in an upper network. The computer system may perform an iterative process of selecting trial locations for connecting arcs and evaluating the connections to choose the best ones.

公开(公告)号：US11010670B2

公开(公告)日：2021-05-18

申请号：US16620058

申请日：2019-08-23

申请人： D5AI LLC

发明人： James K. Baker

IPC分类号： G06N3/08 ,  G06N5/04 ,  G06N20/20 ,  G06N3/04

摘要： A deep neural network architecture comprises a stack of strata in which each stratum has its individual input and an individual objective, in addition to being activated from the system input through lower strata in the stack and receiving back propagation training from the system objective back propagated through higher strata in the stack of strata. The individual objective for a stratum may comprise an individualized target objective designed to achieve diversity among the strata. Each stratum may have a stratum support subnetwork with various specialized subnetworks. These specialized subnetworks may comprise a linear subnetwork to facilitate communication across strata and various specialized subnetworks that help encode features in a more compact way, not only to facilitate communication across strata but also to increase interpretability for human users and to facilitate communication with other machine learning systems.

公开(公告)号：US10956818B2

公开(公告)日：2021-03-23

申请号：US16618931

申请日：2018-06-01

申请人： D5AI LLC

发明人： James K. Baker

IPC分类号： G06N3/08 ,  G06N3/04

摘要： Systems and methods improve the performance of a network that has converged such that the gradient of the network and all the partial derivatives are zero (or close to zero) by splitting the training data such that, on each subset of the split training data, some nodes or arcs (i.e., connections between a node and previous or subsequent layers of the network) have individual partial derivative values that are different from zero on the split subsets of the data, although their partial derivatives averaged over the whole set of training data is close to zero. The present system and method can create a new network by splitting the candidate nodes or arcs that diverge from zero and then trains the resulting network with each selected node trained on the corresponding cluster of the data. Because the direction of the gradient is different for each of the nodes or arcs that are split, the nodes and their arcs in the new network will train to be different. Therefore, the new network is not at a stationary point.

公开(公告)号：US20200334541A1

公开(公告)日：2020-10-22

申请号：US16922057

申请日：2020-07-07

申请人： D5AI LLC

发明人： James K. Baker ,  Bradley J. Baker

IPC分类号： G06N3/08 ,  G06N3/04 ,  G06N20/20

摘要： A system and method for controlling a nodal network. The method includes estimating an effect on the objective caused by the existence or non-existence of a direct connection between a pair of nodes and changing a structure of the nodal network based at least in part on the estimate of the effect. A nodal network includes a strict partially ordered set, a weighted directed acyclic graph, an artificial neural network, and/or a layered feed-forward neural network.

公开(公告)号：US20240273387A1

公开(公告)日：2024-08-15

申请号：US18440119

申请日：2024-02-13

申请人： D5AI LLC

发明人： James K. Baker

IPC分类号： G06N5/04 ,  G06N20/00

CPC分类号： G06N5/04 ,  G06N20/00

摘要： A machine learning (ML) system includes a student ML system, a learning coach ML system, and a reference system that generates training data for the student ML system. The learning coach ML system learns to make an enhancement to the student ML system or to its learning process, such as updated hyperparameter or a network structural change, based on training of the student ML system with the training data generated by the reference system. The system may also comprise a learning experimentation system that communicates with the reference system to conduct experiments on the learning of the student learning system. Also, the learning experimentation system can determine a cost function for the learning coach ML system.