-
公开(公告)号：US12045725B1
公开(公告)日：2024-07-23
申请号：US16923006
申请日：2020-07-07
申请人： Perceive Corporation
发明人： Eric A. Sather , Steven L. Teig
摘要： Some embodiments provide a method for training a network including layers that each includes multiple nodes. The method identifies a set of related layers of the network. Each node in one of the related layers has corresponding nodes in each of the other related layers. Each set of corresponding nodes receives a same set of inputs and applies different sets of weights to the inputs to generate an output. The method identifies an element-wise addition layer including nodes that each add outputs of a different set of corresponding nodes from the related layers to generate a sum. The method uses a set of outputs generated by the nodes of each related layer to determine batch normalization parameters specific to each layer of the set of related layers. The method uses data generated by the element-wise addition layer to determine batch normalization parameters for the set of related layers.
-
公开(公告)号：US11610154B1
公开(公告)日：2023-03-21
申请号：US16780841
申请日：2020-02-03
申请人： Perceive Corporation
发明人： Steven L. Teig , Eric A. Sather
摘要： Some embodiments provide a method for training a machine-trained (MT) network. The method uses a first set of inputs to train parameters of the MT network according to a set of hyperparameters that define aspects of the training. The method uses a second set of inputs to validate the MT network as trained by the first set of inputs. Based on the validation, the method modifies the hyperparameters for subsequent training of the MT network, wherein the hyperparameter modification is constrained to prevent overfitting of the modified hyperparameters to the second set of inputs.
-
公开(公告)号：US11604973B1
公开(公告)日：2023-03-14
申请号：US16698942
申请日：2019-11-27
申请人： Perceive Corporation
发明人： Eric A. Sather , Steven L. Teig
摘要： Some embodiments provide a method for training parameters of a machine-trained (MT) network. The method receives an MT network with multiple layers of nodes, each of which computes an output value based on a set of input values and a set of trained weight values. Each layer has a set of allowed weight values. For a first layer with a first set of allowed weight values, the method defines a second layer with nodes corresponding to each of the nodes of the first layer, each second-layer node receiving the same input values as the corresponding first-layer node. The second layer has a second, different set of allowed weight values, with the output values of the nodes of the first layer added with the output values of the corresponding nodes of the second layer to compute output values that are passed to a subsequent layer. The method trains the weight values.
-
公开(公告)号：US11537870B1
公开(公告)日：2022-12-27
申请号：US15921622
申请日：2018-03-14
申请人： Perceive Corporation
发明人： Steven L. Teig , Eric A. Sather
摘要： Some embodiments provide a method for training a machine-trained (MT) network. The method propagates multiple inputs through the MT network to generate an output for each of the inputs. each of the inputs is associated with an expected output, the MT network uses multiple network parameters to process the inputs, and each network parameter of a set of the network parameters is defined during training as a probability distribution across a discrete set of possible values for the network parameter. The method calculates a value of a loss function for the MT network that includes (i) a first term that measures network error based on the expected outputs compared to the generated outputs and (ii) a second term that penalizes divergence of the probability distribution for each network parameter in the set of network parameters from a predefined probability distribution for the network parameter.
-
公开(公告)号：US11163986B2
公开(公告)日：2021-11-02
申请号：US16852329
申请日：2020-04-17
申请人： Perceive Corporation
发明人： Eric A. Sather , Steven L. Teig , Andrew C. Mihal
摘要： Some embodiments provide a method for training a machine-trained (MT) network that processes inputs using network parameters. The method propagates a set of input training items through the MT network to generate a set of output values. The set of input training items comprises multiple training items for each of multiple categories. The method identifies multiple training item groupings in the set of input training items. Each grouping includes at least two training items in a first category and at least one training item in a second category. The method calculates a value of a loss function as a summation of individual loss functions for each of the identified training item groupings. The individual loss function for each particular training item grouping is based on the output values for the training items of the grouping. The method trains the network parameters using the calculated loss function value.
-
公开(公告)号：US10592732B1
公开(公告)日：2020-03-17
申请号：US15901459
申请日：2018-02-21
申请人： Perceive Corporation
发明人： Eric A. Sather , Steven L. Teig , Andrew C. Mihal
摘要： Some embodiments provide a method for training a machine-trained (MT) network that processes images using multiple network parameters. The method propagates a triplet of input images through the MT network to generate an output value for each of the input images. The triplet includes an anchor first image, a second image of a same category as the anchor image, and a third image of a different category as the anchor image. The method calculates a value of a loss function for the triplet that is based on a probabilistic classification of an output value for the anchor image compared to output values for the second and third images. The method uses the calculated loss function value to train the network parameters.
-
公开(公告)号：US12112254B1
公开(公告)日：2024-10-08
申请号：US16780843
申请日：2020-02-03
申请人： Perceive Corporation
发明人： Steven L. Teig , Eric A. Sather
摘要： Some embodiments provide a method for training a machine-trained (MT) network. The method uses a set of training inputs to train parameters of the MT network according to an initial loss function. The method uses a set of validation inputs to compute an error measure for the MT network as trained by the first set of training inputs. The method modifies the loss function for subsequent training of the MT network based on the computed error measure. The method uses the set of training inputs to train the parameters of the MT network according to the modified loss function.
-
公开(公告)号：US11995537B1
公开(公告)日：2024-05-28
申请号：US15921633
申请日：2018-03-14
申请人： Perceive Corporation
发明人： Eric A. Sather , Steven L. Teig , Andrew C. Mihal
CPC分类号： G06N3/08 , G06F11/004 , G06F2201/87
摘要： Some embodiments provide a method for training a machine-trained (MT) network that processes input data using network parameters. The method maps a set of input instances to a set of output values by propagating the set of input instances through the MT network. The set of input instances include input instances for each of multiple categories. The method selects multiple input instances as anchor instances. For each anchor instance, the method computes a loss function as a comparison between the output value for the anchor instance and each output value for an input instance in a different category than the anchor. The method computes a total loss function for the MT network as a sum of the loss function computed for each anchor instance. The method trains the network parameters using the computed total loss function.
-
公开(公告)号：US11741369B2
公开(公告)日：2023-08-29
申请号：US17514701
申请日：2021-10-29
申请人： Perceive Corporation
发明人： Eric A. Sather , Steven L. Teig , Andrew C. Mihal
IPC分类号： G06N3/08 , G06N3/084 , G06T7/00 , G06N3/04 , G06V40/16 , G06F18/214 , G06F18/21 , G06V10/764 , G06V10/82 , G06V10/44
CPC分类号： G06N3/084 , G06F18/214 , G06F18/217 , G06N3/04 , G06N3/08 , G06T7/97 , G06V10/454 , G06V10/764 , G06V10/82 , G06V40/167 , G06V40/172 , G06T2207/20081 , G06T2207/30201
摘要： Some embodiments provide a method for training a machine-trained (MT) network that processes inputs using network parameters. The method propagates a set of input training items through the MT network to generate a set of output values. The set of input training items comprises multiple training items for each of multiple categories. The method identifies multiple training item groupings in the set of input training items. Each grouping includes at least two training items in a first category and at least one training item in a second category. The method calculates a value of a loss function as a summation of individual loss functions for each of the identified training item groupings. The individual loss function for each particular training item grouping is based on the output values for the training items of the grouping. The method trains the network parameters using the calculated loss function value.
-
公开(公告)号：US11017295B1
公开(公告)日：2021-05-25
申请号：US15815251
申请日：2017-11-16
申请人： Perceive Corporation
发明人： Steven L. Teig , Eric A. Sather
IPC分类号： G06N3/08
摘要： Some embodiments provide a set of processing units and a set of machine-readable media. The set of machine-readable media stores sets of instructions for applying a network of computation nodes to an input received by the device. The network of computation nodes includes multiple layers of nodes. The set of machine-readable media stores a set of machine-trained weight parameters for configuring the network to perform a specific function. Each layer of nodes has an associated value, and each of the weight parameters is associated with a computation node. Each weight parameter is zero, the associated value for the layer of the computation node with which the weight parameter is associated, or the negative of the associated value for the layer of the computation node with which the weight parameter is associated. Each weight value is stored using two bits or less of data.
-
-
-
-
-
-
-
-
-
搜索结果
33 条记录 (耗时 0.02 秒)
