公开(公告)号：US20220237465A1

公开(公告)日：2022-07-28

申请号：US17235516

申请日：2021-04-20

申请人： Numenta, Inc.

发明人： Marcus Anthony Lewis ,  Subutai Ahmad

IPC分类号： G06N3/08 ,  G06N3/04 ,  G06K9/62

摘要： A sparse neural network is trained such that weights or layer outputs of the neural network satisfy sparsity constraints. The sparsity is controlled by pruning one or more subsets of weights based on their signal-to-noise ratio (SNR). During the training process, an inference system generates outputs for a current layer by applying a set of weights for the current layer to a layer output of a previous layer. The set of weights for the current layer may be modeled as random variables sampled from probability distributions. The inference system determines a loss function and updates the set of weights by backpropagating error terms obtained from the loss function. This process is repeated until a convergence criterion is reached. One or more subsets of weights are then pruned based on their SNR depending on sparsity constraints for the weights of the neural network.

公开(公告)号：US20210201181A1

公开(公告)日：2021-07-01

申请号：US17198808

申请日：2021-03-11

申请人： Numenta, Inc.

发明人： Jeffrey C. Hawkins ,  Subutai Ahmad ,  Yuwei Cui ,  Marcus Anthony Lewis

IPC分类号： G06N5/04 ,  G06N20/00

摘要： Embodiments relate to performing inference, such as object recognition, based on sensory inputs received from sensors and location information associated with the sensory inputs. The sensory inputs describe one or more features of the objects. The location information describes known or potential locations of the sensors generating the sensory inputs. An inference system learns representations of objects by characterizing a plurality of feature-location representations of the objects, and then performs inference by identifying or updating candidate objects consistent with feature-location representations observed from the sensory input data and location information. In one instance, the inference system learns representations of objects for each sensor. The set of candidate objects for each sensor is updated to those consistent with candidate objects for other sensors, as well as the observed feature-location representations for the sensor.

公开(公告)号：US20210158168A1

公开(公告)日：2021-05-27

申请号：US16696991

申请日：2019-11-26

申请人： Numenta, Inc.

发明人： Subutai Ahmad ,  Luiz Scheinkman

IPC分类号： G06N3/08 ,  G06N3/04

摘要： An inference system trains and performs inference using a sparse neural network. The sparse neural network may include one or more layers, and each layer may be associated with a set of sparse weights that represent sparse connections between nodes of a layer and nodes of a previous layer. A layer output may be generated by applying the set of sparse weights associated with the layer to the layer output of a previous layer. Moreover, the one or more layers of the sparse neural network may generate sparse layer outputs. By using sparse representations of weights and layer outputs, robustness and stability of the neural network can be significantly improved, while maintaining competitive accuracy.

公开(公告)号：US10528863B2

公开(公告)日：2020-01-07

申请号：US15089175

申请日：2016-04-01

申请人： Numenta, Inc.

发明人： Jeffrey C. Hawkins ,  Subutai Ahmad

IPC分类号： G06N3/04 ,  G06N3/08 ,  G06N5/04

摘要： Embodiments relate to a first processing node that processes an input data having a temporal sequence of spatial patterns by retaining a higher-level context of the temporal sequence. The first processing node performs temporal processing based at least on feedback inputs received from a second processing node. The first processing node determines whether learned temporal sequences are included in the input data based on sequence inputs transmitted within the same level of a hierarchy of processing nodes and the feedback inputs received from an upper level of the hierarchy of processing nodes.

公开(公告)号：US09530091B2

公开(公告)日：2016-12-27

申请号：US13438670

申请日：2012-04-03

申请人： Jeffrey Hawkins ,  Dileep George

发明人： Jeffrey Hawkins ,  Dileep George

IPC分类号： G06N7/00 ,  G06N3/02

CPC分类号： G06N3/02 ,  G06N7/005

摘要： Sophisticated memory systems and intelligent machines may be constructed by creating an active memory system with a hierarchical architecture. Specifically, a system may comprise a plurality of individual cortical processing units arranged into a hierarchical structure. Each individual cortical processing unit receives a sequence of patterns as input. Each cortical processing unit processes the received input sequence of patterns using a memory containing previously encountered sequences with structure and outputs another pattern. As several input sequences are processed by a cortical processing unit, it will therefore generate a sequence of patterns on its output. The sequence of patterns on its output may be passed as an input to one or more cortical processing units in next higher layer of the hierarchy. A lowest layer of cortical processing units may receive sensory input from the outside world. The sensory input also comprises a sequence of patterns.

公开(公告)号：US09159021B2

公开(公告)日：2015-10-13

申请号：US13658200

申请日：2012-10-23

申请人： Numenta, Inc.

发明人： Jeffrey C. Hawkins ,  Ronald Marianetti

IPC分类号： G06N3/063 ,  G06N3/04 ,  G06N99/00

CPC分类号： G06N3/049 ,  G06N3/063 ,  G06N99/005

摘要： Embodiments relate to making predictions for values or states to follow multiple time steps after receiving a certain input data in a spatial and temporal memory system. During a training stage, relationships between states of the spatial and temporal memory system at certain times and spatial patterns of the input data detected a plurality of time steps later after the certain time steps are established. Using the established relationships, the spatial and temporal memory system can make predictions multiple time steps into the future based on the input data received at a current time.

摘要翻译： 实施例涉及在空间和时间存储器系统中接收到特定输入数据之后对值或状态进行预测以跟随多个时间步长。 在训练阶段期间，在某些时间步骤建立之后，时间存储系统的状态与特定时间的状态之间的关系以及输入数据的空间模式在后续的多个时间步骤中被检测。 使用已建立的关系，空间和时间记忆系统可以根据当前时间接收到的输入数据，预测未来的多个时间步长。

公开(公告)号：US08732098B2

公开(公告)日：2014-05-20

申请号：US13415713

申请日：2012-03-08

申请人： Subutai Ahmad ,  Dileep George ,  Jeffrey L. Edwards ,  William C. Saphir ,  Frank Astier ,  Ronald Marianetti

发明人： Subutai Ahmad ,  Dileep George ,  Jeffrey L. Edwards ,  William C. Saphir ,  Frank Astier ,  Ronald Marianetti

IPC分类号： G06N3/04 ,  G06N3/08

CPC分类号： H04L67/42 ,  G06K9/00979 ,  G06K9/66 ,  G06N3/04 ,  G06N3/08 ,  H04L51/12 ,  H04L67/02 ,  H04L67/1017 ,  H04L67/1097

摘要： A web-based hierarchical temporal memory (HTM) system in which one or more client devices communicate with a remote server via a communication network. The remote server includes at least a HTM server for implementing a hierarchical temporal memory (HTM). The client devices generate input data including patterns and sequences, and send the input data to the remote server for processing. The remote server (specifically, the HTM server) performs processing in order to determine the causes of the input data, and sends the results of this processing to the client devices. The client devices need not have processing and/or storage capability for running the HTM but may nevertheless take advantage of the HTM by submitting a request to the HTM server.

摘要翻译： 一种基于网络的分级时间存储器（HTM）系统，其中一个或多个客户端设备经由通信网络与远程服务器进行通信。 远程服务器至少包括用于实现分级时间存储器（HTM）的HTM服务器。 客户端设备生成包括模式和序列的输入数据，并将输入数据发送到远程服务器进行处理。 远程服务器（特别是HTM服务器）执行处理以确定输入数据的原因，并将该处理的结果发送给客户端设备。 客户端设备不需要具有用于运行HTM的处理和/或存储能力，但仍然可以通过向HTM服务器提交请求来利用HTM。

公开(公告)号：US08666917B2

公开(公告)日：2014-03-04

申请号：US13604543

申请日：2012-09-05

申请人： Robert G. Jaros ,  Dileep George ,  Jeffrey Hawkins ,  Frank Astier

发明人： Robert G. Jaros ,  Dileep George ,  Jeffrey Hawkins ,  Frank Astier

IPC分类号： G06F15/18

CPC分类号： G06N3/08 ,  G06F15/18 ,  G06N3/04 ,  G06N3/0454 ,  G06N3/0472 ,  G06N3/049 ,  G06N5/02 ,  G06N7/005 ,  G06N99/005

摘要： A hierarchy of computing modules is configured to learn a cause of input data sensed over space and time, and is further configured to determine a cause of novel sensed input data dependent on the learned cause. At least one of the computing modules has a sequence learner module configured to associate sequences of input data received by the computing module to a set of causes previously learned in the hierarchy.

摘要翻译： 计算模块的层次结构被配置为学习在空间和时间上感测的输入数据的原因，并且还被配置为根据学习的原因确定新感测的输入数据的原因。 计算模块中的至少一个具有序列学习器模块，其被配置为将由计算模块接收的输入数据的序列与先前在层次结构中学习的一组原因相关联。

公开(公告)号：US20130054496A1

公开(公告)日：2013-02-28

申请号：US13218202

申请日：2011-08-25

申请人： Ronald Marianetti, II ,  Anosh Raj ,  Subutai Ahmad

发明人： Ronald Marianetti, II ,  Anosh Raj ,  Subutai Ahmad

IPC分类号： G06F15/18

CPC分类号： G06N5/047 ,  G06N99/005

摘要： A spatial and temporal memory system (STMS) processes input data to detect whether spatial patterns and/or temporal sequences of spatial patterns exist within the data, and to make predictions about future data. The data processed by the STMS may be retrieved from, for example, one or more database fields and is encoded into a distributed representation format using a coding scheme. The performance of the STMS in predicting future data is evaluated for the coding scheme used to process the data as performance data. The selection and prioritization of STMS experiments to perform may be based on the performance data for an experiment. The best fields, encodings, and time aggregations for generating predictions can be determined by an automated search and evaluation of multiple STMS systems.

摘要翻译： 空间和时间存储系统（STMS）处理输入数据以检测数据中是否存在空间模式的空间模式和/或时间序列，以及对未来数据的预测。 可以从例如一个或多个数据库字段检索由STMS处理的数据，并使用编码方案将其编码为分布式表示格式。 对用于处理数据的编码方案作为性能数据，评估STMS预测未来数据的性能。 要执行的STMS实验的选择和优先级可以基于实验的性能数据。 可以通过自动搜索和评估多个STMS系统来确定用于生成预测的最佳字段，编码和时间聚合。

公开(公告)号：US08290886B2

公开(公告)日：2012-10-16

申请号：US13333865

申请日：2011-12-21

申请人： Dileep George ,  Jeffrey C. Hawkins

发明人： Dileep George ,  Jeffrey C. Hawkins

IPC分类号： G06E1/00 ,  G06E3/00 ,  G06F15/18 ,  G06G7/00

CPC分类号： G06N3/08

摘要： Memory networks and methods are provided. Machine intelligence is achieved by a plurality of linked processor units in which child modules receive input data. The input data are processed to identify patterns and/or sequences. Data regarding the observed patterns and/or sequences are passed to a patent module which may receive as inputs data from one or more child modules. the parent module examines its input data for patterns and/or sequences and then provides feedback to the child module or modules regarding the parent-level patterns that correlate with the child-level patterns. These systems and methods are extensible to large networks of interconnected processor modules.