Framework for Continual Learning Method in Vision Transformers with Representation Replay

    公开(公告)号:US20240054337A1

    公开(公告)日:2024-02-15

    申请号:US17902421

    申请日:2022-09-02

    CPC classification number: G06N3/08 G06N3/0454

    Abstract: A computer-implemented method for continual task learning in a training framework. The method includes: providing a first deep neural network (θw) including a first function (Gw) and a second function (Fw) which are nested; providing a second deep neural network (θs) including a third function (Fs) as a counterpart to the second nested function (Fw); feeding input images to the first neural network (θw), such as through a filter and/or via patch embedding; generating representations of task samples using the first function (Gw); providing a memory (Dm) for storing at least some of the generated representations of task samples and/or having pre-stored task representation; providing the generated and memory stored representations of task samples to the second function (Fw); and providing memory stored representations of task samples to the third function (Fs).

    Computer-Implemented Method of Synaptic Consolidation and Experience Replay in a Dual Memory Architecture

    公开(公告)号:US20230281451A1

    公开(公告)日:2023-09-07

    申请号:US17686263

    申请日:2022-03-03

    CPC classification number: G06N3/082 G06F17/16

    Abstract: A computer-implemented method of synaptic consolidation for training a neural network using an episodic memory, and a semantic memory, by using a Fisher information matrix for estimating the importance of each synapse in the network to previous tasks of the neural network; evaluating the Fisher information matrix on the episodic memory using the semantic memory; adjusting the importance estimate such that functional integrity of the filters in the convolutional layers is maintained whereby the importance of each filter is given by the mean importance of its parameters; using the weights of the semantic memory as the anchor parameters for constraining an update of the synapses of the network based on the adjusted importance estimate; updating the semantic memory and fisher information matrix stochastically using exponential moving average, and interleaving samples from a current task with samples from the episodic memory for performing the training.

    Differencing Based Self-Supervised Pretraining for Change Detection (D-SSCD)

    公开(公告)号:US20230123493A1

    公开(公告)日:2023-04-20

    申请号:US17502729

    申请日:2021-10-15

    Abstract: A computer implemented network for executing a self-supervised scene change detection method, wherein at least one image pair with images captured at different instances of time is processed to detect structural changes caused by an appearance or disappearance of an object in the image pair, and wherein a self-supervised pretraining method is employed that utilizes an unlabelled image pair or pairs to learn representations for scene change detection, and wherein the aligned image pair is subjected to a differencing based self-supervised pre-training method to maximize a correlation between changed regions in the images which provide the structural changes that occur in the image pairs.

    Computer-Implemented Method and a System for a Biologically Plausible Framework for Continual Learning in Artificial Neural Network

    公开(公告)号:US20240046102A1

    公开(公告)日:2024-02-08

    申请号:US17900272

    申请日:2022-08-31

    CPC classification number: G06N3/084 G06N3/0481

    Abstract: A computer-implemented method for general continual learning (CL) in artificial neural network that provides a biologically plausible framework for continual learning which incorporates different mechanisms inspired by the brain. The underlying model comprises separate populations of exclusively excitatory and exclusively inhibitory neurons in each layer which adheres to Dale's principle and the excitatory neurons (mimicking pyramidal cells) are augmented with dendrite-like structures for context-dependent processing of information. The dendritic segments process an additional context signal encoding task information and subsequently modulate the feedforward activity of the excitatory neuron. Additionally, it provides an efficient mechanism for controlling the sparsity in activations using k-WTA (k-Winners-Take-All) activations and Heterogeneous dropout mechanism that encourages the model to use a different set of neurons for each task. This provides an effective approach for maintaining a balance between reusability of features and interference which is critical for enabling CL. Furthermore, it complements the error-based learning with the “fire together, wire together” learning paradigm which further strengthen the association between the context signal and dendritic segments which process them and facilitates context-dependent gating. To further mitigate forgetting, it incorporates synaptic consolidation in conjunction with experience replay.

    Method and System for Dynamic Compositional General Continual Learning

    公开(公告)号:US20230385644A1

    公开(公告)日:2023-11-30

    申请号:US17853682

    申请日:2022-06-29

    CPC classification number: G06N3/082 G06N3/0481

    Abstract: A computer-implemented method for general continual learning combines rehearsal-based methods with dynamic modularity and compositionality. Concretely, the method aims at achieving three objectives: dynamic, sparse, and compositional response to inputs; competent application performance; and—reducing catastrophic forgetting. The proposed method can work without knowledge of task-identities at test-time, it does not employ task-boundaries and it has bounded memory even when training on longer sequences.

    Semantic segmentation architecture
    10.
    发明授权

    公开(公告)号:US11538166B2

    公开(公告)日:2022-12-27

    申请号:US17107283

    申请日:2020-11-30

    Abstract: A semantic segmentation architecture comprising an asymmetric encoder-decoder structure, wherein the architecture comprises further an adapter for linking different stages of the encoder and the decoder. The adapter amalgamates information from both the encoder and the decoder for preserving and refining information between multiple levels of the encoder and decoder. In this way the adapter aggregates features from different levels and intermediates between encoder and decoder.

Patent Agency Ranking