Abstract:
Described herein are means for the generation of Transferable Visual Word (TransVW) models through self-supervised learning in the absence of manual labeling, in which the trained TransVW models are then utilized for the processing of medical imaging. For instance, an exemplary system is specially configured to perform self-supervised learning for an AI model in the absence of manually labeled input, by performing the following operations: receiving medical images as input; performing a self-discovery operation of anatomical patterns by building a set of the anatomical patterns from the medical images received at the system, performing a self-classification operation of the anatomical patterns; performing a self-restoration operation of the anatomical patterns within cropped and transformed 2D patches or 3D cubes derived from the medical images received at the system by recovering original anatomical patterns to learn different sets of visual representation; and providing a semantics-enriched pre-trained AI model having a trained encoder-decoder structure with skip connections in between based on the performance of the self-discovery operation, the self-classification operation, and the self-restoration operation. Other related embodiments are disclosed.
Abstract:
Described herein are means for learning semantics-enriched representations via self-discovery, self-classification, and self-restoration in the context of medical imaging. Embodiments include the training of deep models to learn semantically enriched visual representation by self-discovery, self-classification, and self-restoration of the anatomy underneath medical images, resulting in a collection of semantics-enriched pre-trained models, called Semantic Genesis. Other related embodiments are disclosed.
Abstract:
A system and methods for polyp detection using optical colonoscopy images are provided. In some aspects, the system includes an input configured to receive a series of optical images, and a processor configured to process the series of optical images with steps comprising of receiving an optical image from the input, constructing an edge map corresponding to the optical image, the edge map comprising a plurality of edge pixel, and generating a refined edge map by applying a classification scheme based on patterns of intensity variation to the plurality of edge pixels in the edge map. The processor may also process the series with steps of identifying polyp candidates using the refined edge map, computing probabilities that identified polyp candidates are polyps, and generating a report, using the computed probabilities, indicating detected polyps. The system also includes an output for displaying the report.
Abstract:
A system for quality assessment of optical colonoscopy images includes an input device configured to acquire a series of images during an optical colonoscopy. A computing device is coupled in communication with the input device and configured to acquire from the input device an input image from the series of images captured during the optical colonoscopy; form a cell grid including a plurality of cells on the input image; perform an image transformation onto the input image with each cell of the plurality of cells within the cell grid; reconstruct each cell to form a reconstructed image; compute a difference image of a sum of a plurality of differences between the input image and the reconstructed image; compute a histogram of the difference image; and apply a probabilistic classifier to the histogram to calculate an informativeness score for the input image.
Abstract:
A system and method for detecting central pulmonary embolisms in a subject's vasculature is provided. In some aspects, the method includes receiving, using the input, a set of images representing a vasculature of the subject's lungs, automatically analyzing the set of images to segment the main arteries associated with the subject's lungs and separate the main arteries from surrounding tissues. The method also includes automatically extracting central pulmonary embolism candidates from the set of images after segmenting and separating the main arteries, and automatically evaluating the central pulmonary embolism candidates in three-dimensional (3D) space by applying a series of rules. The method further includes automatically displaying a report indicating evaluated central pulmonary embolism candidates on a display.
Abstract:
A self-supervised learning framework for empowering instance discrimination in medical imaging using Context-Aware instance Discrimination (CAiD), in which the trained deep models are then utilized for the processing of medical imaging. An exemplary system receives a plurality of medical images; trains a self-supervised learning framework to increasing instance discrimination for medical imaging using a Context-Aware instance Discrimination (CAiD) model using the received plurality of medical images; generates multiple cropped image samples and augments samples using image distortion; applies instance discrimination learning a mapping back to a corresponding original image; reconstructs the cropped image samples and applies an auxiliary context-aware learning loss operation; and generates as output, a pre-trained CAiD model based on the application of both (i) the instance discrimination learning and (ii) the auxiliary context-aware learning loss operation.
Abstract:
Described herein are systems, methods, and apparatuses for actively and continually fine-tuning convolutional neural networks to reduce annotation requirements, in which the trained networks are then utilized in the context of medical imaging. The success of convolutional neural networks (CNNs) in computer vision is largely attributable to the availability of massive annotated datasets, such as ImageNet and Places. However, it is tedious, laborious, and time consuming to create large annotated datasets, and demands costly, specialty-oriented skills. A novel method to naturally integrate active learning and transfer learning (fine-tuning) into a single framework is presented to dramatically reduce annotation cost, starting with a pre-trained CNN to seek “worthy” samples for annotation and gradually enhances the (fine-tuned) CNN via continual fine-tuning. The described method was evaluated using three distinct medical imaging applications, demonstrating that it can reduce annotation efforts by at least half compared with random selection.
Abstract:
Described herein are means for generating source models for transfer learning to application specific models used in the processing of medical imaging. In some embodiments, the method comprises: identifying a group of training samples, wherein each training sample in the group of training samples includes an image; for each training sample in the group of training samples: identifying an original patch of the image corresponding to the training sample; identifying one or more transformations to be applied to the original patch; generating a transformed patch by applying the one or more transformations to the identified patch; and training an encoder-decoder network using a group of transformed patches corresponding to the group of training samples, wherein the encoder-decoder network is trained to generate an approximation of the original patch from a corresponding transformed patch, and wherein the encoder-decoder network is trained to minimize a loss function that indicates a difference between the generated approximation of the original patch and the original patch. The source models significantly enhance the transfer learning performance for many medical imaging tasks including, but not limited to, disease/organ detection, classification, and segmentation. Other related embodiments are disclosed.
Abstract:
Described herein are means for the generation of semantic genesis models through self-supervised learning in the absence of manual labeling, in which the trained semantic genesis models are then utilized for the processing of medical imaging. For instance, an exemplary system is specially configured with means for performing a self-discovery operation which crops 2D patches or crops 3D cubes from similar patient scans received at the system as input; means for transforming each anatomical pattern represented within the cropped 2D patches or the cropped 3D cubes to generate transformed 2D anatomical patterns or transformed 3D anatomical patterns; means for performing a self-classification operation of the transformed anatomical patterns by formulating a C-way multi-class classification task for representation learning; means for performing a self-restoration operation by recovering original anatomical patterns from the transformed 2D patches or transformed 3D cubes having transformed anatomical patterns embedded therein to learn different sets of visual representation; and means for providing a semantics-enriched pre-trained AI model having a trained encoder-decoder structure with skip connections in between based on the performance of the self-discovery operation, the self-classification operation, and the self-restoration operation. Other related embodiments are disclosed.
Abstract:
Described herein are means for training a deep model to learn contrastive representations embedded within part-whole semantics via a self-supervised learning framework, in which the trained deep models are then utilized for the processing of medical imaging. For instance, an exemplary system is specifically configured for performing a random cropping operation to crop a 3D cube from each of a plurality of medical images received at the system as input, performing a resize operation of the cropped 3D cubes, performing an image reconstruction operation of the resized and cropped 3D cubes to predict the resized whole image represented by the original medical images received; and generating a reconstructed image which is analyzed for reconstruction loss against the original image representing a known ground truth image to the reconstruction loss function. Other related embodiments are disclosed.