Cross-lingual regularization for multilingual generalization

    公开(公告)号:US11829727B2

    公开(公告)日:2023-11-28

    申请号:US17239297

    申请日:2021-04-23

    CPC classification number: G06F40/58 G06F40/51 G06N3/08 G06N20/00

    Abstract: Approaches for cross-lingual regularization for multilingual generalization include a method for training a natural language processing (NLP) deep learning module. The method includes accessing a first dataset having a first training data entry, the first training data entry including one or more natural language input text strings in a first language; translating at least one of the one or more natural language input text strings of the first training data entry from the first language to a second language; creating a second training data entry by starting with the first training data entry and substituting the at least one of the natural language input text strings in the first language with the translation of the at least one of the natural language input text strings in the second language; adding the second training data entry to a second dataset; and training the deep learning module using the second dataset.

    DEEP NEURAL NETWORK-BASED DECISION NETWORK

    公开(公告)号:US20220164635A1

    公开(公告)日:2022-05-26

    申请号:US17670368

    申请日:2022-02-11

    Abstract: The technology disclosed proposes using a combination of computationally cheap, less-accurate bag of words (BoW) model and computationally expensive, more-accurate long short-term memory (LSTM) model to perform natural processing tasks such as sentiment analysis. The use of cheap, less-accurate BoW model is referred to herein as “skimming”. The use of expensive, more-accurate LSTM model is referred to herein as “reading”. The technology disclosed presents a probability-based guider (PBG). PBG combines the use of BoW model and the LSTM model. PBG uses a probability thresholding strategy to determine, based on the results of the BoW model, whether to invoke the LSTM model for reliably classifying a sentence as positive or negative. The technology disclosed also presents a deep neural network-based decision network (DDN) that is trained to learn the relationship between the BoW model and the LSTM model and to invoke only one of the two models.

    SYSTEMS AND METHODS FOR LANGUAGE MODELING OF PROTEIN ENGINEERING

    公开(公告)号:US20210249104A1

    公开(公告)日:2021-08-12

    申请号:US17001090

    申请日:2020-08-24

    Abstract: The present disclosure provides systems and methods for controllable protein generation. According to some embodiments, the systems and methods leverage neural network models and techniques that have been developed for other fields, in particular, natural language processing (NLP). In some embodiments, the systems and methods use or employ models implemented with transformer architectures developed for language modeling and apply the same to generative modeling for protein engineering.

    Cross-Lingual Regularization for Multilingual Generalization

    公开(公告)号:US20200285706A1

    公开(公告)日:2020-09-10

    申请号:US16399429

    申请日:2019-04-30

    Abstract: Approaches for cross-lingual regularization for multilingual generalization include a method for training a natural language processing (NLP) deep learning module. The method includes accessing a first dataset having a first training data entry, the first training data entry including one or more natural language input text strings in a first language; translating at least one of the one or more natural language input text strings of the first training data entry from the first language to a second language; creating a second training data entry by starting with the first training data entry and substituting the at least one of the natural language input text strings in the first language with the translation of the at least one of the natural language input text strings in the second language; adding the second training data entry to a second dataset; and training the deep learning module using the second dataset.

    Systems and methods for unifying question answering and text classification via span extraction

    公开(公告)号:US11657233B2

    公开(公告)日:2023-05-23

    申请号:US17673709

    申请日:2022-02-16

    CPC classification number: G06F40/30 G06F40/284 G06F16/3329 G06N3/08

    Abstract: Systems and methods for unifying question answering and text classification via span extraction include a preprocessor for preparing a source text and an auxiliary text based on a task type of a natural language processing task, an encoder for receiving the source text and the auxiliary text from the preprocessor and generating an encoded representation of a combination of the source text and the auxiliary text, and a span-extractive decoder for receiving the encoded representation and identifying a span of text within the source text that is a result of the NLP task. The task type is one of entailment, classification, or regression. In some embodiments, the source text includes one or more of text received as input when the task type is entailment, a list of classifications when the task type is entailment or classification, or a list of similarity options when the task type is regression.

    Multitask learning as question answering

    公开(公告)号:US11600194B2

    公开(公告)日:2023-03-07

    申请号:US16006691

    申请日:2018-06-12

    Abstract: Approaches for natural language processing include a multi-layer encoder for encoding words from a context and words from a question in parallel, a multi-layer decoder for decoding the encoded context and the encoded question, a pointer generator for generating distributions over the words from the context, the words from the question, and words in a vocabulary based on an output from the decoder, and a switch. The switch generates a weighting of the distributions over the words from the context, the words from the question, and the words in the vocabulary, generates a composite distribution based on the weighting of the distribution over the first words from the context, the distribution over the second words from the question, and the distribution over the words in the vocabulary, and selects words for inclusion in an answer using the composite distribution.

    Leveraging language models for generating commonsense explanations

    公开(公告)号:US11366969B2

    公开(公告)日:2022-06-21

    申请号:US16393801

    申请日:2019-04-24

    Abstract: According to some embodiments, systems and methods are provided to develop or provide common sense auto-generated explanations (CAGE) for the reasoning used by an artificial intelligence, neural network, or deep learning model to make a prediction. In some embodiments, the systems and methods use supervised fine-tuning on a language model (LM) to generate such explanations. These explanations may then be used for downstream classification.

Patent Agency Ranking