公开(公告)号：US20250068626A1

公开(公告)日：2025-02-27

申请号：US18593316

申请日：2024-03-01

Applicant: Oracle International Corporation

Inventor： Gioacchino Tangari ,  Steve Wai-Chun Siu ,  Dalu Guo ,  Cong Duy Vu Hoang ,  Berk Sarioz ,  Chang Xu ,  Stephen Andrew McRitchie ,  Mark Edward Johnson ,  Christopher Mark Broadbent ,  Thanh Long Duong ,  Srinivasa Phani Kumar Gadde ,  Vishal Vishnoi ,  Chandan Basavaraju ,  Kenneth Khiaw Hong Eng

IPC: G06F16/2452 ,  G06F16/28

Abstract: The present disclosure relates to manufacturing training data by leveraging an automated pipeline that manufactures visualization training datasets to train a machine learning model to convert a natural language utterance into meaning representation language logical form that includes one or more visualization actions. Aspects are directed towards accessing an original training dataset, a visualization query dataset, an incremental visualization dataset, a manipulation visualization dataset, or any combination thereof. One or more visualization training datasets are generated by: (i) modifying examples in the original training dataset, the visualization query dataset, or both to include visualization actions, (ii) generating examples, using the incremental visualization dataset, the manipulation visualization dataset, or both, that include visualization actions, or (iii) both (i) and (ii). An augmented training dataset is generated by adding the one or more visualization training datasets to the original training dataset and then used to train the machine learning model.

公开(公告)号：US12026468B2

公开(公告)日：2024-07-02

申请号：US17452743

申请日：2021-10-28

Applicant: Oracle International Corporation

Inventor： Elias Luqman Jalaluddin ,  Vishal Vishnoi ,  Thanh Long Duong ,  Mark Edward Johnson ,  Poorya Zaremoodi ,  Gautam Singaraju ,  Ying Xu ,  Vladislav Blinov ,  Yu-Heng Hong

IPC: G06F40/289 ,  G06F40/30 ,  G06N3/08 ,  H04L51/02

CPC classification number: G06F40/289 ,  G06F40/30 ,  G06N3/08 ,  H04L51/02

Abstract: Techniques for out-of-domain data augmentation for training chatbot systems in natural language processing. In one particular aspect, a method is provided that includes receiving a training set of utterances for training a machine-learning model to identify one or more intents for one or more utterances, and augmenting the training set of utterances with out-of-domain (OOD) examples. The augmenting includes: generating a data set of OOD examples, filtering out OOD examples from the data set of OOD examples, determining a difficulty value for each OOD example remaining within the filtered data set of the OOD examples, and generating augmented batches of utterances comprising utterances from the training set of utterances and utterances from the filtered data set of the OOD based on the difficulty value for each OOD. Thereafter, the machine-learning model is trained using the augmented batches of utterances in accordance with a curriculum training protocol.

公开(公告)号：US12019994B2

公开(公告)日：2024-06-25

申请号：US17456916

申请日：2021-11-30

Applicant: Oracle International Corporation

Inventor： Ying Xu ,  Poorya Zaremoodi ,  Thanh Tien Vu ,  Cong Duy Vu Hoang ,  Vladislav Blinov ,  Yu-Heng Hong ,  Yakupitiyage Don Thanuja Samodhye Dharmasiri ,  Vishal Vishnoi ,  Elias Luqman Jalaluddin ,  Manish Parekh ,  Thanh Long Duong ,  Mark Edward Johnson

IPC: G06F17/18 ,  G06F40/35 ,  G06N20/00 ,  H04L51/02 ,  G06F40/205 ,  G06F40/253

CPC classification number: G06F40/35 ,  G06N20/00 ,  H04L51/02 ,  G06F40/205 ,  G06F40/253

Abstract: Techniques for using logit values for classifying utterances and messages input to chatbot systems in natural language processing. A method can include a chatbot system receiving an utterance generated by a user interacting with the chatbot system. The chatbot system can input the utterance into a machine-learning model including a set of binary classifiers. Each binary classifier of the set of binary classifiers can be associated with a modified logit function. The method can also include the machine-learning model using the modified logit function to generate a set of distance-based logit values for the utterance. The method can also include the machine-learning model applying an enhanced activation function to the set of distance-based logit values to generate a predicted output. The method can also include the chatbot system classifying, based on the predicted output, the utterance as being associated with the particular class.

公开(公告)号：US11972220B2

公开(公告)日：2024-04-30

申请号：US17456687

申请日：2021-11-29

Applicant: Oracle International Corporation

Inventor： Ying Xu ,  Poorya Zaremoodi ,  Thanh Tien Vu ,  Cong Duy Vu Hoang ,  Vladislav Blinov ,  Yu-Heng Hong ,  Yakupitiyage Don Thanuja Samodhye Dharmasiri ,  Vishal Vishnoi ,  Elias Luqman Jalaluddin ,  Manish Parekh ,  Thanh Long Duong ,  Mark Edward Johnson

IPC: G06F40/35 ,  G06F40/205 ,  G06F40/253 ,  G06N3/08 ,  H04L51/02

CPC classification number: G06F40/35 ,  G06N3/08 ,  H04L51/02 ,  G06F40/205 ,  G06F40/253

Abstract: Techniques for using enhanced logit values for classifying utterances and messages input to chatbot systems in natural language processing. A method can include a chatbot system receiving an utterance generated by a user interacting with the chatbot system and inputting the utterance into a machine-learning model including a series of network layers. A final network layer of the series of network layers can include a logit function. The machine-learning model can map a first probability for a resolvable class to a first logit value using the logit function. The machine-learning model can map a second probability for a unresolvable class to an enhanced logit value. The method can also include the chatbot system classifying the utterance as the resolvable class or the unresolvable class based on the first logit value and the enhanced logit value.

公开(公告)号：US20240135116A1

公开(公告)日：2024-04-25

申请号：US18485779

申请日：2023-10-12

Applicant: Oracle International Corporation

Inventor： Duy Vu ,  Poorya Zaremoodi ,  Nagaraj N. Bhat ,  Srijon Sarkar ,  Varsha Kuppur Rajendra ,  Thanh Long Duong ,  Mark Edward Johnson ,  Pramir Sarkar ,  Shahid Reza

IPC: G06F40/58 ,  G06F40/20

CPC classification number: G06F40/58 ,  G06F40/20

Abstract: A computer-implemented method includes: accessing a plurality of datasets, where each dataset of the plurality of datasets includes training examples; selecting datasets that include the training examples in a source language and a target language; and sampling, based on a sampling weight that is determined for each of the selected datasets, the training examples from the selected datasets to generate the training batches; training an ML model for performing at least a first task using the training examples of the training batches, by interleavingly inputting the training batches to the ML model; and outputting the trained ML model configured to perform the at least the first task on input utterances provided in at least one among the source language and the target language. The sampling weight is determined for each of the selected datasets based on one or more attributes common to the training examples of the selected dataset.

公开(公告)号：US20240061833A1

公开(公告)日：2024-02-22

申请号：US18218385

申请日：2023-07-05

Applicant: Oracle International Corporation

Inventor： Gioacchino Tangari ,  Nitika Mathur ,  Philip Arthur ,  Cong Duy Vu Hoang ,  Aashna Devang Kanuga ,  Steve Wai-Chun Siu ,  Syed Najam Abbas Zaidi ,  Poorya Zaremoodi ,  Thanh Long Duong ,  Mark Edward Johnson

IPC: G06F16/2452 ,  G06F16/242 ,  G06F40/247 ,  G06F40/284

CPC classification number: G06F16/24522 ,  G06F16/243 ,  G06F40/247 ,  G06F40/284

Abstract: Techniques are disclosed for augmenting training data for training a machine learning model to generate database queries. Training data comprising a first training example comprising a first natural language utterance, a logical form for the first natural language utterance, and associated first metadata is obtained. From the first training example, a template utterance is generated. A second natural language utterance is generated by filling slots in the template utterance based on a database schema and database values. Updated metadata is produced based on the first metadata and the second natural language utterance. A second training example is generated, comprising the second natural language utterance, the logical form for the first natural language utterance, and the updated metadata. The training data is augmented by adding the second training example. A machine learning model is trained to generate a database query comprising the database operation using the augmented training data set.

公开(公告)号：US20230206125A1

公开(公告)日：2023-06-29

申请号：US18087647

申请日：2022-12-22

Applicant: Oracle International Corporation

Inventor： Tuyen Quang Pham ,  Cong Duy Vu Hoang ,  Thanh Tien Vu ,  Mark Edward Johnson ,  Thanh Long Duong

IPC: G06N20/00 ,  G06F40/35 ,  G06F40/284 ,  G06F40/295 ,  G06F40/253

CPC classification number: G06N20/00 ,  G06F40/35 ,  G06F40/284 ,  G06F40/295 ,  G06F40/253 ,  G06F40/205

Abstract: Techniques are provided for improved training of a machine learning model using lexical dropout. A machine learning model and a training data set are accessed. The training data set can include sample utterances and corresponding labels. A dropout parameter is identified. The dropout parameter can indicate a likelihood for dropping out one or more feature vectors for tokens associated with respective entities during training of the machine learning model. The dropout parameter is applied to feature vectors for tokens associated with respective entities. The machine learning model is trained using the training data set and the dropout parameter to generate a trained machine learning model. The use of the trained the machine learning model is facilitated.

公开(公告)号：US20230186025A1

公开(公告)日：2023-06-15

申请号：US18065387

申请日：2022-12-13

Applicant: Oracle International Corporation

Inventor： Jae Min John ,  Vishal Vishnoi ,  Mark Edward Johnson ,  Thanh Long Duong ,  Srinivasa Phani Kumar Gadde ,  Balakota Srinivas Vinnakota ,  Shivashankar Subramanian ,  Cong Duy Vu Hoang ,  Yakupitiyage Don Thanuja Samodhye Dharmasiri ,  Nitika Mathur ,  Aashna Devang Kanuga ,  Philip Arthur ,  Gioacchino Tangari ,  Steve Wai-Chun Siu

IPC: G06F40/284 ,  G06F40/295 ,  G06F40/42

CPC classification number: G06F40/284 ,  G06F40/295 ,  G06F40/42

Abstract: Techniques for preprocessing data assets to be used in a natural language to logical form model based on scalable search and content-based schema linking. In one particular aspect, a method includes accessing an utterance, classifying named entities within the utterance into predefined classes, searching value lists within the database schema using tokens from the utterance to identify and output value matches including: (i) any value within the value lists that matches a token from the utterance and (ii) any attribute associated with a matching value, generating a data structure by organizing and storing: (i) each of the named entities and an assigned class for each of the named entities, (ii) each of the value matches and the token matching each of the value matches, and (iii) the utterance, in a predefined format for the data structure, and outputting the data structure.

公开(公告)号：US20230154455A1

公开(公告)日：2023-05-18

申请号：US17988125

申请日：2022-11-16

Applicant: Oracle International Corporation

Inventor： Thanh Tien Vu ,  Tuyen Quang Pham ,  Mark Edward Johnson ,  Thanh Long Duong

IPC: G10L15/06 ,  G10L15/18 ,  G10L15/22 ,  G06F40/279 ,  G06F40/30

CPC classification number: G10L15/063 ,  G10L15/1815 ,  G10L15/22 ,  G06F40/279 ,  G06F40/30

Abstract: Techniques are provided for improved training of a machine-learning model that includes multiple layers and is configured to process textual language input. The machine-learning model includes one or more blocks in which each block includes a multi-head self-attention network, a first connection for providing input to the multi-head self-attention network, and a second (residual) connection for providing the input to a normalization layer, bypassing the multi-head self-attention network. During training, the second connection is dropped out according to a dropout parameter. Additionally, or alternatively, an attention weight matrix is used for dropout by blocking diagonal entries in the attention weight matrix. As a result, the machine-learning model increasingly focuses on contextual information, which provides more accurate language processing results.

公开(公告)号：US20230153528A1

公开(公告)日：2023-05-18

申请号：US17984743

申请日：2022-11-10

Applicant: Oracle International Corporation

Inventor： Duy Vu ,  Varsha Kuppur Rajendra ,  Dai Hoang Tran ,  Shivashankar Subramanian ,  Poorya Zaremoodi ,  Thanh Long Duong ,  Mark Edward Johnson

IPC: G06F40/279 ,  G06F40/166 ,  G06N5/02

CPC classification number: G06F40/279 ,  G06F40/166 ,  G06N5/022

Abstract: Techniques for augmentation and batch balancing of training data to enhance negation and fairness of a machine learning model. In one particular aspect, a method is provided that includes generating a list of demographic words associated with a demographic group, searching an unlabeled corpus of text to identify unlabeled examples in a target domain comprising at least one demographic word from the list of demographic words, rewriting the unlabeled examples to create one or more versions of each of the unlabeled examples and generate a fairness invariance data set, and training the machine learning model using unlabeled examples from the fairness invariance data set.