公开(公告)号：US20250156649A1

公开(公告)日：2025-05-15

申请号：US18505498

申请日：2023-11-09

Applicant: Oracle International Corporation

Inventor： Gioacchino Tangari ,  Chang Xu ,  Nitika Mathur ,  Philip Arthur ,  Syed Najam Abbas Zaidi ,  Aashna Devang Kanuga ,  Cong Duy Vu Hoang ,  Poorya Zaremoodi ,  Thanh Long Duong ,  Mark Edward Johnson ,  Vishal Vishnoi

IPC: G06F40/40 ,  G06F40/211 ,  G06F40/284

Abstract: Techniques are disclosed herein for improving model robustness on operators and triggering keywords in natural language to a meaning representation language system. The techniques include augmenting an original set of training data for a target robustness bucket by leveraging a combination of two training data generation techniques: (1) modification of existing training examples and (2) synthetic template-based example generation. The resulting set of augmented data examples from the two training data generation techniques are appended to the original set of training data to generate an augmented training data set and the augmented training data set is used to train a machine learning model to generate logical forms for utterances.

公开(公告)号：US12293155B2

公开(公告)日：2025-05-06

申请号：US18630772

申请日：2024-04-09

Applicant: Oracle International Corporation

Inventor： Elias Luqman Jalaluddin ,  Vishal Vishnoi ,  Thanh Long Duong ,  Mark Edward Johnson ,  Poorya Zaremoodi ,  Gautam Singaraju ,  Ying Xu ,  Vladislav Blinov ,  Yu-Heng Hong

IPC: G06F40/289 ,  G06F40/30 ,  G06N3/08 ,  H04L51/02

Abstract: A method includes receiving a training set of utterances for training a machine-learning model to identify one or more intents for one or more utterances, and augmenting the training set of utterances with out-of-domain (OOD) examples. The augmenting includes: generating a data set of OOD examples, filtering out OOD examples from the data set of OOD examples, determining a difficulty value for each OOD example remaining within the filtered data set of the OOD examples, and generating augmented batches of utterances including utterances from the training set of utterances and utterances from the filtered data set of the OOD based on the difficulty value for each OOD. Thereafter, the machine-learning model is trained using the augmented batches of utterances in accordance with a curriculum training protocol.

公开(公告)号：US20250068627A1

公开(公告)日：2025-02-27

申请号：US18616801

申请日：2024-03-26

Applicant: Oracle International Corporation

Inventor： Cong Duy Vu Hoang ,  Gioacchino Tangari ,  Stephen Andrew McRitchie ,  Nitika Mathur ,  Aashna Devang Kanuga ,  Steve Wai-Chun Siu ,  Dalu Guo ,  Chang Xu ,  Mark Edward Johnson ,  Christopher Mark Broadbent ,  Thanh Long Duong ,  Srinivasa Phani Kumar Gadde ,  Vishal Vishnoi ,  Chandan Basavaraju ,  Kenneth Khiaw Hong Eng

IPC: G06F16/2452 ,  G06F16/2457 ,  G06F16/28

Abstract: Techniques are disclosed herein for transforming natural language conversations into a visual output. In one aspect, a computer-implement method includes generating an input string by concatenating a natural language utterance with a schema representation comprising a set of entities for visualization actions, generating, by a first encoder of a machine learning model, one or more embeddings of the input string, encoding, by a second encoder of the machine learning model, relations between elements in the schema representation and words in the natural language utterance based on the one or more embeddings, generating, by a grammar-based decoder of the machine learning model and based on the encoded relations and the one or more embeddings, an intermediate logical form that represents at least the query, the one or more visualization actions, or the combination thereof, and generating, based on the intermediate logical form, a command for a computing system.

公开(公告)号：US12223276B2

公开(公告)日：2025-02-11

申请号：US18424178

申请日：2024-01-26

Applicant: Oracle International Corporation

Inventor： Vishal Vishnoi ,  Xin Xu ,  Elias Luqman Jalaluddin ,  Srinivasa Phani Kumar Gadde ,  Crystal C. Pan ,  Mark Edward Johnson ,  Thanh Long Duong ,  Balakota Srinivas Vinnakota ,  Manish Parekh

IPC: G06F40/295 ,  G06F40/211 ,  G06F40/35 ,  G06F40/56 ,  G06N5/043

Abstract: Techniques for automatically switching between chatbot skills in the same domain. In one particular aspect, a method is provided that includes receiving an utterance from a user within a chatbot session, where a current skill context is a first skill and a current group context is a first group, inputting the utterance into a candidate skills model for the first group, obtaining, using the candidate skills model, a ranking of skills within the first group, determining, based on the ranking of skills, a second skill is a highest ranked skill, changing the current skill context of the chatbot session to the second skill, inputting the utterance into a candidate flows model for the second skill, obtaining, using the candidate flows model, a ranking of intents within the second skill that match the utterance, and determining, based on the ranking of intents, an intent that is a highest ranked intent.

公开(公告)号：US12210830B2

公开(公告)日：2025-01-28

申请号：US17750240

申请日：2022-05-20

Applicant: Oracle International Corporation

Inventor： Thanh Tien Vu ,  Tuyen Quang Pham ,  Mark Edward Johnson ,  Thanh Long Duong ,  Ying Xu ,  Poorya Zaremoodi ,  Omid Mohamad Nezami ,  Budhaditya Saha ,  Cong Duy Vu Hoang

IPC: G06F40/30 ,  G06F40/169 ,  G06F40/284 ,  G06F40/295

Abstract: In some aspects, a computing device may receive, at a data processing system, a set of utterances for training or inferencing with a named entity recognizer to assign a label to each token piece from the set of utterances. The computing device may determine a length of each utterance in the set and when the length of the utterance exceeds a pre-determined threshold of token pieces: dividing the utterance into a plurality of overlapping chunks of token pieces; assigning a label together with a confidence score for each token piece in a chunk; determining a final label and an associated confidence score for each chunk of token pieces by merging two confidence scores; determining a final annotated label for the utterance based at least on the merging the two confidence scores; and storing the final annotated label in a memory.

公开(公告)号：US12153885B2

公开(公告)日：2024-11-26

申请号：US17580535

申请日：2022-01-20

Applicant: Oracle International Corporation

Inventor： Thanh Long Duong ,  Vishal Vishnoi ,  Mark Edward Johnson ,  Elias Luqman Jalaluddin ,  Tuyen Quang Pham ,  Cong Duy Vu Hoang ,  Poorya Zaremoodi ,  Srinivasa Phani Kumar Gadde ,  Aashna Devang Kanuga ,  Zikai Li ,  Yuanxu Wu

IPC: G06F40/289 ,  G06F40/166 ,  G06F40/205 ,  G06F40/263 ,  G06F40/279 ,  G06F40/295 ,  G06N3/08 ,  H04L51/02

Abstract: Techniques are disclosed for systems including techniques for multi-feature balancing for natural langue processors. In an embodiment, a method includes receiving a natural language query to be processed by a machine learning model, the machine learning model utilizing a dataset of natural language phrases for processing natural language queries, determining, based on the machine learning model and the natural language query, a feature dropout value, generating, and based on the natural language query, one or more contextual features and one or more expressional features that may be input to the machine learning model, modifying at least one or the one or more contextual features and the one or more expressional features based on the feature dropout value to generate a set of input features for the machine learning model, and processing the set of input features to cause generating an output dataset for corresponding to the natural language query.

公开(公告)号：US20240256777A1

公开(公告)日：2024-08-01

申请号：US18630772

申请日：2024-04-09

Applicant: Oracle International Corporation

Inventor： Elias Luqman Jalaluddin ,  Vishal Vishnoi ,  Thanh Long Duong ,  Mark Edward Johnson ,  Poorya Zaremoodi ,  Gautam Singaraju ,  Ying Xu ,  Vladislav Blinov ,  Yu-Heng Hong

IPC: G06F40/289 ,  G06F40/30 ,  G06N3/08 ,  H04L51/02

CPC classification number: G06F40/289 ,  G06F40/30 ,  G06N3/08 ,  H04L51/02

Abstract: A method includes receiving a training set of utterances for training a machine-learning model to identify one or more intents for one or more utterances, and augmenting the training set of utterances with out-of-domain (OOD) examples. The augmenting includes: generating a data set of OOD examples, filtering out OOD examples from the data set of OOD examples, determining a difficulty value for each OOD example remaining within the filtered data set of the OOD examples, and generating augmented batches of utterances including utterances from the training set of utterances and utterances from the filtered data set of the OOD based on the difficulty value for each OOD. Thereafter, the machine-learning model is trained using the augmented batches of utterances in accordance with a curriculum training protocol.

公开(公告)号：US20240232541A1

公开(公告)日：2024-07-11

申请号：US18611039

申请日：2024-03-20

Applicant: Oracle International Corporation

Inventor： Ying Xu ,  Poorya Zaremoodi ,  Thanh Tien Vu ,  Cong Duy Vu Hoang ,  Vladislav Blinov ,  Yu-Heng Hong ,  Yakupitiyage Don Thanuja Samodhye Dharmasiri ,  Vishal Vishnoi ,  Elias Luqman Jalaluddin ,  Manish Parekh ,  Thanh Long Duong ,  Mark Edward Johnson

IPC: G06F40/35 ,  G06F40/205 ,  G06F40/253 ,  G06N3/08 ,  H04L51/02

CPC classification number: G06F40/35 ,  G06N3/08 ,  H04L51/02 ,  G06F40/205 ,  G06F40/253

Abstract: Techniques for using enhanced logit values for classifying utterances and messages input to chatbot systems in natural language processing. A method can include a chatbot system receiving an utterance generated by a user interacting with the chatbot system and inputting the utterance into a machine-learning model including a series of network layers. A final network layer of the series of network layers can include a logit function. The machine-learning model can map a first probability for a resolvable class to a first logit value using the logit function. The machine-learning model can map a second probability for a unresolvable class to an enhanced logit value. The method can also include the chatbot system classifying the utterance as the resolvable class or the unresolvable class based on the first logit value and the enhanced logit value.

公开(公告)号：US12014146B2

公开(公告)日：2024-06-18

申请号：US18364298

申请日：2023-08-02

Applicant: Oracle International Corporation

Inventor： Thanh Long Duong ,  Mark Edward Johnson ,  Vishal Vishnoi ,  Crystal C. Pan ,  Vladislav Blinov ,  Cong Duy Vu Hoang ,  Elias Luqman Jalaluddin ,  Duy Vu ,  Balakota Srinivas Vinnakota

IPC: G06F40/30 ,  G06F40/205 ,  G06F40/289 ,  G06N20/00 ,  H04L51/02

CPC classification number: G06F40/30 ,  G06F40/289 ,  G06N20/00 ,  H04L51/02 ,  G06F40/205

Abstract: The present disclosure relates to techniques for identifying out-of-domain utterances. One particular technique includes receiving an utterance and a target domain of a chatbot, generating a sentence embedding for the utterance, obtaining an embedding representation for each cluster of in-domain utterances associated with the target domain, predicting, using a metric learning model, a first probability that the utterance belongs to the target domain based on a similarity or difference between the sentence embedding and each embedding representation for each cluster, predicting, using an outlier detection model, a second probability that the utterance belongs to the target domain based on a determined distance or density deviation between the sentence embedding and embedding representations for neighboring clusters, evaluating the first probability and the second probability to determine a final probability, and classifying the utterance as in-domain or out-of-domain for the chatbot based on the final probability.

公开(公告)号：US11972755B2

公开(公告)日：2024-04-30

申请号：US17993130

申请日：2022-11-23

Applicant: Oracle International Corporation

Inventor： Elias Luqman Jalaluddin ,  Vishal Vishnoi ,  Mark Edward Johnson ,  Thanh Long Duong ,  Yu-Heng Hong ,  Balakota Srinivas Vinnakota

IPC: G10L15/22 ,  G10L15/05 ,  G10L15/06 ,  G10L15/18 ,  G10L15/26

CPC classification number: G10L15/063 ,  G10L15/05 ,  G10L15/18 ,  G10L15/22 ,  G10L15/26 ,  G10L2015/0633 ,  G10L2015/0638 ,  G10L2015/227

Abstract: Techniques for noise data augmentation for training chatbot systems in natural language processing. In one particular aspect, a method is provided that includes receiving a training set of utterances for training an intent classifier to identify one or more intents for one or more utterances; augmenting the training set of utterances with noise text to generate an augmented training set of utterances; and training the intent classifier using the augmented training set of utterances. The augmenting includes: obtaining the noise text from a list of words, a text corpus, a publication, a dictionary, or any combination thereof irrelevant of original text within the utterances of the training set of utterances, and incorporating the noise text within the utterances relative to the original text in the utterances of the training set of utterances at a predefined augmentation ratio to generate augmented utterances.