公开(公告)号：US20250094138A1

公开(公告)日：2025-03-20

申请号：US18743866

申请日：2024-06-14

Applicant: ORACLE INTERNATIONAL CORPORATION

Inventor： Yazhe HU ,  Mengqing GUO ,  Zheng WANG ,  Tao SHENG ,  Jun QIAN ,  Vinod MAMTANI

IPC: G06F8/35 ,  G06N3/0895

Abstract: Systems, methods, and other embodiments associated with automated fine-tuning of software code generation by large language models are described herein. In one embodiment, a method accesses a collection of software code samples that intermix sample code and human language description. The method generates prompts to an LLM to write code that performs as described by the human language description of the sample code. The method fine-tunes a large language model to generate software code based on a code generation loss function that evaluates code generated by the LLM in response to the prompts. The method generates an evaluation score for performance of the tuned large language model as a code generator based on code generation loss for second generated code. And, the method automatically signals that fine-tuning of the tuned large language is complete in response to the evaluation score satisfying a threshold.

公开(公告)号：US20250097171A1

公开(公告)日：2025-03-20

申请号：US18768363

申请日：2024-07-10

Applicant: ORACLE INTERNATIONAL CORPORATION

Inventor： Yazhe HU ,  Mengqing GUO ,  Zheng WANG ,  Tao SHENG ,  Jun QIAN ,  Vinod MAMTANI

IPC: H04L51/02 ,  H04L51/04

Abstract: Systems, methods, and other embodiments automated fine-tuning of chatbot performance for large language models are described herein. In one embodiment, a method accesses a collection of sample conversations between two entities. An individual sample conversation includes one or more rounds of natural language example prompt by a querent and example response by an agent. The method fine-tunes an LLM to generate responses in natural language based on a chatbot loss function that evaluates first responses generated by the LLM to the example prompts by the querent. The method generates an evaluation score for performance of the tuned LLM as a chatbot based on second responses generated by the tuned LLM to test prompts from a test conversation. And, the method automatically signals that the fine-tuning of the tuned LLM is complete in response to the evaluation score satisfying a threshold.

公开(公告)号：US20250094816A1

公开(公告)日：2025-03-20

申请号：US18650904

申请日：2024-04-30

Applicant: ORACLE INTERNATIONAL CORPORATION

Inventor： Yazhe HU ,  Mengqing GUO ,  Zheng WANG ,  Tao SHENG ,  Jun QIAN ,  Vinod MAMTANI

IPC: G06N3/091

Abstract: Systems, methods, and other embodiments associated with automated fine-tuning of text generation for large language models are described herein. In one embodiment, a method accesses a collection of text samples. The text samples include a natural language text prompt that combines content and instructions. The method extracts the instructions from the text prompt. The method fine-tunes a large language model to generate text in natural language based on a text generation loss function that penalizes non-compliance with the extracted instructions by a generated text response to the text prompt. The method generates an evaluation score for performance of the tuned large language model as a text generator based on a value of the text generation loss function for a second generated text response. And, the method automatically signals that the fine tuning of the tuned large language model is complete in response to the evaluation score satisfying a threshold.

公开(公告)号：US20250094704A1

公开(公告)日：2025-03-20

申请号：US18627860

申请日：2024-04-05

Applicant: ORACLE INTERNATIONAL CORPORATION

Inventor： Yazhe HU ,  Mengqing GUO ,  Zheng WANG ,  Tao SHENG ,  Jun QIAN ,  Vinod MAMTANI

IPC: G06F40/226

Abstract: Systems, methods, and other embodiments associated with automated fine-tuning of text summarization for large language models are described herein. In one embodiment, a method accesses a collection of text samples. The text samples include a body of text and an example summary. The method fine-tunes a large language model (LLM) based on a loss function that compares the example summary and a generated summary generated by the LLM. The example and generated summaries are compared at sentence, paragraph, and/or article levels. The method generates an evaluation score for performance of the tuned LLM as a text summarizer based on a further comparison of a reference summary and a summary generated by the tuned LLM. The method then automatically determines to deploy the tuned LLM to a text summarization task in response to the evaluation score satisfying a threshold.