公开(公告)号：US20250139411A1

公开(公告)日：2025-05-01

申请号：US18498229

申请日：2023-10-31

Applicant: Salesforce, Inc.

Inventor： Rithesh Murthy ,  Shelby Heinecke ,  Juan Carlos Niebles Duque ,  Zhiwei Liu ,  Le Xue ,  Weiran Yao ,  Yihao Feng ,  Zeyuan Chen ,  Akash Gokul ,  Devansh Arpit ,  Ran Xu ,  Lik Mui ,  Huan Wang ,  Caiming Xiong ,  Silvio Savarese

IPC: G06N3/0455 ,  G06N3/084

Abstract: Embodiments described herein provide a large language model (LLM) based AI agent that adopts Monte-Carlo Tree Search (MCTS) to execute a task. The LLM is prompted with a task description and it responds with its first attempted list of actions. Based on the success or failure of the first attempt, the LLM is prompted with an updated prompt which includes feedback from the first attempt based on a determined reward. The prompt may include a relative “score” for each action taken at each step. A numeric score may be mapped to a set of pre-defined text labels, such as “high” or “low” value putting the score in a form more suited for an LLM prompt. In this way, the LLM is iteratively given prompts which are updated with the scores from each action taken at each previous iterations so that it traverses different paths on the tree in each iteration.

公开(公告)号：US20240070744A1

公开(公告)日：2024-02-29

申请号：US17891564

申请日：2022-08-19

Applicant: Salesforce, Inc.

Inventor： Yongjun Chen ,  Zhiwei Liu ,  Jianguo Zhang ,  Huan Wang ,  Caiming Xiong

IPC: G06Q30/06 ,  G06Q30/02 ,  H04L67/50

CPC classification number: G06Q30/0631 ,  G06Q30/0201 ,  H04L67/535

Abstract: Embodiments described herein provide systems and methods for training a sequential recommendation model. Methods include determining a difficulty and quality (DQ) score associated with user behavior sequences from a training dataset. User behavior sequences are sampled during training based on their DQ scores. A meta-extrapolator may also be trained based on user behavior sequences sampled according to DQ score. The meta-extrapolator may be trained with high quality low difficulty sequences. The meta-extrapolator may then be used with an input of high quality high difficulty sequences to generate synthetic user behavior sequences. The synthetic user behavior sequences may be used to augment the training dataset to fine-tune the sequential recommendation model, while continuing to sample user behavior sequences based on DQ score. As the DQ score is based on current model predictions, DQ scores iteratively update during the training process.

公开(公告)号：US20250053793A1

公开(公告)日：2025-02-13

申请号：US18494393

申请日：2023-10-25

Applicant: Salesforce, Inc.

Inventor： Zhiwei Liu ,  Weiran Yao ,  Jianguo Zhang ,  Le Xue ,  Shelby Heinecke ,  Rithesh Murthy ,  Yihao Feng ,  Zeyuan Chen ,  Juan Carlos Niebles Duque ,  Devansh Arpit ,  Ran Xu ,  Lik Mui ,  Huan Wang ,  Caiming Xiong ,  Silvio Savarese

IPC: G06N3/047 ,  G06N3/092

Abstract: Embodiments described herein provide a method of predicting an action by a plurality of language model augmented agents (LAAs). In at least one embodiment, a controller receives a task instruction to be performed using an environment. The controller receives an observation of a first state from the environment. The controller selects a LAA from the plurality of LAAs based on the task instruction and the observation. The controller obtains an output from the selected LAA generated using an input combining the task instruction, the observation, and an LAA-specific prompt template. The controller determines the action based on the output. The controller causes the action to be performed on the environment thereby causing the first state of the environment to change to a second state.

公开(公告)号：US20250053787A1

公开(公告)日：2025-02-13

申请号：US18429119

申请日：2024-01-31

Applicant: Salesforce, Inc.

Inventor： Liangwei Yang ,  Shelby Heinecke ,  Jianguo Zhang ,  Rithesh Murthy ,  Huan Wang ,  Caiming Xiong ,  Zhiwei Liu

IPC: G06N3/0455 ,  G06N3/084 ,  G06Q30/0601

Abstract: Embodiments described herein provide a method for training a recommendation neural network model using multiple data sources. The method may include: receiving, via a data interface, time series data indicating a user-item interaction history; transforming the time series data into a user-item graph; encoding, by a neural network encoder, the user-item graph into user embeddings and item embeddings; generating a plurality of losses according to a plurality of training tasks performed based on the user embeddings and, item embeddings; training the recommendation neural network model by updating the user embeddings and the item embeddings via backpropagation based on a weighted sum of gradients of the plurality of losses; and generating, by a neural network decoder, one or more recommended items for a given user based on the updated user embeddings and the updated item embeddings.

公开(公告)号：US20240062071A1

公开(公告)日：2024-02-22

申请号：US18148735

申请日：2022-12-30

Applicant: Salesforce, Inc.

Inventor： Juntao TAN ,  Shelby Heinecke ,  Zhiwei Liu ,  Yongjun Chen

IPC: G06N3/094 ,  G06N3/084

CPC classification number: G06N3/094 ,  G06N3/084

Abstract: Embodiments described herein provide a cascade-guided adversarial training method for sequential recommendation models. A system may compute cascade scores for items in a user interaction sequence. The cascade scores may be based on the position in the sequence, as well as the appearance of the same item in other sequences. Based on the computed cascade score, the system may perturb item embeddings. The perturbed user interaction sequences with perturbed item embeddings may be used to train a sequential recommendation model.

公开(公告)号：US20240046330A1

公开(公告)日：2024-02-08

申请号：US18182944

申请日：2023-03-13

Applicant: Salesforce, Inc.

Inventor： Ziwei Fan ,  Yongjun Chen ,  Zhiwei Liu ,  Huan Wang

IPC: G06Q30/0601 ,  G06Q30/0201

CPC classification number: G06Q30/0631 ,  G06Q30/0201

Abstract: Embodiments described herein provide a universal item learning framework that generates universal item embeddings for zero-shot items. Specifically, the universal item learning framework performs generic features extraction of items and product knowledge characterization based on a product knowledge graph (PKG) to generate embeddings of input items. A pretrained language model (PLM) may be adopted to extract features from generic item side information, such as titles, descriptions, etc., of an item. A PKG may be constructed to represent recommendation-oriented knowledge, which comprise a plurality of nodes representing items and a plurality of edges connecting nodes represent different relations between items. As those relations in PKG are usually retrieved from user-item interactions, the PKG adapts the universal representation for recommendation with knowledge of user-item interactions.

公开(公告)号：US20250045567A1

公开(公告)日：2025-02-06

申请号：US18498257

申请日：2023-10-31

Applicant: Salesforce, Inc.

Inventor： Weiran Yao ,  Shelby Heinecke ,  Juan Carlos Niebles Duque ,  Zhiwei Liu ,  Yihao Feng ,  Le Xue ,  Rithesh Murthy ,  Zeyuan Chen ,  Jianguo Zhang ,  Devansh Arpit ,  Ran Xu ,  Lik Mui ,  Huan Wang ,  Caiming Xiong ,  Silvio Savarese

IPC: G06N3/0455 ,  G06N3/092

Abstract: Embodiments described herein provide for optimizing a language model (LM) agent. In at least one embodiment, and LM agent comprises an “actor” LM and a “retrospective LM which provides reflections on attempts by the actor LM. The reflections are used to update subsequent prompts to the actor LM. Optimizing the LM agent comprises fine-tuning parameters of the retrospective LM while keeping parameters of the actor LM frozen. A gradient may be determined by a change in reward from the environment based on actions taken by the actor LM with and without a reflection of the retrospective LM. Using this gradient, parameters of the retrospective LM may be updated via backpropagation.