公开(公告)号：US20230237352A1

公开(公告)日：2023-07-27

申请号：US17581782

申请日：2022-01-21

Applicant: salesforce.com, inc.

Inventor： Tian Lan ,  Stephan Tao Zheng ,  Sunil Srinivasa

IPC: G06N5/04 ,  G06F9/54 ,  G06N20/00 ,  G06F9/50

CPC classification number: G06N5/043 ,  G06F9/545 ,  G06N20/00 ,  G06F9/5072

Abstract: Embodiments provide a fast multi-agent reinforcement learning (RL) pipeline that runs the full RL workflow end-to-end on a single GPU, using a single store of data for simulation roll-outs, inference, and training. Specifically, simulations and agents in each simulation are run in tandem, taking advantage of the parallel capabilities of the GPU. This way, the costly GPU-CPU communication and copying is significantly reduced, and simulation sampling and learning rates are in turn improved. In this way, a large number of simulations may be concurrently run on the GPU, thus largely improving efficiency of the RL training.

公开(公告)号：US11620572B2

公开(公告)日：2023-04-04

申请号：US16545279

申请日：2019-08-20

Applicant: salesforce.com, inc.

Inventor： Alexander Richard Trott ,  Stephan Tao Zheng

IPC: G06Q30/00 ,  G06N20/00 ,  G06K9/62 ,  G06F17/18

Abstract: Approaches for using self-balancing shaped rewards include randomly selecting a start and goal state, traversing first and second trajectories for moving from the start state toward the goal state where a first terminal state of the first trajectory is closer to the goal state than a second terminal state of the second trajectory, updating rewards for the first and trajectories using a self-balancing reward function based the terminal states of the other trajectory, determining a gradient for the goal-oriented task module, and updating one or more parameters of the goal-oriented task module based on the gradient. The second trajectory contributes to the determination of the gradient and the first trajectory contributes to the determination of the gradient when the first terminal state is within a first threshold distance of the second terminal state or the first terminal state is within a second threshold distance of the goal state.

公开(公告)号：US11562251B2

公开(公告)日：2023-01-24

申请号：US16533575

申请日：2019-08-06

Applicant: salesforce.com, inc.

Inventor： Wenling Shang ,  Alexander Richard Trott ,  Stephan Tao Zheng

IPC: G06N3/08 ,  G05D1/02 ,  G06N3/04

Abstract: Systems and methods are provided for learning world graphs to accelerate hierarchical reinforcement learning (HRL) for the training of a machine learning system. The systems and methods employ or implement a two-stage framework or approach that includes (1) unsupervised world graph discovery, and (2) accelerated hierarchical reinforcement learning by integrating the graph.