公开(公告)号：US20190130206A1

公开(公告)日：2019-05-02

申请号：US15882220

申请日：2018-01-29

Applicant: salesforce.com, inc.

Inventor： Alexander Richard Trott ,  Caiming XIONG ,  Richard SOCHER

IPC: G06K9/46 ,  G06F17/30

Abstract: Approaches for interpretable counting for visual question answering include a digital image processor, a language processor, and a counter. The digital image processor identifies objects in an image, maps the identified objects into an embedding space, generates bounding boxes for each of the identified objects, and outputs the embedded objects paired with their bounding boxes. The language processor embeds a question into the embedding space. The scorer determines scores for the identified objects. Each respective score determines how well a corresponding one of the identified objects is responsive to the question. The counter determines a count of the objects in the digital image that are responsive to the question based on the scores. The count and a corresponding bounding box for each object included in the count are output. In some embodiments, the counter determines the count interactively based on interactions between counted and uncounted objects.

公开(公告)号：US11270145B2

公开(公告)日：2022-03-08

申请号：US16781179

申请日：2020-02-04

Applicant: salesforce.com, inc.

Inventor： Alexander Richard Trott ,  Caiming Xiong ,  Richard Socher

IPC: G06K9/00 ,  G06K9/46 ,  G06F16/332 ,  G06N5/04 ,  G06N3/04

Abstract: Approaches for interpretable counting for visual question answering include a digital image processor, a language processor, and a counter. The digital image processor identifies objects in an image, maps the identified objects into an embedding space, generates bounding boxes for each of the identified objects, and outputs the embedded objects paired with their bounding boxes. The language processor embeds a question into the embedding space. The scorer determines scores for the identified objects. Each respective score determines how well a corresponding one of the identified objects is responsive to the question. The counter determines a count of the objects in the digital image that are responsive to the question based on the scores. The count and a corresponding bounding box for each object included in the count are output. In some embodiments, the counter determines the count interactively based on interactions between counted and uncounted objects.

公开(公告)号：US10592767B2

公开(公告)日：2020-03-17

申请号：US15882220

申请日：2018-01-29

Applicant: salesforce.com, inc.

Inventor： Alexander Richard Trott ,  Caiming Xiong ,  Richard Socher

IPC: G06K9/00 ,  G06K9/46 ,  G06F16/332 ,  G06N5/04 ,  G06N3/04

Abstract: Approaches for interpretable counting for visual question answering include a digital image processor, a language processor, and a counter. The digital image processor identifies objects in an image, maps the identified objects into an embedding space, generates bounding boxes for each of the identified objects, and outputs the embedded objects paired with their bounding boxes. The language processor embeds a question into the embedding space. The scorer determines scores for the identified objects. Each respective score determines how well a corresponding one of the identified objects is responsive to the question. The counter determines a count of the objects in the digital image that are responsive to the question based on the scores. The count and a corresponding bounding box for each object included in the count are output. In some embodiments, the counter determines the count interactively based on interactions between counted and uncounted objects.

公开(公告)号：US11620572B2

公开(公告)日：2023-04-04

申请号：US16545279

申请日：2019-08-20

Applicant: salesforce.com, inc.

Inventor： Alexander Richard Trott ,  Stephan Tao Zheng

IPC: G06Q30/00 ,  G06N20/00 ,  G06K9/62 ,  G06F17/18

Abstract: Approaches for using self-balancing shaped rewards include randomly selecting a start and goal state, traversing first and second trajectories for moving from the start state toward the goal state where a first terminal state of the first trajectory is closer to the goal state than a second terminal state of the second trajectory, updating rewards for the first and trajectories using a self-balancing reward function based the terminal states of the other trajectory, determining a gradient for the goal-oriented task module, and updating one or more parameters of the goal-oriented task module based on the gradient. The second trajectory contributes to the determination of the gradient and the first trajectory contributes to the determination of the gradient when the first terminal state is within a first threshold distance of the second terminal state or the first terminal state is within a second threshold distance of the goal state.

公开(公告)号：US11562251B2

公开(公告)日：2023-01-24

申请号：US16533575

申请日：2019-08-06

Applicant: salesforce.com, inc.

Inventor： Wenling Shang ,  Alexander Richard Trott ,  Stephan Tao Zheng

IPC: G06N3/08 ,  G05D1/02 ,  G06N3/04

Abstract: Systems and methods are provided for learning world graphs to accelerate hierarchical reinforcement learning (HRL) for the training of a machine learning system. The systems and methods employ or implement a two-stage framework or approach that includes (1) unsupervised world graph discovery, and (2) accelerated hierarchical reinforcement learning by integrating the graph.