公开(公告)号：US11106902B2

公开(公告)日：2021-08-31

申请号：US15920027

申请日：2018-03-13

Applicant: Adobe Inc.

Inventor： Zimo Li ,  Vladimir Kim ,  Mehmet Ersin Yumer

IPC: G06K9/00 ,  G06K9/46 ,  G06N5/04 ,  G06T7/70

Abstract: Certain embodiments detect human-object interactions in image content. For example, human-object interaction metadata is applied to an input image, thereby identifying contact between a part of a depicted human and a part of a depicted object. Applying the human-object interaction metadata involves computing a joint-location heat map by applying a pose estimation subnet to the input image and a contact-point heat map by applying an object contact subnet to the to the input image. The human-object interaction metadata is generated by applying an interaction-detection subnet to the joint-location heat map and the contact-point heat map. The interaction-detection subnet is trained to identify an interaction based on joint-object contact pairs, where a joint-object contact pair includes a relationship between a human joint location and a contact point. An image search system or other computing system is provided with access to the input image having the human-object interaction metadata.

公开(公告)号：US20190286892A1

公开(公告)日：2019-09-19

申请号：US15920027

申请日：2018-03-13

Applicant: Adobe Inc.

Inventor： Zimo Li ,  Vladimir Kim ,  Mehmet Ersin Yumer

IPC: G06K9/00 ,  G06K9/46 ,  G06T7/70 ,  G06N5/04

Abstract: Certain embodiments detect human-object interactions in image content. For example, human-object interaction metadata is applied to an input image, thereby identifying contact between a part of a depicted human and a part of a depicted object. Applying the human-object interaction metadata involves computing a joint-location heat map by applying a pose estimation subnet to the input image and a contact-point heat map by applying an object contact subnet to the to the input image. The human-object interaction metadata is generated by applying an interaction-detection subnet to the joint-location heat map and the contact-point heat map. The interaction-detection subnet is trained to identify an interaction based on joint-object contact pairs, where a joint-object contact pair includes a relationship between a human joint location and a contact point. An image search system or other computing system is provided with access to the input image having the human-object interaction metadata.