Self-aware visual-textual co-grounded navigation agent

    公开(公告)号:US11029694B2

    公开(公告)日:2021-06-08

    申请号:US16176955

    申请日:2018-10-31

    Abstract: An agent for navigating a mobile automated system is disclosed herein. The navigation agent receives a navigation instruction and visual information for one or more observed images. The navigation agent is provided or equipped with self-awareness, which provides or supports the following abilities: identifying which direction to go or proceed by determining the part of the instruction that corresponds to the observed images (visual grounding), and identifying which part of the instruction has been completed or ongoing and which part is potentially needed for the next action selection (textual grounding). In some embodiments, the navigation agent applies regularization to ensures that the grounded instruction can correctly be used to estimate the progress made towards the navigation goal (progress monitoring).

    SELF-AWARE VISUAL-TEXTUAL CO-GROUNDED NAVIGATION AGENT

    公开(公告)号:US20210286369A1

    公开(公告)日:2021-09-16

    申请号:US17332756

    申请日:2021-05-27

    Abstract: An agent for navigating a mobile automated system is disclosed herein. The navigation agent receives a navigation instruction and visual information for one or more observed images. The navigation agent is provided or equipped with self-awareness, which provides or supports the following abilities: identifying which direction to go or proceed by determining the part of the instruction that corresponds to the observed images (visual grounding), and identifying which part of the instruction has been completed or ongoing and which part is potentially needed for the next action selection (textual grounding). In some embodiments, the navigation agent applies regularization to ensures that the grounded instruction can correctly be used to estimate the progress made towards the navigation goal (progress monitoring).

Patent Agency Ranking