公开(公告)号：US11263276B1

公开(公告)日：2022-03-01

申请号：US16176312

申请日：2018-10-31

Applicant: Meta Platforms, Inc.

Inventor： Emmanouil Koukoumidis ,  Michael Robert Hanson ,  Rajen Subba ,  Heidi Young ,  Rushin Shah ,  Jinsong Yu ,  Benoit F. Dumoulin ,  Jeremy Gillmor Kahn ,  Chandrasekhar Iyer

IPC: G06F16/00 ,  G06F16/951 ,  G06N20/00 ,  G06F40/40

Abstract: In one embodiment, a method includes receiving a user query from a client system associated with a first user, determining a plurality of dialog-intents, each being associated with a particular agent of a plurality of agents, executing a plurality of tasks corresponding to the user query via the plurality of agents corresponding to the plurality of dialog-intents, receiving a plurality of execution results corresponding to the plurality of tasks from the plurality of agents, respectively, selecting two or more of the plurality of execution results for combination, generating a multi-perspective response based on the selected execution results by a stitching model, wherein the multi-perspective response comprises a natural-language response combining the selected execution results, and sending instructions for presenting the multi-perspective response to the first user to the client system in response to the user query.

公开(公告)号：US12131523B2

公开(公告)日：2024-10-29

申请号：US17182951

申请日：2021-02-23

Applicant: Meta Platforms, Inc.

Inventor： Xiaohu Liu ,  Baiyang Liu ,  Rajen Subba

IPC: G06V10/82 ,  G06F3/01 ,  G06F3/16 ,  G06F7/14 ,  G06F9/451 ,  G06F16/176 ,  G06F16/22 ,  G06F16/23 ,  G06F16/242 ,  G06F16/2455 ,  G06F16/2457 ,  G06F16/248 ,  G06F16/33 ,  G06F16/332 ,  G06F16/338 ,  G06F16/903 ,  G06F16/9032 ,  G06F16/9038 ,  G06F16/904 ,  G06F16/951 ,  G06F16/9535 ,  G06F18/2411 ,  G06F40/205 ,  G06F40/295 ,  G06F40/30 ,  G06F40/40 ,  G06N3/006 ,  G06N3/08 ,  G06N7/01 ,  G06N20/00 ,  G06Q50/00 ,  G06V10/764 ,  G06V20/10 ,  G06V40/20 ,  G10L15/02 ,  G10L15/06 ,  G10L15/07 ,  G10L15/16 ,  G10L15/18 ,  G10L15/183 ,  G10L15/187 ,  G10L15/22 ,  G10L15/26 ,  G10L17/06 ,  G10L17/22 ,  H04L5/02 ,  H04L12/28 ,  H04L41/00 ,  H04L41/22 ,  H04L43/0882 ,  H04L43/0894 ,  H04L51/02 ,  H04L51/18 ,  H04L51/216 ,  H04L51/52 ,  H04L67/306 ,  H04L67/50 ,  H04L67/5651 ,  H04L67/75 ,  H04W12/08 ,  G10L13/00 ,  G10L13/04 ,  H04L51/046 ,  H04L67/10 ,  H04L67/53

CPC classification number: G06V10/82 ,  G06F3/011 ,  G06F3/013 ,  G06F3/017 ,  G06F3/167 ,  G06F7/14 ,  G06F9/453 ,  G06F16/176 ,  G06F16/2255 ,  G06F16/2365 ,  G06F16/243 ,  G06F16/24552 ,  G06F16/24575 ,  G06F16/24578 ,  G06F16/248 ,  G06F16/3323 ,  G06F16/3329 ,  G06F16/3344 ,  G06F16/338 ,  G06F16/90332 ,  G06F16/90335 ,  G06F16/9038 ,  G06F16/904 ,  G06F16/951 ,  G06F16/9535 ,  G06F18/2411 ,  G06F40/205 ,  G06F40/295 ,  G06F40/30 ,  G06F40/40 ,  G06N3/006 ,  G06N3/08 ,  G06N7/01 ,  G06N20/00 ,  G06Q50/01 ,  G06V10/764 ,  G06V20/10 ,  G06V40/28 ,  G10L15/02 ,  G10L15/063 ,  G10L15/07 ,  G10L15/16 ,  G10L15/1815 ,  G10L15/1822 ,  G10L15/183 ,  G10L15/187 ,  G10L15/22 ,  G10L15/26 ,  G10L17/06 ,  G10L17/22 ,  H04L5/02 ,  H04L12/2816 ,  H04L41/20 ,  H04L41/22 ,  H04L43/0882 ,  H04L43/0894 ,  H04L51/02 ,  H04L51/18 ,  H04L51/216 ,  H04L51/52 ,  H04L67/306 ,  H04L67/535 ,  H04L67/5651 ,  H04L67/75 ,  H04W12/08 ,  G06F2216/13 ,  G10L13/00 ,  G10L13/04 ,  G10L2015/223 ,  G10L2015/225 ,  H04L51/046 ,  H04L67/10 ,  H04L67/53

Abstract: In one embodiment, a method includes by a client system associated with a user, receiving, at the client system, a user input from the user, parsing, by the client system, the first user input to identify a request to execute a function to be performed by an assistant system of several assistant systems associated with the client system, determining whether the user is authorized to access the assistant system by comparing a voiceprint of the user to several voiceprints stored on the client system, sending, from the client system to the assistant system in response to determining the user is authorized to access the assistant system, a request to set an assistant xbot of the assistant system into a listening mode, and receiving, at the client system from the assistant system, an indication that the assistant xbot is in listening mode.

公开(公告)号：US11704745B2

公开(公告)日：2023-07-18

申请号：US17006339

申请日：2020-08-28

Applicant: Meta Platforms, Inc.

Inventor： Shivani Poddar ,  Seungwhan Moon ,  Paul Anthony Crook ,  Rajen Subba

IPC: G06F40/30 ,  G06F3/01 ,  G06F9/451 ,  G06F9/48 ,  G06F9/54 ,  G06F16/332 ,  G06F16/9032 ,  G06F16/9536 ,  G06F40/205 ,  G06F40/253 ,  G06Q50/00 ,  H04N7/14 ,  G06N20/00 ,  G06F40/35 ,  G06F40/56 ,  G06F40/242 ,  G06V20/20 ,  G06V10/82 ,  G06V40/16 ,  G06V20/30 ,  G06V10/20 ,  G06V10/764 ,  G06V20/00 ,  G06V40/20 ,  H04L51/222 ,  H04L51/224 ,  H04L51/52 ,  H04L51/212 ,  H04L67/75 ,  G06N3/047 ,  G06N3/045 ,  G06F18/2321 ,  G06N3/08 ,  G06Q10/109 ,  G10L15/06 ,  G10L15/08 ,  G10L15/16 ,  G10L15/18 ,  G10L15/22 ,  G10L15/30 ,  G10L15/32 ,  H04L51/18 ,  H04L67/306 ,  G06V20/40 ,  G06F3/16

CPC classification number: G06Q50/01 ,  G06F3/011 ,  G06F3/013 ,  G06F9/453 ,  G06F9/485 ,  G06F9/4881 ,  G06F9/547 ,  G06F16/3329 ,  G06F16/90332 ,  G06F16/9536 ,  G06F18/2321 ,  G06F40/205 ,  G06F40/242 ,  G06F40/253 ,  G06F40/30 ,  G06F40/35 ,  G06F40/56 ,  G06N3/045 ,  G06N3/047 ,  G06N3/08 ,  G06N20/00 ,  G06Q10/109 ,  G06V10/255 ,  G06V10/764 ,  G06V10/82 ,  G06V20/00 ,  G06V20/20 ,  G06V20/30 ,  G06V40/16 ,  G06V40/25 ,  G10L15/063 ,  G10L15/08 ,  G10L15/16 ,  G10L15/1815 ,  G10L15/1822 ,  G10L15/22 ,  G10L15/30 ,  G10L15/32 ,  H04L51/18 ,  H04L51/212 ,  H04L51/222 ,  H04L51/224 ,  H04L51/52 ,  H04L67/306 ,  H04L67/75 ,  H04N7/147 ,  G06F3/017 ,  G06F3/167 ,  G06V20/41 ,  G06V40/174 ,  G06V2201/10 ,  G10L2015/088 ,  G10L2015/223 ,  G10L2015/227

Abstract: In one embodiment, a method includes receiving, from a client system associated with a user, a user request comprising a reference to a target object, accessing visual data from the client system, wherein the visual data comprises images portraying the target object and one or more additional objects, and wherein attribute information of the target object is recorded in a multimodal dialog state, resolving the reference to the target object based on the attribute information recorded in the multimodal dialog state, determining relational information between the target object and one or more of the additional objects portrayed in the visual data, and sending, to the client system, instructions for presenting a response to the user request, wherein the response comprises the attribute information and the determined relational information.

公开(公告)号：US20230186618A1

公开(公告)日：2023-06-15

申请号：US18164480

申请日：2023-02-03

Applicant: Meta Platforms, Inc.

Inventor： Emmanouil Koukoumidis ,  Michael Robert Hanson ,  Rajen Subba ,  Heidi Young ,  Rushin Shah ,  Jinsong Yu ,  Benoit F. Dumoulin ,  Jeremy Gillmor Kahn ,  Chandrasekhar Iyer

IPC: G06V10/82 ,  G06F9/451 ,  G10L15/18 ,  G10L15/183 ,  G10L15/22 ,  G06F16/338 ,  G06F16/332 ,  G06F16/33 ,  G06N20/00 ,  G06F16/9535 ,  G06Q50/00 ,  H04L67/306 ,  G06F16/176 ,  G10L15/06 ,  G10L15/16 ,  G06F3/01 ,  G06F16/9032 ,  G06F16/2457 ,  H04L51/02 ,  G06F3/16 ,  G06N3/08 ,  G10L15/26 ,  G06F16/9038 ,  G06F16/904 ,  G06F40/30 ,  G06F40/40 ,  G06F16/22 ,  G06F16/23 ,  G06F7/14 ,  H04L43/0882 ,  H04L43/0894 ,  H04L12/28 ,  H04L41/00 ,  H04L41/22 ,  H04W12/08 ,  G10L15/07 ,  G10L17/22 ,  G06N3/006 ,  G10L17/06 ,  G06F16/248 ,  G06F16/951 ,  G06F16/242 ,  G06F16/2455 ,  G10L15/02 ,  G10L15/187 ,  G06V20/10 ,  G06V40/20 ,  G06F40/295 ,  H04L51/52 ,  H04L51/216 ,  H04L67/50 ,  H04L67/5651 ,  H04L51/18 ,  H04L67/75 ,  G06F16/903 ,  G06F40/205 ,  G06F18/2411 ,  G06N7/01 ,  G06V10/764

CPC classification number: G06V10/82 ,  G06F3/011 ,  G06F3/013 ,  G06F3/017 ,  G06F3/167 ,  G06F7/14 ,  G06F9/453 ,  G06F16/176 ,  G06F16/243 ,  G06F16/248 ,  G06F16/338 ,  G06F16/904 ,  G06F16/951 ,  G06F16/2255 ,  G06F16/2365 ,  G06F16/3323 ,  G06F16/3329 ,  G06F16/3344 ,  G06F16/9038 ,  G06F16/9535 ,  G06F16/24552 ,  G06F16/24575 ,  G06F16/24578 ,  G06F16/90332 ,  G06F16/90335 ,  G06F18/2411 ,  G06F40/30 ,  G06F40/40 ,  G06F40/205 ,  G06F40/295 ,  G06N3/006 ,  G06N3/08 ,  G06N7/01 ,  G06N20/00 ,  G06Q50/01 ,  G06V10/764 ,  G06V20/10 ,  G06V40/28 ,  G10L15/02 ,  G10L15/07 ,  G10L15/16 ,  G10L15/22 ,  G10L15/26 ,  G10L15/063 ,  G10L15/183 ,  G10L15/187 ,  G10L15/1815 ,  G10L15/1822 ,  G10L17/06 ,  G10L17/22 ,  H04L12/2816 ,  H04L41/20 ,  H04L41/22 ,  H04L43/0882 ,  H04L43/0894 ,  H04L51/02 ,  H04L51/18 ,  H04L51/52 ,  H04L51/216 ,  H04L67/75 ,  H04L67/306 ,  H04L67/535 ,  H04L67/5651 ,  H04W12/08 ,  H04L67/10

Abstract: In one embodiment, a method includes receiving a user query inputted on a head-mounted device from the head-mounted device, wherein the user query corresponds to multiple dialog-intents, executing multiple tasks corresponding to the multiple dialog-intents, generating a multi-perspective response by a stitching model based on two or more of execution results of the multiple tasks, wherein the stitching model combines the two or more of the execution results based on natural language processing, and wherein the multi-perspective response comprises a natural-language response combining the two or more execution results, and sending instructions to the head-mounted device for presenting the multi-perspective response on the head-mounted device.

公开(公告)号：US20220382989A1

公开(公告)日：2022-12-01

申请号：US17878778

申请日：2022-08-01

Applicant: Meta Platforms, Inc.

Inventor： Shivani Poddar ,  Seungwhan Moon ,  Paul Anthony Crook ,  Rajen Subba

IPC: G06F40/30 ,  G06F9/54 ,  G06F40/205 ,  G06F40/242 ,  G06N3/04 ,  G06N3/08 ,  G06F16/9536 ,  G10L15/18 ,  G10L15/22 ,  G10L15/30 ,  G10L15/32 ,  G06F40/253 ,  G06N20/00 ,  G06F3/01 ,  G06Q50/00 ,  G06F16/9032 ,  G06F9/48 ,  G10L15/08 ,  H04N7/14 ,  H04L67/306 ,  G06V10/20 ,  G06V20/20 ,  G06V20/30 ,  G06V20/40 ,  G06V40/16 ,  H04L51/52 ,  H04L51/212 ,  H04L67/75 ,  G06F9/451 ,  G06F16/332 ,  G06F40/35 ,  G06K9/62 ,  G10L15/06 ,  G10L15/16

Abstract: In one embodiment, a method includes receiving, at a client system, an audio input, where the audio input comprises a coreference to a target object, accessing visual data from one or more camera associated with the client system, where the visual data comprises images portraying one or more objects, resolving the coreference to the target object from among the one or more objects, resoling the target object to a specific entity, and providing, at the client system, a response to the audio input, where the response comprises information about the specific entity.

公开(公告)号：US11842522B2

公开(公告)日：2023-12-12

申请号：US18164480

申请日：2023-02-03

Applicant: Meta Platforms, Inc.

Inventor： Emmanouil Koukoumidis ,  Michael Robert Hanson ,  Rajen Subba ,  Heidi Young ,  Rushin Shah ,  Jinsong Yu ,  Benoit F. Dumoulin ,  Jeremy Gillmor Kahn ,  Chandrasekhar Iyer

IPC: G06F16/00 ,  G06V10/82 ,  G06F9/451 ,  G10L15/18 ,  G10L15/183 ,  G10L15/22 ,  G06F16/338 ,  G06F16/332 ,  G06F16/33 ,  G06N20/00 ,  G06F16/9535 ,  G06Q50/00 ,  H04L67/306 ,  G06F16/176 ,  G10L15/06 ,  G10L15/16 ,  G06F3/01 ,  G06F16/9032 ,  G06F16/2457 ,  H04L5/02 ,  G06F3/16 ,  G06N3/08 ,  G10L15/26 ,  G06F16/9038 ,  G06F16/904 ,  G06F40/30 ,  G06F40/40 ,  G06F16/22 ,  G06F16/23 ,  G06F7/14 ,  H04L43/0882 ,  H04L43/0894 ,  H04L12/28 ,  H04L41/00 ,  H04L41/22 ,  H04W12/08 ,  G10L15/07 ,  G10L17/22 ,  G06N3/006 ,  G10L17/06 ,  G06F16/248 ,  G06F16/951 ,  G06F16/242 ,  G06F16/2455 ,  G10L15/02 ,  G10L15/187 ,  G06V20/10 ,  G06V40/20 ,  G06F40/295 ,  H04L51/52 ,  H04L51/216 ,  H04L67/50 ,  H04L67/5651 ,  H04L51/18 ,  H04L67/75 ,  G06F16/903 ,  G06F40/205 ,  G06F18/2411 ,  G06N7/01 ,  G06V10/764 ,  H04L67/10 ,  H04L51/046 ,  G10L13/00 ,  G10L13/04 ,  H04L67/53

CPC classification number: G06V10/82 ,  G06F3/011 ,  G06F3/013 ,  G06F3/017 ,  G06F3/167 ,  G06F7/14 ,  G06F9/453 ,  G06F16/176 ,  G06F16/2255 ,  G06F16/2365 ,  G06F16/243 ,  G06F16/248 ,  G06F16/24552 ,  G06F16/24575 ,  G06F16/24578 ,  G06F16/338 ,  G06F16/3323 ,  G06F16/3329 ,  G06F16/3344 ,  G06F16/904 ,  G06F16/9038 ,  G06F16/90332 ,  G06F16/90335 ,  G06F16/951 ,  G06F16/9535 ,  G06F18/2411 ,  G06F40/205 ,  G06F40/295 ,  G06F40/30 ,  G06F40/40 ,  G06N3/006 ,  G06N3/08 ,  G06N7/01 ,  G06N20/00 ,  G06Q50/01 ,  G06V10/764 ,  G06V20/10 ,  G06V40/28 ,  G10L15/02 ,  G10L15/063 ,  G10L15/07 ,  G10L15/16 ,  G10L15/183 ,  G10L15/187 ,  G10L15/1815 ,  G10L15/1822 ,  G10L15/22 ,  G10L15/26 ,  G10L17/06 ,  G10L17/22 ,  H04L5/02 ,  H04L12/2816 ,  H04L41/20 ,  H04L41/22 ,  H04L43/0882 ,  H04L43/0894 ,  H04L51/18 ,  H04L51/216 ,  H04L51/52 ,  H04L67/306 ,  H04L67/535 ,  H04L67/5651 ,  H04L67/75 ,  H04W12/08 ,  G06F2216/13 ,  G10L13/00 ,  G10L13/04 ,  G10L2015/223 ,  G10L2015/225 ,  H04L51/046 ,  H04L67/10 ,  H04L67/53

Abstract: In one embodiment, a method includes receiving a user query inputted on a head-mounted device from the head-mounted device, wherein the user query corresponds to multiple dialog-intents, executing multiple tasks corresponding to the multiple dialog-intents, generating a multi-perspective response by a stitching model based on two or more of execution results of the multiple tasks, wherein the stitching model combines the two or more of the execution results based on natural language processing, and wherein the multi-perspective response comprises a natural-language response combining the two or more execution results, and sending instructions to the head-mounted device for presenting the multi-perspective response on the head-mounted device.

公开(公告)号：US11721093B2

公开(公告)日：2023-08-08

申请号：US17203286

申请日：2021-03-16

Applicant: Meta Platforms, Inc.

Inventor： Xiaohu Liu ,  Baiyang Liu ,  Rajen Subba ,  Benoit F. Dumoulin

IPC: G06F16/951 ,  G06F16/242 ,  G06F16/2455 ,  G10L15/02 ,  G10L15/187 ,  G06V20/10 ,  G06V40/20 ,  G06F40/295 ,  H04L51/52 ,  H04L51/216 ,  H04L67/50 ,  H04L67/5651 ,  H04L51/18 ,  H04L67/75 ,  G06F16/903 ,  G06F40/205 ,  G06F18/2411 ,  G06N7/01 ,  G06V10/764 ,  G06V10/82 ,  G06F9/451 ,  G10L15/18 ,  G10L15/183 ,  G10L15/22 ,  G06F16/338 ,  G06F16/332 ,  G06F16/33 ,  G06N20/00 ,  G06F16/9535 ,  G06Q50/00 ,  H04L67/306 ,  G06F16/176 ,  G10L15/06 ,  G10L15/16 ,  G06F3/01 ,  G06F16/9032 ,  G06F16/2457 ,  H04L51/02 ,  G06F3/16 ,  G06N3/08 ,  G10L15/26 ,  G06F16/9038 ,  G06F16/904 ,  G06F40/30 ,  G06F40/40 ,  G06F16/22 ,  G06F16/23 ,  G06F7/14 ,  H04L43/0882 ,  H04L43/0894 ,  H04L12/28 ,  H04L41/00 ,  H04L41/22 ,  H04W12/08 ,  G10L15/07 ,  G10L17/22 ,  G06N3/006 ,  G10L17/06 ,  G06F16/248 ,  H04L67/10 ,  H04L51/046 ,  G10L13/00 ,  G10L13/04 ,  H04L67/53

CPC classification number: G06V10/82 ,  G06F3/011 ,  G06F3/013 ,  G06F3/017 ,  G06F3/167 ,  G06F7/14 ,  G06F9/453 ,  G06F16/176 ,  G06F16/2255 ,  G06F16/2365 ,  G06F16/243 ,  G06F16/248 ,  G06F16/24552 ,  G06F16/24575 ,  G06F16/24578 ,  G06F16/338 ,  G06F16/3323 ,  G06F16/3329 ,  G06F16/3344 ,  G06F16/904 ,  G06F16/9038 ,  G06F16/90332 ,  G06F16/90335 ,  G06F16/951 ,  G06F16/9535 ,  G06F18/2411 ,  G06F40/205 ,  G06F40/295 ,  G06F40/30 ,  G06F40/40 ,  G06N3/006 ,  G06N3/08 ,  G06N7/01 ,  G06N20/00 ,  G06Q50/01 ,  G06V10/764 ,  G06V20/10 ,  G06V40/28 ,  G10L15/02 ,  G10L15/063 ,  G10L15/07 ,  G10L15/16 ,  G10L15/183 ,  G10L15/187 ,  G10L15/1815 ,  G10L15/1822 ,  G10L15/22 ,  G10L15/26 ,  G10L17/06 ,  G10L17/22 ,  H04L12/2816 ,  H04L41/20 ,  H04L41/22 ,  H04L43/0882 ,  H04L43/0894 ,  H04L51/02 ,  H04L51/18 ,  H04L51/216 ,  H04L51/52 ,  H04L67/306 ,  H04L67/535 ,  H04L67/5651 ,  H04L67/75 ,  H04W12/08 ,  G06F2216/13 ,  G10L13/00 ,  G10L13/04 ,  G10L2015/223 ,  G10L2015/225 ,  H04L51/046 ,  H04L67/10 ,  H04L67/53

Abstract: In one embodiment, a method includes, by a client system, receiving, by an assistant xbot of the client system, a request from a first user for a summary of user content from a first content source, retrieving, from the first content source, a plurality of content items corresponding to the request, generating a personalized summary of the retrieved content items, wherein the personalization of the summary is based on a user profile of the first user, and presenting, by the assistant xbot, the personalized summary responsive to the request within a separate communication interface between the assistant xbot and the first user, wherein the personalized summary is interactable by the first user to react to one or more of the plurality of content items.

公开(公告)号：US11663678B2

公开(公告)日：2023-05-30

申请号：US17006339

申请日：2020-08-28

Applicant: Meta Platforms, Inc.

Inventor： Shivani Poddar ,  Seungwhan Moon ,  Paul Anthony Crook ,  Rajen Subba

IPC: G06F40/30 ,  G06F3/01 ,  G06F9/451 ,  G06F9/48 ,  G06F9/54 ,  G06F16/332 ,  G06F16/9032 ,  G06F16/9536 ,  G06F40/205 ,  G06F40/253 ,  G06F40/242 ,  G06N3/08 ,  G10L15/18 ,  G10L15/22 ,  G10L15/30 ,  G10L15/32 ,  G06N20/00 ,  G06Q50/00 ,  G10L15/08 ,  H04N7/14 ,  H04L67/306 ,  G06V10/20 ,  G06V20/20 ,  G06V20/30 ,  G06V20/40 ,  G06V40/16 ,  H04L51/52 ,  H04L51/212 ,  H04L67/75 ,  G06F40/35 ,  G10L15/06 ,  G10L15/16 ,  G06F18/2321 ,  G06N3/045 ,  G06N3/047 ,  G06F3/16

CPC classification number: G06F40/30 ,  G06F3/011 ,  G06F3/013 ,  G06F9/453 ,  G06F9/485 ,  G06F9/4881 ,  G06F9/547 ,  G06F16/3329 ,  G06F16/90332 ,  G06F16/9536 ,  G06F18/2321 ,  G06F40/205 ,  G06F40/242 ,  G06F40/253 ,  G06F40/35 ,  G06N3/045 ,  G06N3/047 ,  G06N3/08 ,  G06N20/00 ,  G06Q50/01 ,  G06V10/255 ,  G06V20/20 ,  G06V20/30 ,  G06V20/41 ,  G06V40/174 ,  G10L15/063 ,  G10L15/08 ,  G10L15/16 ,  G10L15/1815 ,  G10L15/1822 ,  G10L15/22 ,  G10L15/30 ,  G10L15/32 ,  H04L51/212 ,  H04L51/52 ,  H04L67/306 ,  H04L67/75 ,  H04N7/147 ,  G06F3/017 ,  G06F3/167 ,  G06V2201/10 ,  G10L2015/088 ,  G10L2015/223 ,  G10L2015/227

Abstract: In one embodiment, a method includes receiving, from a client system associated with a user, a user request comprising a reference to a target object, accessing visual data from the client system, wherein the visual data comprises images portraying the target object and one or more additional objects, and wherein attribute information of the target object is recorded in a multimodal dialog state, resolving the reference to the target object based on the attribute information recorded in the multimodal dialog state, determining relational information between the target object and one or more of the additional objects portrayed in the visual data, and sending, to the client system, instructions for presenting a response to the user request, wherein the response comprises the attribute information and the determined relational information.

公开(公告)号：US11651451B1

公开(公告)日：2023-05-16

申请号：US16921665

申请日：2020-07-06

Applicant: Meta Platforms, Inc.

Inventor： Honglei Liu ,  Hao Zhou ,  Seungwhan Moon ,  Bing Liu ,  Yulong Qiu ,  Daniel Chai ,  Pararth Paresh Shah ,  Xiaolei Li ,  Rajen Subba ,  Hu Xu

IPC: H04L51/52 ,  H04L51/18 ,  G06F16/9032 ,  G06F40/56 ,  G06F9/451

CPC classification number: H04L51/52 ,  G06F9/453 ,  G06F16/90332 ,  G06F40/56 ,  H04L51/18

Abstract: In one embodiment, a method includes receiving a user request from a client system associated with a user, generating a response to the user request which references one or more entities, generating a personalized recommendation based on the user request and the response, wherein the personalized recommendation references one or more of the entities of the response, and sending instructions for presenting the response and the personalized recommendation to the client system.