公开(公告)号：US20240362279A1

公开(公告)日：2024-10-31

申请号：US18306638

申请日：2023-04-25

Applicant: Google LLC

Inventor： Harshit Kharbanda ,  Belinda Luna Zeng ,  Viviana Caso Corella ,  Christopher James Kelley ,  Jessica Lee ,  Pendar Yousefi ,  Dounia Berrada ,  Sundeep Vaddadi ,  Kai Yu ,  Balint Miklos ,  Severin Heiniger ,  Louis Wang

IPC: G06F16/9532 ,  G06F16/538 ,  G06F40/40

CPC classification number: G06F16/9532 ,  G06F16/538 ,  G06F40/40

Abstract: A multimodal search system is described. The system can receive image data captured by a camera of a user device. Additionally, the system can receive audio data associated with the image data. The audio data can be captured by a microphone of the user device. Moreover, the system can process the image data to generate visual features. Furthermore, the system can process the audio data to generate a plurality of words. The system can generate a plurality of search terms based on the plurality of words and the visual features. Subsequently, the system can determine one or more search results associated with the plurality of search terms and provide the one or more search results as an output.

公开(公告)号：US20220335661A1

公开(公告)日：2022-10-20

申请号：US17753425

申请日：2020-02-28

Applicant: Google LLC

Inventor： Max Spear ,  Nicholas Solochin ,  Anish Dhesikan ,  Kai Yu ,  Jacob Hanshaw ,  Yue Li ,  Charles DiFazio

IPC: G06T11/00 ,  G06V20/20

Abstract: Systems and methods for accessing augmented reality (AR) content, including dynamic AR content, relatively quickly, and relatively easily, are provided. An enabled target entity may be identified within a camera view of a mobile computing device, and a server computing device may identify the target entity, match the target entity with dynamic AR content, and transmit the dynamic AR content to the client computing device. This may allow users to consume content-rich information and/or to experience real time interaction with the images and objects in their surroundings, that would otherwise be difficult and/or unrealistic to condense into physical media, and/or that would be otherwise difficult and/or unrealistic to provide in a real time, interactive experience.

公开(公告)号：US20240403362A1

公开(公告)日：2024-12-05

申请号：US18326496

申请日：2023-05-31

Applicant: Google LLC

Inventor： Harshit Kharbanda ,  Belinda Luna Zeng ,  Viviana Caso Corella ,  Aashi Jain ,  David William Hendon ,  Christopher James Kelley ,  Jessica Lee ,  Dounia Berrada ,  Kai Yu ,  Louis Wang ,  Thomas J. Duerig ,  Radu Soricut ,  Robin Dua

IPC: G06F16/735 ,  G06F16/732 ,  G06F16/783 ,  G06T7/70 ,  G06V10/62 ,  G06V10/774 ,  G06V20/40

Abstract: A multimodal search system using a video query is described. The system can receive video data captured by a camera of a user device. The video data can have a sequence of image frames. Additionally, the system can receive audio data associated with the video data captured by the user device. Moreover, the system can process, using one or more machine-learned models, the sequence of image frames to generate video embeddings related to the sequence of the image frames. The video embeddings can have a plurality of image embeddings associated with the sequence of image frames. Furthermore, the system can determine one or more video results based on the video embeddings and the audio data. Subsequently, the system can transmit, to the user device, the one or more video results.

公开(公告)号：US20240339217A1

公开(公告)日：2024-10-10

申请号：US18620434

申请日：2024-03-28

Applicant: Google LLC

Inventor： Peggy Yen Phuong Bui ,  Bianca Madalina Buisman ,  Quang Anh Duong ,  Anastasia Martynova ,  Ayush Jain ,  Yuan Liu ,  Jonathan David Krause ,  Amit Sanjay Talreja ,  Rajeev Vijay Rikhye ,  Mahvish A. Nagda ,  Pinal Bavishi ,  Christopher James Eicher ,  Abigail Ward ,  Jieming Yu ,  Louis Wang ,  Dounia Berrada ,  Dale Richard Webster ,  Harshit Kharbanda ,  Igor Bonaci ,  Kai Yu ,  Ke Lan ,  Kaan Yücer ,  Willa Angel Chen Miller ,  Lars Thomas Hansen

IPC: G16H50/20 ,  G06T7/00 ,  G16H30/40

CPC classification number: G16H50/20 ,  G06T7/0012 ,  G16H30/40 ,  G06T2207/20104 ,  G06T2207/30088

Abstract: Systems and methods for diagnostic visual search can include processing a search query with a plurality of classification models to determine a search query intent and predict potential diagnosis. The search query can include an image that is processed to determine the presence of a body part and may be processed to determine if the search query is descriptive of a diagnostic search query. Based on the intent determination, the image may then be processed by a conditions classification model to determine one or more predicted condition classifications. Condition information can then be obtained and provided based on the one or more predicted condition classifications.