公开(公告)号：US20240370717A1

公开(公告)日：2024-11-07

申请号：US18313189

申请日：2023-05-05

Applicant: Google LLC

Inventor： Qifei Wang ,  Yicheng Fan ,  Wei Xu ,  Jiayu Ye ,  Lu Wang ,  Chuo-Ling Chang ,  Dana Alon ,  Erik Nathan Vee ,  Hongkun Yu ,  Matthias Grundmann ,  Shanmugasundaram Ravikumar ,  Andrew Stephen Tomkins

IPC: G06N3/08

Abstract: A method for a cross-platform distillation framework includes obtaining a plurality of training samples. The method includes generating, using a student neural network model executing on a first processing unit, a first output based on a first training sample. The method also includes generating, using a teacher neural network model executing on a second processing unit, a second output based on the first training sample. The method includes determining, based on the first output and the second output, a first loss. The method further includes adjusting, based on the first loss, one or more parameters of the student neural network model. The method includes repeating the above steps for each training sample of the plurality of training samples.

公开(公告)号：US20230351724A1

公开(公告)日：2023-11-02

申请号：US17800688

申请日：2020-02-18

Applicant: Google LLC

Inventor： Tingbo Hou ,  Adel Ahmadyan ,  Jianing Wei ,  Matthias Grundmann

IPC: G06V10/75 ,  G06V10/80 ,  G06V20/40 ,  G06T7/70

CPC classification number: G06V10/751 ,  G06V10/817 ,  G06V20/46 ,  G06T7/70 ,  G06V2201/12 ,  G06V2201/07 ,  G06T2207/20081

Abstract: The present disclosure is directed to systems and methods for performing object detection and pose estimation in 3D from 2D images. Object detection can be performed by a machine-learned model configured to determine various object properties. Implementations according to the disclosure can use these properties to estimate object pose and size.

公开(公告)号：US11770551B2

公开(公告)日：2023-09-26

申请号：US17122292

申请日：2020-12-15

Applicant: Google LLC

Inventor： Adel Ahmadyan ,  Tingbo Hou ,  Jianing Wei ,  Liangkai Zhang ,  Artsiom Ablavatski ,  Matthias Grundmann

IPC: G06V10/00 ,  H04N19/54 ,  H04N19/593 ,  H04N19/17 ,  H04N19/105 ,  H04N19/62 ,  G06V20/40

CPC classification number: H04N19/54 ,  G06V20/49 ,  H04N19/105 ,  H04N19/17 ,  H04N19/593 ,  H04N19/62

Abstract: A method includes receiving a video comprising images representing an object, and determining, using a machine learning model, based on a first image of the images, and for each respective vertex of vertices of a bounding volume for the object, first two-dimensional (2D) coordinates of the respective vertex. The method also includes tracking, from the first image to a second image of the images, a position of each respective vertex along a plane underlying the bounding volume, and determining, for each respective vertex, second 2D coordinates of the respective vertex based on the position of the respective vertex along the plane. The method further includes determining, for each respective vertex, (i) first three-dimensional (3D) coordinates of the respective vertex based on the first 2D coordinates and (ii) second 3D coordinates of the respective vertex based on the second 2D coordinates.

公开(公告)号：US11726637B1

公开(公告)日：2023-08-15

申请号：US17978086

申请日：2022-10-31

Applicant: Google LLC

Inventor： Matthias Grundmann ,  Jokubas Zukerman ,  Marco Paglia ,  Kenneth Conley ,  Karthik Raveendran ,  Reed Morse

IPC: G06F3/0482 ,  G11B27/029 ,  G06F3/0485 ,  G06F3/04883 ,  H04N5/262 ,  G11B27/028

CPC classification number: G06F3/0482 ,  G06F3/0485 ,  G06F3/04883 ,  G11B27/028 ,  G11B27/029 ,  H04N5/2628

Abstract: The technology disclosed herein includes a user interface for viewing and combining media items into a video. An example method includes presenting a user interface that displays media items in a first portion of the user interface; receiving user input in the first portion that comprises a selection of a first media item; upon receiving the user input, adding the first media item to a set of selected media items and updating the user interface to comprise a control element and a second portion, wherein the first and second portions are concurrently displayed and are each scrollable along a different axis, and the second portion displays image content of the set and the control element enables a user to initiate the creation of the video based on the set of selected media items; and creating the video based on video content of the set of selected media items.

公开(公告)号：US11721039B2

公开(公告)日：2023-08-08

申请号：US17745125

申请日：2022-05-16

Applicant: Google LLC

Inventor： Jianing Wei ,  Matthias Grundmann

IPC: G06T7/73 ,  G06T19/20 ,  G06T19/00 ,  G01C19/00 ,  G02B27/01

CPC classification number: G06T7/74 ,  G01C19/00 ,  G02B27/017 ,  G06T19/006 ,  G06T19/20 ,  G06T2219/2004 ,  G06T2219/2016

Abstract: The present disclosure provides systems and methods for calibration-free instant motion tracking useful, for example, for rending virtual content in augmented reality settings. In particular, a computing system can iteratively augment image frames that depict a scene to insert virtual content at an anchor region within the scene, including situations in which the anchor region moves relative to the scene. To do so, the computing system can estimate, for each of a number of sequential image frames: a rotation of an image capture system that captures the image frames; and a translation of the anchor region relative to an image capture system, thereby providing sufficient information to determine where and at what orientation to render the virtual content within the image frame.

公开(公告)号：US20220415030A1

公开(公告)日：2022-12-29

申请号：US17778085

申请日：2019-11-19

Applicant: Tingbo HOU ,  Jianing WEI ,  Adel AHMADYAN ,  Matthias GRUNDMANN ,  Google LLC

Inventor： Tingbo Hou ,  Jianing Wei ,  Adel Ahmadyan ,  Matthias Grundmann

IPC: G06V10/774 ,  G06V20/64

Abstract: The present disclosure is directed to systems and methods for generating synthetic training data using augmented reality (AR) techniques. For example, images of a scene can be used to generate a three-dimensional mapping of the scene. The three-dimensional mapping may be associated with the images to indicate locations for positioning a virtual object. Using an AR rendering engine, implementations can generate an and orientation. The augmented image can then be stored in a machine learning dataset and associated with a label based on aspects of the virtual object.

公开(公告)号：US20220191542A1

公开(公告)日：2022-06-16

申请号：US17122292

申请日：2020-12-15

Applicant: Google LLC

Inventor： Adel Ahmadyan ,  Tingbo Hou ,  Jianing Wei ,  Liangkai Zhang ,  Artsiom Ablavatski ,  Matthias Grundmann

IPC: H04N19/54 ,  G06K9/00 ,  H04N19/62 ,  H04N19/17 ,  H04N19/105 ,  H04N19/593

Abstract: A method includes receiving a video comprising images representing an object, and determining, using a machine learning model, based on a first image of the images, and for each respective vertex of vertices of a bounding volume for the object, first two-dimensional (2D) coordinates of the respective vertex. The method also includes tracking, from the first image to a second image of the images, a position of each respective vertex along a plane underlying the bounding volume, and determining, for each respective vertex, second 2D coordinates of the respective vertex based on the position of the respective vertex along the plane. The method further includes determining, for each respective vertex, (i) first three-dimensional (3D) coordinates of the respective vertex based on the first 2D coordinates and (ii) second 3D coordinates of the respective vertex based on the second 2D coordinates.

公开(公告)号：US11158122B2

公开(公告)日：2021-10-26

申请号：US16591359

申请日：2019-10-02

Applicant: Google LLC

Inventor： Artsiom Ablavatski ,  Yury Kartynnik ,  Ivan Grishchenko ,  Matthias Grundmann

IPC: G06T17/20 ,  G06N3/08 ,  G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a neural network model to predict mesh vertices corresponding to a three-dimensional surface geometry of an object depicted in an image.

公开(公告)号：US20210133508A1

公开(公告)日：2021-05-06

申请号：US16668303

申请日：2019-10-30

Applicant: Google LLC

Inventor： Valentin Bazarevsky ,  Yury Kartynnik ,  Andrei Vakunov ,  Karthik Raveendran ,  Matthias Grundmann

IPC: G06K9/62 ,  G06N3/04 ,  G06N3/08 ,  G06K9/00

Abstract: A computing system is disclosed including a convolutional neural configured to receive an input that describes a facial image and generate a facial object recognition output that describes one or more facial feature locations with respect to the facial image. The convolutional neural network can include a plurality of convolutional blocks. At least one of the convolutional blocks can include one or more separable convolutional layers configured to apply a depthwise convolution and a pointwise convolution during processing of an input to generate an output. The depthwise convolution can be applied with a kernel size that is greater than 3×3. At least one of the convolutional blocks can include a residual shortcut connection from its input to its output.

公开(公告)号：US10229326B2

公开(公告)日：2019-03-12

申请号：US16125045

申请日：2018-09-07

Applicant: Google LLC

Inventor： Matthias Grundmann ,  Alexandra Ivanna Hawkins ,  Sergey Ioffe

IPC: G06K9/00 ,  G06K9/62

Abstract: Methods, systems, and media for summarizing a video with video thumbnails are provided. In some embodiments, the method comprises: receiving a plurality of video frames corresponding to the video and associated information associated with each of the plurality of video frames; extracting, for each of the plurality of video frames, a plurality of features; generating candidate clips that each includes at least a portion of the received video frames based on the extracted plurality of features and the associated information; calculating, for each candidate clip, a clip score based on the extracted plurality of features from the video frames associated with the candidate clip; calculating, between adjacent candidate clips, a transition score based at least in part on a comparison of video frame features between frames from the adjacent candidate clips; selecting a subset of the candidate clips based at least in part on the clip score and the transition score associated with each of the candidate clips; and automatically generating an animated video thumbnail corresponding to the video that includes a plurality of video frames selected from each of the subset of candidate clips.