公开(公告)号：US20250008181A1

公开(公告)日：2025-01-02

申请号：US18216010

申请日：2023-06-29

Applicant: Google LLC

Inventor： Tristan Schmelcher ,  Markus Kuhn ,  Laura McKnight ,  Alexander Kharitonov ,  David Kim

IPC: H04N21/439 ,  H04N21/2187 ,  H04N21/845

Abstract: A method includes receiving, by a processor, from a first server, a first plurality of frames associated with a media item. Each frame of the first plurality of frames is associated with a respective timestamp of a first plurality of timestamps generated by the first server. A second plurality of frames associated with the media item are received from a second server. The second plurality of frames are each associated with a respective timestamp of a second plurality of timestamps generated by the second server. An offset value between a first timestamp of the first plurality of timestamps and a second timestamp of the second plurality of timestamps is determined. A modified plurality of frames is generated by modifying, based on the offset value, each timestamp of a subset of the second plurality of timestamps. The modified plurality of frames is sent to the client device.

公开(公告)号：US11868583B2

公开(公告)日：2024-01-09

申请号：US17656818

申请日：2022-03-28

Applicant: Google LLC

Inventor： Ruofei Du ,  Alex Olwal ,  Mathieu Simon Le Goc ,  David Kim ,  Danhang Tang

IPC: G06F3/04815 ,  G02B27/01 ,  G10L15/22

CPC classification number: G06F3/04815 ,  G02B27/0172 ,  G02B27/0176 ,  G10L15/22 ,  G02B2027/0178 ,  G10L2015/223

Abstract: Systems and methods are provided in which physical objects in the ambient environment can function as user interface implements in an augmented reality environment. A physical object detected within a field of view of a camera of a computing device may be designated as a user interface implement in response to a user command. User interfaces may be attached to the designated physical object, to provide a tangible user interface implement for user interaction with the augmented reality environment.

公开(公告)号：US20230393665A1

公开(公告)日：2023-12-07

申请号：US18174358

申请日：2023-02-24

Applicant: GOOGLE LLC

Inventor： Dongeek Shin ,  Shahram Izadi ,  David Kim ,  Sofien Bouaziz ,  Steven Benjamin Goldberg ,  Ivan Poupyrev ,  Shwetak N. Patel

IPC: G06F3/01 ,  G06F1/16 ,  G06V40/20

CPC classification number: G06F3/017 ,  G06F1/163 ,  G06F3/014 ,  G06V40/28

Abstract: Techniques of identifying gestures include detecting and classifying inner-wrist muscle motions at a user's wrist using micron-resolution radar sensors. For example, a user of an AR system may wear a band around their wrist. When the user makes a gesture to manipulate a virtual object in the AR system as seen in a head-mounted display (HMD), muscles and ligaments in the user's wrist make small movements on the order of 1-3 mm. The band contains a small radar device that has a transmitter and a number of receivers (e.g., three) of electromagnetic (EM) radiation on a chip (e.g., a Soli chip. This radiation reflects off the wrist muscles and ligaments and is received by the receivers on the chip in the band. The received reflected signal, or signal samples, are then sent to processing circuitry for classification to identify the wrist movement as a gesture.

公开(公告)号：US20230206568A1

公开(公告)日：2023-06-29

申请号：US18000237

申请日：2020-06-22

Applicant: GOOGLE LLC

Inventor： Ruofei Du ,  David Kim

IPC: G06T19/00 ,  G06T7/50 ,  G06T15/50 ,  G06T7/90 ,  G06T7/70 ,  G06T19/20

CPC classification number: G06T19/006 ,  G06T7/50 ,  G06T15/506 ,  G06T7/90 ,  G06T7/70 ,  G06T19/20 ,  G06T2219/2012 ,  G06T2207/10028

Abstract: A system and method for relighting of an augmented reality scene are provided. Image data and depth data corresponding to a physical environment may be captured by an image sensor of a computing device, for display in an AR scene generated by the computing device. The system may use the image data and depth data to generate a live depth map of the physical environment. A virtual light source may be placed in the AR scene of the physical environment in response to a user input. The system may relight the AR scene in response to the placement of the virtual light source based on the image data and the depth map.

公开(公告)号：US11030773B2

公开(公告)日：2021-06-08

申请号：US16798881

申请日：2020-02-24

Applicant: Google LLC

Inventor： Jonathan James Taylor ,  Vladimir Tankovich ,  Danhang Tang ,  Cem Keskin ,  Adarsh Prakash Murthy Kowdle ,  Philip L. Davidson ,  Shahram Izadi ,  David Kim

IPC: G06K9/00 ,  G06T7/73 ,  G06T7/246 ,  G06T7/149 ,  G06T7/194 ,  G06T7/162 ,  G06F3/01

Abstract: An electronic device estimates a pose of a hand by volumetrically deforming a signed distance field using a skinned tetrahedral mesh to locate a local minimum of an energy function, wherein the local minimum corresponds to the hand pose. The electronic device identifies a pose of the hand by fitting an implicit surface model of a hand to the pixels of a depth image that correspond to the hand. The electronic device uses a skinned tetrahedral mesh to warp space from a base pose to a deformed pose to define an articulated signed distance field from which the hand tracking module derives candidate poses of the hand. The electronic device then minimizes an energy function based on the distance of each corresponding pixel to identify the candidate pose that most closely approximates the pose of the hand.

公开(公告)号：US20250094116A1

公开(公告)日：2025-03-20

申请号：US18884920

申请日：2024-09-13

Applicant: Google LLC

Inventor： Ruofei Du ,  David Kim ,  Fengyuan Zhu ,  Daniel Kalmar

IPC: G06F3/14 ,  G06F3/01 ,  G06F3/0346 ,  G06F3/04886

Abstract: A method can include determining, by a head-mounted device mounted on a head of a user, a number of degrees of freedom of an input modality of an auxiliary device; determining an input modality of the head-mounted device based on the number of degrees of freedom of the input modality of the auxiliary device; and presenting the input modality of the head-mounted device to the user.

公开(公告)号：US12236541B2

公开(公告)日：2025-02-25

申请号：US17998736

申请日：2020-05-29

Applicant: GOOGLE LLC

Inventor： David Kim ,  Ruofei Du

IPC: G06T19/00 ,  G06T7/50 ,  G06T15/06 ,  G06T15/60 ,  G06T17/20

Abstract: Techniques of introducing virtual objects into a physical environment of AR system include displacing vertices of a mesh representing the physical environment based on a live depth map. For example, an AR system generates a mesh template, i.e., an initial mesh with vertices that represents a physical environment and a depth map that indicates a geometry of real objects within the physical environment. The AR system is configured to represent the real objects in the physical environment by displacing the vertices of the mesh based on depth values of the depth map and parameter values of a pinhole camera model. The depth values may be taken from the perspective of an illumination source in the physical environment.

公开(公告)号：US20220300082A1

公开(公告)日：2022-09-22

申请号：US17249966

申请日：2021-03-19

Applicant: GOOGLE LLC

Inventor： Dongeek Shin ,  Shahram Izadi ,  David Kim ,  Sofien Bouaziz ,  Steven Benjamin Goldberg ,  Ivan Poupyrev ,  Shwetak N. Patel

IPC: G06F3/01 ,  G06K9/00 ,  G06F1/16

Abstract: Techniques of identifying gestures include detecting and classifying inner-wrist muscle motions at a user's wrist using micron-resolution radar sensors. For example, a user of an AR system may wear a band around their wrist. When the user makes a gesture to manipulate a virtual object in the AR system as seen in a head-mounted display (HMD), muscles and ligaments in the user's wrist make small movements on the order of 1-3 mm. The band contains a small radar device that has a transmitter and a number of receivers (e.g., three) of electromagnetic (EM) radiation on a chip (e.g., a Soli chip. This radiation reflects off the wrist muscles and ligaments and is received by the receivers on the chip in the band. The received reflected signal, or signal samples, are then sent to processing circuitry for classification to identify the wrist movement as a gesture.

公开(公告)号：US20240428473A1

公开(公告)日：2024-12-26

申请号：US18792033

申请日：2024-08-01

Applicant: Google LLC

Inventor： Diego Rivas Vetencourt ,  Yilong Yao ,  David Kim ,  Steven Soon Leong Toh

IPC: G06T11/00 ,  G06F3/01 ,  G06K7/14 ,  G06T7/70 ,  G06V20/20 ,  G06V20/40 ,  H04L67/131

Abstract: Methods and systems for displaying screens in extended reality are disclosed herein. The method can include receiving video data of a real-world environment of a user of an extended reality device and identifying a screen of a separate computing device in the video data. The method can also include pairing the extended reality device and the separate computing device, generating a graphical outline around the screen of the separate computing device, and displaying, by the processor, the screen of the separate computing device and the graphical outline around the screen of the separate computing device on a display of the extended reality device. The method can further include detecting a user interaction with the graphical outline around the screen of the separate computing device based on the received video data and in response to detecting the user interaction, performing an action associated with the user interaction.

公开(公告)号：US12079903B1

公开(公告)日：2024-09-03

申请号：US18341461

申请日：2023-06-26

Applicant: Google LLC

Inventor： Diego Rivas Vetencourt ,  Yilong Yao ,  David Kim ,  Steven Soon Leong Toh

IPC: G06F3/01 ,  G06T7/70 ,  G06T11/00 ,  G06V20/20 ,  G06V20/40 ,  G06K7/14 ,  H04L67/131

CPC classification number: G06T11/00 ,  G06F3/011 ,  G06F3/017 ,  G06T7/70 ,  G06V20/20 ,  G06V20/41 ,  G06K7/1417 ,  G06T2207/10016 ,  G06T2207/30196 ,  H04L67/131

Abstract: Methods and systems for displaying screens in extended reality are disclosed herein. The method can include receiving video data of a real-world environment of a user of an extended reality device and identifying a screen of a separate computing device in the video data. The method can also include pairing the extended reality device and the separate computing device, generating a graphical outline around the screen of the separate computing device, and displaying, by the processor, the screen of the separate computing device and the graphical outline around the screen of the separate computing device on a display of the extended reality device. The method can further include detecting a user interaction with the graphical outline around the screen of the separate computing device based on the received video data and in response to detecting the user interaction, performing an action associated with the user interaction.