公开(公告)号：US10445936B1

公开(公告)日：2019-10-15

申请号：US15666349

申请日：2017-08-01

Applicant: Snap Inc.

Inventor： Ebony James Charlton ,  Jokubas Dargis ,  Michael John Evans ,  Samuel Edward Hare ,  Andrew James McPhee ,  Robert Cornelius Murphy ,  Eitan Pilipski

IPC: G06T19/20 ,  G06T19/00 ,  G06F3/16

Abstract: Systems, devices, media, and methods are presented for generating responsive augmented reality elements in a graphical user interface. The systems and methods receive a video stream having one or more frames containing image data and audio data. The systems and methods identify a set of coordinates within a portion of the one or more frames and identify one or more audio characteristics within the audio data of the video stream. In response to the one or more audio characteristics, the systems and methods generate one or more graphical interface elements and detect a change in the audio data within the video stream. The systems and methods modify the one or more graphical interface elements in a second portion of the video stream in response to the change in the audio data.

公开(公告)号：US12277632B2

公开(公告)日：2025-04-15

申请号：US17662745

申请日：2022-05-10

Applicant: Snap Inc.

Inventor： Kyle Goodrich ,  Pedram Javidpour ,  Maxim Maximov Lazarov ,  Andrew James McPhee ,  Daniel Moreno

IPC: G06T11/60 ,  G06T19/00 ,  H04N23/45 ,  H04N23/62

Abstract: Methods and systems are disclosed for generating AR experiences on a messaging platform. The methods and systems perform operations including: detecting a real-world object depicted in a first image captured by a first camera of a client device, the client device comprising a second camera; extracting one or more textures from the real-world object depicted in the first image; selecting a target object depicted in a second image captured by the second camera, the second image being captured by the second camera simultaneously with the first image captured by the first camera; generating an augmented reality (AR) element comprising the target object modified based on the one or more textures extracted from the real-world object depicted in the first image; and causing display of the AR element within the second image.

公开(公告)号：US12211159B2

公开(公告)日：2025-01-28

申请号：US18380577

申请日：2023-10-16

Applicant: Snap Inc.

Inventor： Samuel Edward Hare ,  Andrew James McPhee ,  Maxim Maximov Lazarov ,  Wentao Shang ,  Kyle Goodrich ,  Tony Mathew

IPC: G06T19/00 ,  G06T19/20 ,  H04L51/04 ,  H04N5/272 ,  H04N23/90

Abstract: Systems and methods are provided for capturing by a camera of a user device, a first image depicting a first environment of the user device; overlaying a first virtual object on a portion of the first image depicting the first environment; modifying a surface of the first virtual object using content captured by the user device; storing a second virtual object comprising the first virtual object with the modified surface; and generating for display the second virtual object on a portion of a second image depicting a second environment.

公开(公告)号：US12182951B2

公开(公告)日：2024-12-31

申请号：US18240373

申请日：2023-08-31

Applicant: Snap Inc.

Inventor： Kyle Goodrich ,  Samuel Edward Hare ,  Maxim Maximov Lazarov ,  Tony Mathew ,  Andrew James McPhee ,  Daniel Moreno ,  Wentao Shang

IPC: G06T19/00 ,  G06F3/04815 ,  G06F3/0482 ,  G06F3/0488 ,  G06T5/77

Abstract: The subject technology generates a segmentation mask based on first image data. The subject technology applies the segmentation mask on first depth data to reduce a set of artifacts in a depth map based on the first depth data. The subject technology generates a packed depth map based at least in part on the depth map. The subject technology converts a single channel floating point texture to a raw depth map. The subject technology generates multiple channels. The subject technology applies, to the first image data and the first depth data, a first augmented reality content generator corresponding to a selected first selectable graphical item, the first image data and the first depth data being captured with a camera. The subject technology generates a message including the applied first augmented reality content generator to the first image data and the first depth data.

公开(公告)号：US12164699B2

公开(公告)日：2024-12-10

申请号：US18375763

申请日：2023-10-02

Applicant: Snap Inc.

Inventor： Dylan Shane Eirinberg ,  Kyle Goodrich ,  Andrew James McPhee ,  Daniel Moreno

IPC: G06F3/044 ,  G06F3/01 ,  G06F3/04815 ,  G06F3/0482 ,  G06T7/20 ,  G06V40/10

Abstract: Systems and methods are provided for performing operations comprising: capturing, by an electronic mirroring device, a video feed received from a camera of the electronic mirroring device, the video feed depicting a user; identifying a finger of a hand of the user in the video feed; determining a direction within the video feed to which the finger is pointing; and accessing a first menu option based on the direction within the video feed to which the finger is pointing.

公开(公告)号：US12094063B2

公开(公告)日：2024-09-17

申请号：US17932019

申请日：2022-09-14

Applicant: Snap Inc.

Inventor： Andrew James McPhee ,  Samuel Edward Hare ,  Peicheng Yu ,  Robert Cornelius Murphy ,  Dhritiman Sagar

IPC: G06T19/00 ,  A63F13/211 ,  G06F3/01 ,  G06F3/03 ,  G06F3/0346 ,  G06F3/038 ,  G06F11/08 ,  G06T7/246 ,  G06T15/20

CPC classification number: G06T19/003 ,  A63F13/211 ,  G06F3/011 ,  G06F3/0304 ,  G06F3/0346 ,  G06F3/038 ,  G06F11/08 ,  G06T7/246 ,  G06T15/20 ,  G06T19/006 ,  G06F2203/0381 ,  G06F2203/0382

Abstract: A redundant tracking system comprising multiple redundant tracking sub-systems, enabling seamless transitions between such tracking sub-systems, provides a solution to this problem by merging multiple tracking approaches into a single tracking system. This system is able to combine tracking objects with six degrees of freedom (6 DoF) and 3 DoF through combining and transitioning between multiple tracking systems based on the availability of tracking indicia tracked by the tracking systems. Thus, as the indicia tracked by any one tracking system becomes unavailable, the redundant tracking system seamlessly switches between tracking in 6 DoF and 3 DoF thereby providing the user with an uninterrupted experience.

公开(公告)号：US12020358B2

公开(公告)日：2024-06-25

申请号：US17514314

申请日：2021-10-29

Applicant: Snap Inc.

Inventor： Kyle Goodrich ,  Christie Marie Heikkinen ,  Pedram Javidpour ,  Andrew James McPhee ,  Daniel Moreno

IPC: G06T13/40 ,  G06F3/0482 ,  G06T11/60 ,  G06T13/80 ,  H04L51/10

CPC classification number: G06T13/40 ,  G06F3/0482 ,  G06T11/60 ,  G06T13/80 ,  H04L51/10 ,  G06T2200/24

Abstract: Methods and systems are disclosed for performing operations for generating a custom sticker. The operations include receiving, by a messaging application implemented on a first device, an image. The operations include receiving, by the messaging application, input that selects a region of the image. The operations include generating, by the messaging application, a graphical element comprising the region of the image selected by the input. The operations include applying, by the messaging application, one or more visual effects to the graphical element. The operations include causing the graphical element to be animated according to a given animation pattern selected from a plurality of animation patterns to generate an animated custom graphic. The operations include sending the animated custom graphic with the one or more visual effects from the first device to a second device.

公开(公告)号：US11978283B2

公开(公告)日：2024-05-07

申请号：US17249855

申请日：2021-03-16

Applicant: Snap Inc.

Inventor： Dylan Shane Eirinberg ,  Kyle Goodrich ,  Andrew James McPhee ,  Daniel Moreno

IPC: G06F3/04847 ,  G02B27/01 ,  G06F3/01 ,  G06F3/0482 ,  G06F3/14 ,  G06V40/20

CPC classification number: G06V40/28 ,  G02B27/0172 ,  G06F3/017 ,  G06F3/0482 ,  G06F3/04847 ,  G06F3/1454 ,  G06V40/23 ,  G02B2027/0178

Abstract: Systems and methods are provided for performing operations comprising: capturing, by an electronic mirroring device, a video feed received from a camera of the electronic mirroring device, the video feed depicting a user; displaying, by one or more processors of the electronic mirroring device, one or more menu options on the video feed that depicts the user; identifying a hand of the user in the video feed; determining that a position of the hand in the video feed overlaps a position of a given menu option of the one or more menu options; and performing an operation associated with the given menu option in response to determining that the position of the hand in the video feed overlaps the position of the given menu option.

公开(公告)号：US11961189B2

公开(公告)日：2024-04-16

申请号：US18144058

申请日：2023-05-05

Applicant: Snap Inc.

Inventor： Kyle Goodrich ,  Samuel Edward Hare ,  Maxim Maximov Lazarov ,  Tony Mathew ,  Andrew James McPhee ,  Daniel Moreno ,  Dhritiman Sagar ,  Wentao Shang

IPC: G06T19/00 ,  G06F3/0482 ,  G06F3/04842 ,  G06T7/50 ,  H04L51/42

CPC classification number: G06T19/00 ,  G06F3/04842 ,  G06T7/50 ,  H04L51/42 ,  G06T2207/20084

Abstract: The subject technology generates depth data using a machine learning model based at least in part on captured image data from at least one camera of a client device. The subject technology applies, to the captured image data and the generated depth data, a 3D effect based at least in part on an augmented reality content generator. The subject technology generates a depth map using at least the depth data. The subject technology generates a packed depth map based at least in part on the depth map, the generating the packed depth map. The subject technology converts a single channel floating point texture to a raw depth map. The subject technology generates multiple channels based at least in part on the raw depth map. The subject technology generates a segmentation mask based at least on the captured image data. The subject technology performs background inpainting and blurring of the captured image data using at least the segmentation mask to generate background inpainted image data.

公开(公告)号：US20240087609A1

公开(公告)日：2024-03-14

申请号：US17941292

申请日：2022-09-09

Applicant: Snap Inc.

Inventor： Kyle Goodrich ,  Maxim Maximov Lazarov ,  Andrew James McPhee ,  Daniel Moreno

IPC: G11B27/031 ,  G06T19/00 ,  G06V40/20

CPC classification number: G11B27/031 ,  G06T19/006 ,  G06V40/28

Abstract: The subject technology receives frames of a source media content. The subject technology detects from the frames of the source media content, a first gesture indicating a cut point at a particular frame of the source media content, the cut point associated with a trimming operation to be performed on the source media content. The subject technology selects a starting frame and an ending frame from the frames based at least in part on the cut point at the particular frame. The subject technology performs the trimming operation based on the starting frame and the ending frame. The subject technology generates a second media content using the third set of frames. The subject technology provides for display at least a portion of the third set of frames of the second media content.