公开(公告)号：US20240395028A1

公开(公告)日：2024-11-28

申请号：US18400677

申请日：2023-12-29

Applicant: Snap Inc.

Inventor： Pavlo Chemerys ,  Colin Eles ,  Ju Hu ,  Qing Jin ,  Yanyu Li ,  Ergeta Muca ,  Jian Ren ,  Dhritiman Sagar ,  Aleksei Stoliar ,  Sergey Tulyakov ,  Huan Wang

IPC: G06V10/82 ,  G06N3/0455

Abstract: Described is a system for improving machine learning models. In some cases, the system improves such models by identifying an autoencoder for a latent diffusion machine learning model, the latent diffusion machine learning model is trained to receive text as input and output an image based on the received text. The system identifies a number of channels in a decoder of the autoencoder, the decoder being configured to receive latent features as input and output images. The system further identifies a performance characteristic of the decoder and changes the node topology of the decoder based on the performance characteristic to generate an updated decoder. The system retrains the latent diffusion machine learning model using the updated decoder by inputting latent features to the updated decoder, receiving an outputted image from the updated decoder, and updating one or more weights of the decoder based on an assessment of the outputted image.

公开(公告)号：US11670025B2

公开(公告)日：2023-06-06

申请号：US17323182

申请日：2021-05-18

Applicant: Snap Inc.

Inventor： Ebony James Charlton ,  Dhritiman Sagar ,  Daniel Vincent Grippi

IPC: G06T11/60 ,  H04W4/021 ,  G06F3/04817 ,  G01S19/13

CPC classification number: G06T11/60 ,  G06F3/04817 ,  H04W4/021 ,  G01S19/13 ,  G06F2203/04806 ,  G06T2200/24

Abstract: A content display system can control which content and how the content is displayed based on viewing parameters, such as a map zoom level, and physical distance parameters, e.g., a geo-fence distance and an icon visibility distance. Different combinations of input (e.g., zoom level and physical distances) yield a myriad of pre-set content displays on the client device, thereby allowing a creator of an icon to finely tune how content displayed otherwise accessed.

公开(公告)号：US20230017627A1

公开(公告)日：2023-01-19

申请号：US17952051

申请日：2022-09-23

Applicant: Snap Inc.

Inventor： Kyle Goodrich ,  Samuel Edward Hare ,  Maxim Maximov Lazarov ,  Tony Mathew ,  Andrew James McPhee ,  Daniel Moreno ,  Dhritiman Sagar ,  Wentao Shang

IPC: G06T19/00 ,  G06F3/04842 ,  G06T7/50 ,  H04L51/42

Abstract: The subject technology generates depth data using a machine learning model based at least in part on captured image data from at least one camera of a client device. The subject technology applies, to the captured image data and the generated depth data, a 3D effect based at least in part on an augmented reality content generator. The subject technology generates a depth map using at least the depth data. The subject technology generates a packed depth map based at least in part on the depth map, the generating the packed depth map. The subject technology converts a single channel floating point texture to a raw depth map. The subject technology generates multiple channels based at least in part on the raw depth map. The subject technology generates a segmentation mask based at least on the captured image data. The subject technology performs background inpainting and blurring of the captured image data using at least the segmentation mask to generate background inpainted image data.

公开(公告)号：US20230015194A1

公开(公告)日：2023-01-19

申请号：US17950761

申请日：2022-09-22

Applicant: Snap Inc.

Inventor： Kyle Goodrich ,  Samuel Edward Hare ,  Maxim Maximov Lazarov ,  Tony Mathew ,  Andrew James McPhee ,  Daniel Moreno ,  Dhritiman Sagar ,  Wentao Shang

IPC: H04N13/128 ,  G06T7/571 ,  H04N13/111 ,  H04L67/131 ,  H04N13/239 ,  G06T19/00 ,  G06T7/593 ,  H04N13/00

Abstract: The subject technology receives, at a client device, a selection of a selectable graphical item from a plurality of selectable graphical items, the selectable graphical item comprising an augmented reality content generator including a 3D effect. The subject technology applies, to image data and depth data, the 3D effect based at least in part on the augmented reality content generator, the applying the 3D effect. The subject technology generates a depth map using at least the depth data, generates a segmentation mask based at least on the image data, and performs background inpainting and blurring of the image data using at least the segmentation mask to generate background inpainted image data. The subject technology generates a 3D message based at least in part on the applied 3D effect.

公开(公告)号：US11481978B2

公开(公告)日：2022-10-25

申请号：US16948381

申请日：2020-09-16

Applicant: Snap Inc.

Inventor： Andrew James McPhee ,  Samuel Edward Hare ,  Peicheng Yu ,  Robert Cornelius Murphy ,  Dhritiman Sagar

IPC: G06F3/03 ,  A63F13/211 ,  G06T19/00 ,  G06T7/246 ,  G06F3/038 ,  G06F3/0346 ,  G06F3/01 ,  G06F11/08 ,  G06T15/20

Abstract: A redundant tracking system comprising multiple redundant tracking sub-systems, enabling seamless transitions between such tracking sub-systems, provides a solution to this problem by merging multiple tracking approaches into a single tracking system. This system is able to combine tracking objects with six degrees of freedom (6DoF) and 3DoF through combining and transitioning between multiple tracking systems based on the availability of tracking indicia tracked by the tracking systems. Thus, as the indicia tracked by any one tracking system becomes unavailable, the redundant tracking system seamlessly switches between tracking in 6DoF and 3DoF thereby providing the user with an uninterrupted experience.

公开(公告)号：US11457196B2

公开(公告)日：2022-09-27

申请号：US17006507

申请日：2020-08-28

Applicant: Snap Inc.

Inventor： Kyle Goodrich ,  Samuel Edward Hare ,  Maxim Maximov Lazarov ,  Tony Mathew ,  Andrew James McPhee ,  Daniel Moreno ,  Dhritiman Sagar ,  Wentao Shang

IPC: H04N13/128 ,  G06T19/00 ,  H04N13/239 ,  G06T7/593 ,  G06T7/571 ,  H04N13/111 ,  H04L67/131 ,  H04N13/00

Abstract: The subject technology selects a set of augmented reality content generators from a plurality of available augmented reality content generator based on metadata associated with each respective augmented reality content generator. The subject technology receives, at a client device, a selection of a selectable graphical item from a plurality of selectable graphical items, the selectable graphical item comprising an augmented reality content generator including a 3D effect. The subject technology captures image data and depth data using at least one camera of the client device. The subject technology applies, to the image data and the depth data, the 3D effect based at least in part on the augmented reality content generator.

公开(公告)号：US20220284682A1

公开(公告)日：2022-09-08

申请号：US17525612

申请日：2021-11-12

Applicant: Snap Inc.

Inventor： Kyle Goodrich ,  Samuel Edward Hare ,  Maxim Maximov Lazarov ,  Tony Mathew ,  Andrew James McPhee ,  Daniel Moreno ,  Dhritiman Sagar ,  Wentao Shang

IPC: G06T19/00 ,  G06T7/194 ,  G06T7/50 ,  H04L67/131

Abstract: The subject technology receives image data and depth data. The subject technology selects an augmented reality content generator corresponding to a three-dimensional (3D) effect. The subject technology applies the 3D effect to the image data and the depth data based at least in part on the selected augmented reality content generator. The subject technology generates, using a processor, a message including information related to the applied 3D effect, the image data, and the depth data.

公开(公告)号：US20210375056A1

公开(公告)日：2021-12-02

申请号：US17321706

申请日：2021-05-17

Applicant: Snap Inc.

Inventor： Ebony James Charlton ,  Jokubas Dargis ,  Eitan Pilipski ,  Dhritiman Sagar ,  Victor Shaburov

IPC: G06T19/00 ,  A63F13/213 ,  A63F13/428 ,  H04W4/021 ,  H04W4/02

Abstract: A context based augmented reality system can be used to display augmented reality elements over a live video feed on a client device. The augmented reality elements can be selected based on a number of context inputs generated by the client device. The context inputs can include location data of the client device and location data of nearby physical places that have preconfigured augmented elements. The preconfigured augmented elements can be preconfigured to exhibit a design scheme of the corresponding physical place.

公开(公告)号：US20200258307A1

公开(公告)日：2020-08-13

申请号：US16745117

申请日：2020-01-16

Applicant: Snap Inc.

Inventor： Ebony James Charlton ,  Jokubas Dargis ,  Eitan Pilipski ,  Dhritiman Sagar ,  Victor Shaburov

IPC: G06T19/00 ,  H04W4/02 ,  H04W4/021 ,  A63F13/428 ,  A63F13/213

Abstract: A context based augmented reality system can be used to display augmented reality elements over a live video feed on a client device. The augmented reality elements can be selected based on a number of context inputs generated by the client device. The context inputs can include location data of the client device and location data of nearby physical places that have preconfigured augmented elements. The preconfigured augmented elements can be preconfigured to exhibit a design scheme of the corresponding physical place.

公开(公告)号：US10264422B2

公开(公告)日：2019-04-16

申请号：US15967201

申请日：2018-04-30

Applicant: Snap Inc.

Inventor： Ebony James Charlton ,  Sumant Hanumante ,  Zhou Ren ,  Dhritiman Sagar

IPC: H04W4/33 ,  G06K9/00 ,  H04L12/58 ,  G06K9/62 ,  H04W88/02 ,  G06T11/60

Abstract: A venue system of a client device can submit a location request to a server, which returns multiple venues that are near the client device. The client device can use one or more machine learning schemes (e.g., convolutional neural networks) to determine that the client device is located in one of specific venues of the possible venues. The venue system can further select imagery for presentation based on the venue selection. The presentation may be published as ephemeral message on a network platform.