公开(公告)号：US12198357B2

公开(公告)日：2025-01-14

申请号：US18367034

申请日：2023-09-12

Applicant: Snap Inc.

Inventor： Shenlong Wang ,  Linjie Luo ,  Ning Zhang ,  Jia Li

IPC: G06T7/246 ,  G06T7/33 ,  G06V10/44 ,  G06V10/764 ,  G06V10/82 ,  G06T7/40

Abstract: Dense feature scale detection can be implemented using multiple convolutional neural networks trained on scale data to more accurately and efficiently match pixels between images. An input image can be used to generate multiple scaled images. The multiple scaled images are input into a feature net, which outputs feature data for the multiple scaled images. An attention net is used to generate an attention map from the input image. The attention map assigns emphasis as a soft distribution to different scales based on texture analysis. The feature data and the attention data can be combined through a multiplication process and then summed to generate dense features for comparison.

公开(公告)号：US20240037141A1

公开(公告)日：2024-02-01

申请号：US18378376

申请日：2023-10-10

Applicant: Snap Inc.

Inventor： Xiaoyu Wang ,  Ning Xu ,  Ning Zhang ,  Vitor Rocha de Carvalho ,  Jia Li

IPC: G06F16/58 ,  G06T1/00 ,  G06N3/08 ,  G06F16/9038 ,  G06N3/04 ,  G06F18/24 ,  G06N3/045 ,  H04N23/63 ,  G06V10/764 ,  G06V10/82 ,  G06V10/75

CPC classification number: G06F16/5866 ,  G06T1/0007 ,  G06N3/08 ,  G06F16/9038 ,  G06N3/04 ,  G06F18/24 ,  G06N3/045 ,  H04N23/63 ,  G06V10/764 ,  G06V10/82 ,  G06V10/751 ,  G06N5/022

Abstract: Systems, methods, devices, media, and computer readable instructions are described for local image tagging in a resource constrained environment. One embodiment involves processing image data using a deep convolutional neural network (DCNN) comprising at least a first subgraph and a second subgraph, the first subgraph comprising at least a first layer and a second layer, processing, the image data using at least the first layer of the first subgraph to generate first intermediate output data; processing, by the mobile device, the first intermediate output data using at least the second layer of the first subgraph to generate first subgraph output data, and in response to a determination that each layer reliant on the first intermediate data have completed processing, deleting the first intermediate data from the mobile device. Additional embodiments involve convolving entire pixel resolutions of the image data against kernels in different layers if the DCNN.

公开(公告)号：US11822600B2

公开(公告)日：2023-11-21

申请号：US17248386

申请日：2021-01-22

Applicant: Snap Inc.

Inventor： Xiaoyu Wang ,  Ning Xu ,  Ning Zhang ,  Vitor R. Carvalho ,  Jia Li

IPC: G06F16/58 ,  G06T1/00 ,  G06N3/08 ,  G06F16/9038 ,  G06N3/04 ,  G06F18/24 ,  G06N3/045 ,  H04N23/63 ,  G06V10/764 ,  G06V10/82 ,  G06V10/75 ,  G06N5/022

CPC classification number: G06F16/5866 ,  G06F16/9038 ,  G06F18/24 ,  G06N3/04 ,  G06N3/045 ,  G06N3/08 ,  G06T1/0007 ,  G06V10/751 ,  G06V10/764 ,  G06V10/82 ,  H04N23/63 ,  G06N5/022 ,  G06V2201/09

Abstract: Systems, methods, devices, media, and computer readable instructions are described for local image tagging in a resource constrained environment. One embodiment involves processing image data using a deep convolutional neural network (DCNN) comprising at least a first subgraph and a second subgraph, the first subgraph comprising at least a first layer and a second layer, processing, the image data using at least the first layer of the first subgraph to generate first intermediate output data; processing, by the mobile device, the first intermediate output data using at least the second layer of the first subgraph to generate first subgraph output data, and in response to a determination that each layer reliant on the first intermediate data have completed processing, deleting the first intermediate data from the mobile device. Additional embodiments involve convolving entire pixel resolutions of the image data against kernels in different layers if the DCNN.

公开(公告)号：US11645834B2

公开(公告)日：2023-05-09

申请号：US16949856

申请日：2020-11-17

Applicant: Snap Inc.

Inventor： Wei Han ,  Jianchao Yang ,  Ning Zhang ,  Jia Li

IPC: G06K9/62 ,  G06K9/46 ,  G06K9/52 ,  G06K9/66 ,  G06T3/40 ,  G06V10/42 ,  G06V10/44 ,  G06V10/46 ,  G06V30/194 ,  G06N3/04 ,  G06N3/084

CPC classification number: G06K9/6267 ,  G06K9/627 ,  G06K9/6256 ,  G06T3/40 ,  G06V10/42 ,  G06V10/44 ,  G06V10/462 ,  G06V30/194 ,  G06N3/0454 ,  G06N3/084 ,  G06V10/454

Abstract: Systems, devices, media, and methods are presented for identifying and categorically labeling objects within a set of images. The systems and methods receive an image depicting an object of interest, detect at least a portion of the object of interest within the image using a multilayer object model, determine context information, and identify the object of interest included in two or more bounding boxes.

公开(公告)号：US11545170B2

公开(公告)日：2023-01-03

申请号：US17247137

申请日：2020-12-01

Applicant: Snap Inc.

Inventor： Jinxi Guo ,  Jia Li ,  Ning Xu

IPC: G10L25/30 ,  H04S7/00 ,  G06K9/62 ,  G06N3/04 ,  G06V20/00

Abstract: An acoustic environment identification system is disclosed that can use neural networks to accurately identify environments. The acoustic environment identification system can use one or more convolutional neural networks to generate audio feature data. A recursive neural network can process the audio feature data to generate characterization data. The characterization data can be modified using a weighting system that weights signature data items. Classification neural networks can be used to generate a classification of an environment.

公开(公告)号：US20220406008A1

公开(公告)日：2022-12-22

申请号：US17856720

申请日：2022-07-01

Applicant: Snap Inc.

Inventor： Nathan Jurgenson ,  Linjie Luo ,  Jonathan M. Rodriguez, II ,  Rahul Bhupendra Sheth ,  Jia Li ,  Xutao Lv

IPC: G06T17/00 ,  G06T19/00

Abstract: Systems and methods for image based location estimation are described. In one example embodiment, a first positioning system is used to generate a first position estimate. Point cloud data describing an environment is then accessed. A two-dimensional surface of an image of an environment is captured, and a portion of the image is matched to a portion of key points in the point cloud data. An augmented reality object is then aligned within one or more images of the environment based on the match of the point cloud with the image. In some embodiments, building façade data may additionally be used to determine a device location and place the augmented reality object within an image.

公开(公告)号：US11483268B2

公开(公告)日：2022-10-25

申请号：US16918343

申请日：2020-07-01

Applicant: Snap Inc.

Inventor： Jianchao Yang ,  Yuke Zhu ,  Ning Xu ,  Kevin Dechau Tang ,  Jia Li

IPC: G06F17/00 ,  H04L51/10 ,  G06F16/22 ,  G06F16/51 ,  G06F16/583 ,  G06N20/00 ,  G06F16/9535 ,  G06F16/954 ,  G06F16/2457 ,  G06F16/951 ,  G06N3/04 ,  G06N3/08 ,  G06V20/00 ,  H04L51/52 ,  G06T7/00 ,  H04L67/01 ,  H04L67/55

Abstract: Systems, devices, methods, media, and instructions for automated image processing and content curation are described. In one embodiment a server computer system communicates at least a portion of a first content collection to a first client device, and receives a first selection communication in response, the first selection communication identifying a first piece of content of the first plurality of pieces of content. The server analyzes analyzing the first piece of content to identify a set of context values for the first piece of content, and accesses accessing a second content collection comprising pieces of content sharing at least a portion of the set of context values of the first piece of content. In various embodiments, different content values, image processing operations, and content selection operations are used to curate the content collections.

公开(公告)号：US20220245907A1

公开(公告)日：2022-08-04

申请号：US17728553

申请日：2022-04-25

Applicant: Snap Inc.

Inventor： Nathan Jurgenson ,  Linjie Luo ,  Jonathan M. Rodriguez, II ,  Rahul Bhupendra Sheth ,  Jia Li ,  Xutao Lv

IPC: G06T19/00 ,  G06T7/73 ,  G06V10/10 ,  G06V20/00 ,  G06V20/10 ,  G06V20/20 ,  G06F3/01 ,  G06F3/04815 ,  G06T7/20 ,  G06T13/80 ,  G06T19/20 ,  G06T7/246

Abstract: Systems and methods for image based location estimation are described. In one example embodiment, a first positioning system is used to generate a first position estimate. A set of structure façade data describing one or more structure façades associated with the first position estimate is then accessed. A first image of an environment is captured, and a portion of the image is matched to part of the structure façade data. A second position is then estimated based on a comparison of the structure façade data with the portion of the image matched to the structure façade data.

公开(公告)号：US11367205B1

公开(公告)日：2022-06-21

申请号：US16721483

申请日：2019-12-19

Applicant: Snap Inc.

Inventor： Shenlong Wang ,  Linjie Luo ,  Ning Zhang ,  Jia Li

IPC: G06T7/33 ,  G06V10/25 ,  G06V10/44 ,  G06T7/40

Abstract: Dense feature scale detection can be implemented using multiple convolutional neural networks trained on scale data to more accurately and efficiently match pixels between images. An input image can be used to generate multiple scaled images. The multiple scaled images are input into a feature net, which outputs feature data for the multiple scaled images. An attention net is used to generate an attention map from the input image. The attention map assigns emphasis as a soft distribution to different scales based on texture analysis. The feature data and the attention data can be combined through a multiplication process and then summed to generate dense features for comparison.

公开(公告)号：US11361489B2

公开(公告)日：2022-06-14

申请号：US16946346

申请日：2020-06-17

Applicant: Snap Inc.

Inventor： Linjie Yang ,  Kevin Dechau Tang ,  Jianchao Yang ,  Jia Li

IPC: G06K9/62 ,  G06T11/60 ,  G06T7/11 ,  G06V10/25 ,  G06V10/70 ,  G06V10/82 ,  G06V20/20 ,  G06V20/70

Abstract: A dense captioning system and method is provided for analyzing an image to generate proposed bounding regions for a plurality of visual concepts within the image, generating a region feature for each proposed bounding region to generate a plurality of region features of the image, and determining a context feature for the image using a proposed bounding region that is a largest in size of the proposed bounding regions. For each region feature of the plurality of region features of the image, the dense captioning system and method further provides for analyzing the region feature to determine for the region feature a detection score that indicates a likelihood that the region feature comprises an actual object, and generating a caption for a visual concept in the image using the region feature and the context feature when a detection score is above a specified threshold value.