PHOTOMETRIC-BASED 3D OBJECT MODELING
    1.
    发明公开

    公开(公告)号:US20230316553A1

    公开(公告)日:2023-10-05

    申请号:US18205940

    申请日:2023-06-05

    申请人: Snap Inc.

    发明人: Oliver Woodford

    IPC分类号: G06T7/55 G06T5/00 G06T5/50

    摘要: Aspects of the present disclosure involve a system and a method for performing operations comprising: accessing a source image depicting a target structure; accessing one or more target images depicting at least a portion of the target structure; computing correspondence between a first set of pixels in the source image of a first portion of the target structure and a second set of pixels in the one or more target images of the first portion of the target structure, the correspondence being computed as a function of camera parameters that vary between the source image and the one or more target images; and generating a three-dimensional (3D) model of the target structure based on the correspondence between the first set of pixels in the source image and the second set of pixels in the one or more target images based on a joint optimization of target structure and camera parameters.

    LIGHT INVARIANT PATTERN MAPPING
    2.
    发明公开

    公开(公告)号:US20230245399A1

    公开(公告)日:2023-08-03

    申请号:US18131824

    申请日:2023-04-06

    申请人: Snap Inc.

    发明人: Oliver Woodford

    IPC分类号: G06T19/00 G06T7/73

    摘要: A correlation system can be implemented on a user device (e.g., smartphone) to perform image processing tasks e.g., point to point correlations for tasks such as alignment, tracking, and reconstruction of model data. The correlation system can implement normalized cross correlation in a least squares optimization scheme without use of approximations. Normalized cross correlation optimized via least squares can further implement global and local cost functions that are invariant to dynamic lighting conditions.

    LIGHT INVARIANT PATTERN MAPPING
    3.
    发明申请

    公开(公告)号:US20220084293A1

    公开(公告)日:2022-03-17

    申请号:US17456487

    申请日:2021-11-24

    申请人: Snap inc.

    发明人: Oliver Woodford

    IPC分类号: G06T19/00 G06T7/73

    摘要: A correlation system can be implemented on a user device (e.g., smartphone) to perform image processing tasks e.g., point to point correlations for tasks such as alignment, tracking, and reconstruction of model data. The correlation system can implement normalized cross correlation in a least squares optimization scheme without use of approximations. Normalized cross correlation optimized via least squares can further implement global and local cost functions that are invariant to dynamic lighting conditions.

    Motion representations for articulated animation

    公开(公告)号:US11798213B2

    公开(公告)日:2023-10-24

    申请号:US17364218

    申请日:2021-06-30

    申请人: Snap Inc.

    摘要: Systems and methods herein describe novel motion representations for animating articulated objects consisting of distinct parts. The described systems and method access source image data, identify driving image data to modify image feature data in the source image sequence data, generate, using an image transformation neural network, modified source image data comprising a plurality of modified source images depicting modified versions of the image feature data, the image transformation neural network being trained to identify, for each image in the source image data, a driving image from the driving image data, the identified driving image being implemented by the image transformation neural network to modify a corresponding source image in the source image data using motion estimation differences between the identified driving image and the corresponding source image, and stores the modified source image data.

    Region-based stabilized face tracking

    公开(公告)号:US10949648B1

    公开(公告)日:2021-03-16

    申请号:US16170997

    申请日:2018-10-25

    申请人: Snap Inc.

    IPC分类号: G06K9/00 G06T13/40 G06T7/73

    摘要: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program and a method for accessing a set of images depicting at least a portion of a face. A set of facial regions of the face is identified, each facial region of the set of facial regions intersecting another facial region with at least one common vertex which is a member of a set of facial vertices. For each facial region of the set of facial regions, a weight formed from a set of region coefficients is generated. Based on the set of facial regions and the weight of each facial region of the set of facial regions, the face is tracked across the set of images.

    Photometric-based 3D object modeling

    公开(公告)号:US11710248B2

    公开(公告)日:2023-07-25

    申请号:US17813887

    申请日:2022-07-20

    申请人: Snap Inc.

    发明人: Oliver Woodford

    IPC分类号: G06T7/55 G06T5/00 G06T5/50

    摘要: Aspects of the present disclosure involve a system and a method for performing operations comprising: accessing a source image depicting a target structure; accessing one or more target images depicting at least a portion of the target structure; computing correspondence between a first set of pixels in the source image of a first portion of the target structure and a second set of pixels in the one or more target images of the first portion of the target structure, the correspondence being computed as a function of camera parameters that vary between the source image and the one or more target images; and generating a three-dimensional (3D) model of the target structure based on the correspondence between the first set of pixels in the source image and the second set of pixels in the one or more target images based on a joint optimization of target structure and camera parameters.

    MOTION REPRESENTATIONS FOR ARTICULATED ANIMATION

    公开(公告)号:US20210407163A1

    公开(公告)日:2021-12-30

    申请号:US17364218

    申请日:2021-06-30

    申请人: Snap Inc.

    摘要: Systems and methods herein describe novel motion representations for animating articulated objects consisting of distinct parts. The described systems and method access source image data, identify driving image data to modify image feature data in the source image sequence data, generate, using an image transformation neural network, modified source image data comprising a plurality of modified source images depicting modified versions of the image feature data, the image transformation neural network being trained to identify, for each image in the source image data, a driving image from the driving image data, the identified driving image being implemented by the image transformation neural network to modify a corresponding source image in the source image data using motion estimation differences between the identified driving image and the corresponding source image, and stores the modified source image data.

    PHOTOMETRIC-BASED 3D OBJECT MODELING

    公开(公告)号:US20210334993A1

    公开(公告)日:2021-10-28

    申请号:US16861034

    申请日:2020-04-28

    申请人: Snap Inc.

    发明人: Oliver Woodford

    IPC分类号: G06T7/55 G06T5/00 G06T5/50

    摘要: Aspects of the present disclosure involve a system and a method for performing operations comprising: accessing a source image depicting a target structure; accessing one or more target images depicting at least a portion of the target structure; computing correspondence between a first set of pixels in the source image of a first portion of the target structure and a second set of pixels in the one or more target images of the first portion of the target structure, the correspondence being computed as a function of camera parameters that vary between the source image and the one or more target images; and generating a three-dimensional (3D) model of the target structure based on the correspondence between the first set of pixels in the source image and the second set of pixels in the one or more target images based on a joint optimization of target structure and camera parameters.