公开(公告)号：US20200082541A1

公开(公告)日：2020-03-12

申请号：US16566082

申请日：2019-09-10

Applicant: Apple Inc.

Inventor： Mark N. Jouppi ,  Alexander Lindskog ,  Michael W. Tao

IPC: G06T7/194 ,  G06T5/00 ,  G06T5/20 ,  H04N13/128

Abstract: This disclosure relates to techniques for generating robust depth estimations for captured images using semantic segmentation. Semantic segmentation may be defined as a process of creating a mask over an image, wherein pixels are segmented into a predefined set of semantic classes. Such segmentations may be binary (e.g., a ‘person pixel’ or a ‘non-person pixel’) or multi-class (e.g., a pixel may be labelled as: ‘person,’ ‘dog,’ ‘cat,’ etc.). As semantic segmentation techniques grow in accuracy and adoption, it is becoming increasingly important to develop methods of utilizing such segmentations and developing flexible techniques for integrating segmentation information into existing computer vision applications, such as depth and/or disparity estimation, to yield improved results in a wide range of image capture scenarios. In some embodiments, an optimization framework may be employed to optimize a camera device's initial scene depth/disparity estimates that employs both semantic segmentation and color regularization in a robust fashion.

公开(公告)号：US11526995B2

公开(公告)日：2022-12-13

申请号：US16566082

申请日：2019-09-10

Applicant: Apple Inc.

Inventor： Mark N. Jouppi ,  Alexander Lindskog ,  Michael W. Tao

IPC: G06T7/194 ,  G06T5/00 ,  G06T5/20 ,  H04N13/128 ,  H04N13/156 ,  H04N13/271 ,  H04N13/00 ,  G06T7/50

Abstract: This disclosure relates to techniques for generating robust depth estimations for captured images using semantic segmentation. Semantic segmentation may be defined as a process of creating a mask over an image, wherein pixels are segmented into a predefined set of semantic classes. Such segmentations may be binary (e.g., a ‘person pixel’ or a ‘non-person pixel’) or multi-class (e.g., a pixel may be labelled as: ‘person,’ ‘dog,’ ‘cat,’ etc.). As semantic segmentation techniques grow in accuracy and adoption, it is becoming increasingly important to develop methods of utilizing such segmentations and developing flexible techniques for integrating segmentation information into existing computer vision applications, such as depth and/or disparity estimation, to yield improved results in a wide range of image capture scenarios. In some embodiments, an optimization framework may be employed to optimize a camera device's initial scene depth/disparity estimates that employs both semantic segmentation and color regularization in a robust fashion.

公开(公告)号：US10762655B1

公开(公告)日：2020-09-01

申请号：US16567581

申请日：2019-09-11

Applicant: Apple Inc.

Inventor： Alexander Lindskog ,  Michael W. Tao ,  Mark N. Jouppi

IPC: H04N13/271 ,  G06T7/571 ,  H04N5/369

Abstract: The disclosure pertains to techniques for image processing. One such technique comprises a method for image processing comprising obtaining first light information from a set of light-sensitive pixels for a scene, the pixels including phase detection (PD) pixels and non-PD pixels, generating a first PD pixel image from the first light information, the first PD pixel image having a first resolution, generating a higher resolution image from the plurality of non-PD pixels, wherein the higher resolution image has a resolution greater than the resolution of the first PD pixel image, matching a first pixel of the first PD pixel image to the higher resolution image, wherein the matching is based on a set of correlations between the first pixel and non-PD pixel within a predetermined distance of the first pixel, and determining a disparity map for an image associated with the first light information, based on the match.