公开(公告)号：US12081721B2

公开(公告)日：2024-09-03

申请号：US17807508

申请日：2022-06-17

Applicant: Adeia Imaging LLC

Inventor： Kartik Venkataraman ,  Dan Lelescu ,  Jacques Duparre

IPC: H04N13/128 ,  G01C11/06 ,  G02B27/01 ,  G06T7/593 ,  H04N13/243 ,  H04N13/271 ,  H04N23/45 ,  H04N23/90 ,  H04N5/222 ,  H04N13/00 ,  H04N23/11 ,  H04N23/56

CPC classification number: H04N13/128 ,  G01C11/06 ,  G02B27/017 ,  G02B27/0172 ,  G06T7/593 ,  H04N13/243 ,  H04N13/271 ,  H04N23/45 ,  H04N23/90 ,  G02B2027/0136 ,  G02B2027/0138 ,  G02B2027/014 ,  G06T2207/10012 ,  G06T2207/10028 ,  G06T2207/20016 ,  H04N5/2226 ,  H04N2013/0081 ,  H04N23/11 ,  H04N23/56

Abstract: Embodiments of the invention provide a camera array imaging architecture that computes depth maps for objects within a scene captured by the cameras, and use a near-field sub-array of cameras to compute depth to near-field objects and a far-field sub-array of cameras to compute depth to far-field objects. In particular, a baseline distance between cameras in the near-field subarray is less than a baseline distance between cameras in the far-field sub-array in order to increase the accuracy of the depth map. Some embodiments provide an illumination near-IR light source for use in computing depth maps.

公开(公告)号：US12002233B2

公开(公告)日：2024-06-04

申请号：US17162542

申请日：2021-01-29

Applicant: Adeia Imaging LLC

Inventor： Florian Ciurea ,  Kartik Venkataraman ,  Gabriel Molina ,  Dan Lelescu

IPC: G06T7/593 ,  G02B27/00 ,  G06T7/557 ,  G06T7/80 ,  G06T15/20 ,  H04N13/128 ,  H04N13/232 ,  H04N13/243 ,  H04N23/16 ,  H04N13/00

CPC classification number: G06T7/593 ,  G02B27/0075 ,  G06T7/557 ,  G06T7/85 ,  G06T15/20 ,  H04N13/128 ,  H04N13/232 ,  H04N13/243 ,  H04N23/16 ,  G06T2200/21 ,  G06T2207/10012 ,  G06T2207/10024 ,  G06T2207/10052 ,  H04N2013/0081 ,  H04N2013/0088

Abstract: Systems in accordance with embodiments of the invention can perform parallax detection and correction in images captured using array cameras. Due to the different viewpoints of the cameras, parallax results in variations in the position of objects within the captured images of the scene. Methods in accordance with embodiments of the invention provide an accurate account of the pixel disparity due to parallax between the different cameras in the array, so that appropriate scene-dependent geometric shifts can be applied to the pixels of the captured images when performing super-resolution processing. In a number of embodiments, generating depth estimates considers the similarity of pixels in multiple spectral channels. In certain embodiments, generating depth estimates involves generating a confidence map indicating the reliability of depth estimates.

公开(公告)号：US11941833B2

公开(公告)日：2024-03-26

申请号：US17162542

申请日：2021-01-29

Applicant: Adeia Imaging LLC

Inventor： Florian Ciurea ,  Kartik Venkataraman ,  Gabriel Molina ,  Dan Lelescu

IPC: G06T7/593 ,  G02B27/00 ,  G06T7/557 ,  G06T7/80 ,  G06T15/20 ,  H04N13/128 ,  H04N13/232 ,  H04N13/243 ,  H04N23/16 ,  H04N13/00

CPC classification number: G06T7/593 ,  G02B27/0075 ,  G06T7/557 ,  G06T7/85 ,  G06T15/20 ,  H04N13/128 ,  H04N13/232 ,  H04N13/243 ,  H04N23/16 ,  G06T2200/21 ,  G06T2207/10012 ,  G06T2207/10024 ,  G06T2207/10052 ,  H04N2013/0081 ,  H04N2013/0088

Abstract: Systems in accordance with embodiments of the invention can perform parallax detection and correction in images captured using array cameras. Due to the different viewpoints of the cameras, parallax results in variations in the position of objects within the captured images of the scene. Methods in accordance with embodiments of the invention provide an accurate account of the pixel disparity due to parallax between the different cameras in the array, so that appropriate scene-dependent geometric shifts can be applied to the pixels of the captured images when performing super-resolution processing. In a number of embodiments, generating depth estimates considers the similarity of pixels in multiple spectral channels. In certain embodiments, generating depth estimates involves generating a confidence map indicating the reliability of depth estimates.

公开(公告)号：US11875475B2

公开(公告)日：2024-01-16

申请号：US17821258

申请日：2022-08-22

Applicant: Adeia Imaging LLC

Inventor： Dan Lelescu ,  Gabriel Molina ,  Kartik Venkataraman

IPC: G06T3/40 ,  H04N23/951 ,  H04N13/106 ,  H04N13/111 ,  H04N13/128 ,  H04N13/139 ,  H04N23/62 ,  G06T5/00 ,  G06T5/50 ,  H04N13/156 ,  H04N13/204 ,  H04N13/246 ,  H04N13/271 ,  G06T5/20

CPC classification number: G06T3/4053 ,  G06T3/4007 ,  G06T3/4076 ,  G06T5/001 ,  G06T5/006 ,  G06T5/50 ,  H04N13/106 ,  H04N13/111 ,  H04N13/128 ,  H04N13/139 ,  H04N23/62 ,  H04N23/951 ,  G06T5/002 ,  G06T5/20 ,  G06T2207/10004 ,  G06T2207/10012 ,  G06T2207/20182 ,  G06T2207/20192 ,  G06T2207/20221 ,  H04N13/156 ,  H04N13/204 ,  H04N13/246 ,  H04N13/271

Abstract: Systems and methods in accordance with embodiments of the invention are disclosed that use super-resolution (SR) processes to use information from a plurality of low resolution (LR) images captured by an array camera to produce a synthesized higher resolution image. One embodiment includes obtaining input images using the plurality of imagers, using a microprocessor to determine an initial estimate of at least a portion of a high resolution image using a plurality of pixels from the input images, and using a microprocessor to determine a high resolution image that when mapped through the forward imaging transformation matches the input images to within at least one predetermined criterion using the initial estimate of at least a portion of the high resolution image. In addition, each forward imaging transformation corresponds to the manner in which each imager in the imaging array generate the input images, and the high resolution image synthesized by the microprocessor has a resolution that is greater than any of the input images.

公开(公告)号：US20240333901A1

公开(公告)日：2024-10-03

申请号：US18742982

申请日：2024-06-13

Applicant: Adeia Imaging LLC

Inventor： Kartik Venkataraman ,  Semyon Nisenzon ,  Dan Lelescu

IPC: H04N13/178 ,  G06T3/4007 ,  G06T3/4053 ,  G06T7/50 ,  G06T7/593 ,  G06T9/00 ,  G06T9/20 ,  G06T15/08 ,  H04N13/00 ,  H04N13/128 ,  H04N13/161 ,  H04N13/243 ,  H04N13/271 ,  H04N19/136 ,  H04N19/597 ,  H04N19/625 ,  H04N19/85

CPC classification number: H04N13/178 ,  G06T3/4007 ,  G06T3/4053 ,  G06T7/50 ,  G06T7/596 ,  G06T9/00 ,  G06T9/20 ,  G06T15/08 ,  H04N13/128 ,  H04N13/161 ,  H04N13/243 ,  H04N13/271 ,  H04N19/136 ,  H04N19/597 ,  H04N19/625 ,  H04N19/85 ,  G06T2207/10052 ,  H04N2013/0081

Abstract: Systems and methods in accordance with embodiments of the invention are configured to decode images containing an image of a scene and a corresponding depth map. A depth-based effect is applied to the image to generate a synthetic image of the scene. The synthetic image can be encoded into a new image file that contains metadata associated with the depth-based effect. In many embodiments, the original decoded image has a different depth-based effect applied to it with respect to the synthetic image.

公开(公告)号：US12052409B2

公开(公告)日：2024-07-30

申请号：US18339775

申请日：2023-06-22

Applicant: Adeia Imaging LLC

Inventor： Kartik Venkataraman ,  Semyon Nisenzon ,  Dan Lelescu

IPC: G06V10/00 ,  G06T3/4007 ,  G06T3/4053 ,  G06T7/50 ,  G06T7/593 ,  G06T9/00 ,  G06T9/20 ,  G06T15/08 ,  H04N13/128 ,  H04N13/161 ,  H04N13/178 ,  H04N13/243 ,  H04N13/271 ,  H04N19/136 ,  H04N19/597 ,  H04N19/625 ,  H04N19/85 ,  H04N13/00

CPC classification number: H04N13/178 ,  G06T3/4007 ,  G06T3/4053 ,  G06T7/50 ,  G06T7/596 ,  G06T9/00 ,  G06T9/20 ,  G06T15/08 ,  H04N13/128 ,  H04N13/161 ,  H04N13/243 ,  H04N13/271 ,  H04N19/136 ,  H04N19/597 ,  H04N19/625 ,  H04N19/85 ,  G06T2207/10052 ,  H04N2013/0081

Abstract: Systems and methods in accordance with embodiments of the invention are configured to render images using light field image files containing an image synthesized from light field image data and metadata describing the image that includes a depth map. One embodiment of the invention includes a processor and memory containing a rendering application and a light field image file including an encoded image, a set of low resolution images, and metadata describing the encoded image, where the metadata comprises a depth map that specifies depths from the reference viewpoint for pixels in the encoded image. In addition, the rendering application configures the processor to: locate the encoded image within the light field image file; decode the encoded image; locate the metadata within the light field image file; and post process the decoded image by modifying the pixels based on the depths indicated within the depth map and the set of low resolution images to create a rendered image.

公开(公告)号：US20240169483A1

公开(公告)日：2024-05-23

申请号：US18504684

申请日：2023-11-08

Applicant: Adeia Imaging LLC

Inventor： Dan Lelescu ,  Gabriel Molina ,  Kartik Venkataraman

IPC: G06T3/4053 ,  G06T3/4007 ,  G06T3/4076 ,  G06T5/00 ,  G06T5/50 ,  G06T5/80 ,  H04N13/106 ,  H04N13/111 ,  H04N13/128 ,  H04N13/139 ,  H04N23/62 ,  H04N23/951

CPC classification number: G06T3/4053 ,  G06T3/4007 ,  G06T3/4076 ,  G06T5/00 ,  G06T5/50 ,  G06T5/80 ,  H04N13/106 ,  H04N13/111 ,  H04N13/128 ,  H04N13/139 ,  H04N23/62 ,  H04N23/951 ,  H04N13/156

Abstract: Systems and methods in accordance with embodiments of the invention are disclosed that use super-resolution (SR) processes to use information from a plurality of low resolution (LR) images captured by an array camera to produce a synthesized higher resolution image. One embodiment includes obtaining input images using the plurality of imagers, using a microprocessor to determine an initial estimate of at least a portion of a high resolution image using a plurality of pixels from the input images, and using a microprocessor to determine a high resolution image that when mapped through the forward imaging transformation matches the input images to within at least one predetermined criterion using the initial estimate of at least a portion of the high resolution image. In addition, each forward imaging transformation corresponds to the manner in which each imager in the imaging array generate the input images, and the high resolution image synthesized by the microprocessor has a resolution that is greater than any of the input images.

公开(公告)号：US11546576B2

公开(公告)日：2023-01-03

申请号：US17195332

申请日：2021-03-08

Applicant: Adeia Imaging LLC

Inventor： Florian Ciurea ,  Dan Lelescu ,  Priyam Chatterjee

IPC: H04N17/00 ,  G06T7/80 ,  G06T5/00 ,  H04N5/247

Abstract: Systems and methods for dynamically calibrating an array camera to accommodate variations in geometry that can occur throughout its operational life are disclosed. The dynamic calibration processes can include acquiring a set of images of a scene and identifying corresponding features within the images. Geometric calibration data can be used to rectify the images and determine residual vectors for the geometric calibration data at locations where corresponding features are observed. The residual vectors can then be used to determine updated geometric calibration data for the camera array. In several embodiments, the residual vectors are used to generate a residual vector calibration data field that updates the geometric calibration data. In many embodiments, the residual vectors are used to select a set of geometric calibration from amongst a number of different sets of geometric calibration data that is the best fit for the current geometry of the camera array.

公开(公告)号：US20240331181A1

公开(公告)日：2024-10-03

申请号：US18434684

申请日：2024-02-06

Applicant: Adeia Imaging LLC

Inventor： Florian Ciurea ,  Kartik Venkataraman ,  Gabriel Molina ,  Dan Lelescu

IPC: G06T7/593 ,  G02B27/00 ,  G06T7/557 ,  G06T7/80 ,  G06T15/20 ,  H04N13/00 ,  H04N13/128 ,  H04N13/232 ,  H04N13/243 ,  H04N23/16

CPC classification number: G06T7/593 ,  G02B27/0075 ,  G06T7/557 ,  G06T7/85 ,  G06T15/20 ,  H04N13/128 ,  H04N13/232 ,  H04N13/243 ,  H04N23/16 ,  G06T2200/21 ,  G06T2207/10012 ,  G06T2207/10024 ,  G06T2207/10052 ,  H04N2013/0081 ,  H04N2013/0088

Abstract: Systems in accordance with embodiments of the invention can perform parallax detection and correction in images captured using array cameras. Due to the different viewpoints of the cameras, parallax results in variations in the position of objects within the captured images of the scene. Methods in accordance with embodiments of the invention provide an accurate account of the pixel disparity due to parallax between the different cameras in the array, so that appropriate scene-dependent geometric shifts can be applied to the pixels of the captured images when performing super-resolution processing. In a number of embodiments, generating depth estimates considers the similarity of pixels in multiple spectral channels. In certain embodiments, generating depth estimates involves generating a confidence map indicating the reliability of depth estimates.

公开(公告)号：US20230421742A1

公开(公告)日：2023-12-28

申请号：US18339775

申请日：2023-06-22

Applicant: Adeia Imaging LLC

Inventor： Kartik Venkataraman ,  Semyon Nisenzon ,  Dan Lelescu

IPC: H04N13/178 ,  H04N13/128 ,  H04N13/161 ,  H04N13/243 ,  H04N13/271 ,  G06T7/50 ,  G06T9/00 ,  G06T9/20 ,  H04N19/597 ,  H04N19/625 ,  H04N19/136 ,  G06T3/40 ,  H04N19/85 ,  G06T15/08 ,  G06T7/593

CPC classification number: H04N13/178 ,  H04N13/128 ,  H04N13/161 ,  H04N13/243 ,  H04N13/271 ,  G06T7/50 ,  G06T9/00 ,  G06T9/20 ,  H04N19/597 ,  H04N19/625 ,  H04N19/136 ,  G06T3/4007 ,  G06T3/4053 ,  H04N19/85 ,  G06T15/08 ,  G06T7/596 ,  H04N2013/0081

Abstract: Systems and methods in accordance with embodiments of the invention are configured to render images using light field image files containing an image synthesized from light field image data and metadata describing the image that includes a depth map. One embodiment of the invention includes a processor and memory containing a rendering application and a light field image file including an encoded image, a set of low resolution images, and metadata describing the encoded image, where the metadata comprises a depth map that specifies depths from the reference viewpoint for pixels in the encoded image. In addition, the rendering application configures the processor to: locate the encoded image within the light field image file; decode the encoded image; locate the metadata within the light field image file; and post process the decoded image by modifying the pixels based on the depths indicated within the depth map and the set of low resolution images to create a rendered image.