公开(公告)号：US20180097867A1

公开(公告)日：2018-04-05

申请号：US15832023

申请日：2017-12-05

Applicant: Lytro, Inc.

Inventor： Derek Pang ,  Colvin Pitts ,  Kurt Akeley

IPC: H04L29/06 ,  H04N19/124 ,  H04N19/13 ,  H04N19/196 ,  H04N19/44

Abstract: A video stream of a scene for a virtual reality or augmented reality experience may be captured by one or more image capture devices. Data from the video stream may be retrieved, including base vantage data with base vantage color data depicting the scene from a base vantage location, and target vantage data with target vantage color data depicting the scene from a target vantage location. The base vantage data may be reprojected to the target vantage location to obtain reprojected target vantage data. The reprojected target vantage data may be compared with the target vantage data to obtain residual data. The residual data may be compressed by removing a subset of the residual data that is likely to be less viewer-discernable than a remainder of the residual data. A compressed video stream may be stored, including the base vantage data and the compressed residual data.

公开(公告)号：US20180082405A1

公开(公告)日：2018-03-22

申请号：US15824574

申请日：2017-11-28

Applicant: Lytro, Inc.

Inventor： Chia-Kai Liang ,  Kent Oberheu ,  Kurt Akeley ,  Garrett Girod ,  Nikhil Karnad ,  Francis A. Benevides, JR.

IPC: G06T5/00 ,  G06F3/0484

Abstract: An image such as a light-field image may be processed to provide depth-based blurring. The image may be received in a data store. At an input device, first and second user input may be received to designate a first focus depth and a second focus depth different from the first focus depth, respectively. A processor may identify one or more foreground portions of the image that have one or more foreground portion depths, each of which is less than the first focus depth. The processor may also identify one or more background portions of the image that have one or more background portion depths, each of which is greater than the second focus depth. The processor may also apply blurring to the one or more foreground portions and the one or more background portions to generate a processed image, which may be displayed on a display device.

公开(公告)号：US20180070067A1

公开(公告)日：2018-03-08

申请号：US15811425

申请日：2017-11-13

Applicant: Lytro, Inc.

Inventor： Timothy Knight ,  Colvin Pitts ,  Kurt Akeley ,  Yuriy Romanenko ,  Carl (Warren) Craddock

IPC: H04N9/64 ,  H04N9/04 ,  G02B27/00 ,  H04N5/225 ,  H04N9/07

CPC classification number: H04N9/646 ,  G02B27/0075 ,  H04N5/2254 ,  H04N9/04 ,  H04N9/045 ,  H04N9/07

Abstract: According to various embodiments of the present invention, the optical systems of light field capture devices are optimized so as to improve captured light field image data. Optimizing optical systems of light field capture devices can result in captured light field image data (both still and video) that is cheaper and/or easier to process. Optical systems can be optimized to yield improved quality or resolution when using cheaper processing approaches whose computational costs fit within various processing and/or resource constraints. As such, the optical systems of light field cameras can be optimized to reduce size and/or cost and/or increase the quality of such optical systems.

公开(公告)号：US20170365068A1

公开(公告)日：2017-12-21

申请号：US15635894

申请日：2017-06-28

Applicant: Lytro, Inc.

Inventor： Jie Tan ,  Gang Pan ,  Jon Karafin ,  Thomas Nonn ,  Julio C. Hernandez Zaragoza

IPC: G06T7/80 ,  G01S17/08 ,  G06T7/521 ,  G01S17/02 ,  G06T3/00 ,  G06K9/46

CPC classification number: G06T7/80 ,  G01S17/023 ,  G01S17/08 ,  G01S17/42 ,  G06K9/4604 ,  G06T3/0068 ,  G06T7/521 ,  H04N13/232 ,  H04N13/271

Abstract: Depths of one or more objects in a scene may be measured with enhanced accuracy through the use of a light-field camera and a depth sensor. The light-field camera may capture a light-field image of the scene. The depth sensor may capture depth sensor data of the scene. Light-field depth data may be extracted from the light-field image and used, in combination with the sensor depth data, to generate a depth map indicative of distance between the light-field camera and one or more objects in the scene. The depth sensor may be an active depth sensor that transmits electromagnetic energy toward the scene; the electromagnetic energy may be reflected off of the scene and detected by the active depth sensor. The active depth sensor may have a 360° field of view; accordingly, one or more mirrors may be used to direct the electromagnetic energy between the active depth sensor and the scene.

公开(公告)号：US09712820B2

公开(公告)日：2017-07-18

申请号：US15085048

申请日：2016-03-30

Applicant: Lytro, Inc.

Inventor： Chia-Kai Liang

IPC: G06K9/46 ,  H04N19/103 ,  H04N19/176 ,  G02B27/00 ,  H04N19/597 ,  H04N19/147

CPC classification number: H04N19/103 ,  G02B27/0075 ,  H04N19/147 ,  H04N19/176 ,  H04N19/597

Abstract: According to various embodiments, a light-field image may be compressed and/or decompressed to facilitate storage, transmission, or other functions related to the light-field image. A light-field image may be captured by a light-field image capture device having an image sensor and a microlens array. The light-field image may be received in a data store. A processor may generate a first refocus image pool with a plurality of refocus images based on the light-field image. The processor may further use the first refocus image pool to compress the light-field image to generate a bitstream, smaller than the light-field image, which is representative of the light-field image. The processor or a different processor may also be used to generate a second refocus image pool with a second plurality of images based on the bitstream. The second refocus image pool may be used to decompress the bitstream to generate a reconstructed light-field image.

公开(公告)号：US09628684B2

公开(公告)日：2017-04-18

申请号：US14573319

申请日：2014-12-17

Applicant: Lytro, Inc.

Inventor： Chia-Kai Liang ,  Colvin Pitts ,  Carl (Warren) Craddock ,  Graham Butler Myhre

IPC: H04N5/217 ,  H04N5/225 ,  H04N17/00 ,  H04N5/357 ,  G02B27/00

CPC classification number: H04N5/2254 ,  G02B27/0075 ,  H04N5/3572 ,  H04N17/002

Abstract: According to various embodiments, the system and method disclosed herein serve to at least partially compensate for departures of an actual main lens of a light-field camera from the properties of an ideal main lens. Light-field data may be captured and processed through the use of product calibration data and unit calibration data. The product calibration data may be descriptive of departure of a main lens design of the light-field camera from an ideal main lens design. The unit calibration data may be descriptive of departure of the actual main lens of the light-field camera from the main lens design. Corrected light-field data may be generated as a result of the processing, and may be used to generate a light-field image.

公开(公告)号：US20170061635A1

公开(公告)日：2017-03-02

申请号：US14837465

申请日：2015-08-27

Applicant: Lytro, Inc.

Inventor： Kent Oberheu ,  Colvin Pitts ,  Duane Petrovich ,  Ariel Braunstein

IPC: G06T7/00

CPC classification number: G06T7/586 ,  G06T11/60 ,  G06T2207/10012 ,  G06T2207/10028 ,  G06T2215/16

Abstract: An image such as a light-field image may be captured with a light-field image capture device with a microlens array. The image may be received in a data store along with a depth map that indicates depths at which objects in different portions of the image are disposed. A function may be applied to the depth map to generate a mask that defines a gradual transition between the different depths. An effect may be applied to the image through the use of the mask such that applicability of the effect is determined by the mask. A processed image may be generated. The first effect may be present in the processed image, as applied previously. The processed image may be displayed on a display device. If desired, multiple effects may be applied through the generation of multiple masks, depth maps, and/or intermediate images prior to generation of the processed image.

Abstract translation: 诸如光场图像的图像可以用具有微透镜阵列的光场图像捕获装置捕获。 可以在数据存储器中接收图像以及指示图像的不同部分中的对象被布置的深度的深度图。 可以将功能应用于深度图以生成限定不同深度之间的渐变的掩模。 可以通过使用掩模对图像施加效果，使得效果的适用性由掩模确定。 可以生成经处理的图像。 如前所述，第一效果可能存在于经处理的图像中。 经处理的图像可以显示在显示装置上。 如果需要，可以在生成处理的图像之前通过生成多个掩模，深度图和/或中间图像来应用多个效果。

公开(公告)号：US20170026592A1

公开(公告)日：2017-01-26

申请号：US14809022

申请日：2015-07-24

Applicant: Lytro, Inc.

Inventor： Jiangtao Kuang ,  Chia-Kai Liang

IPC: H04N5/357 ,  H04N5/217

CPC classification number: H04N5/3572 ,  H04N5/23212

Abstract: According to various embodiments, the system and method disclosed herein process light-field image data so as to mitigate lens flare effects. A light-field image may be captured with a light-field image capture device with a microlens array and received in a data store. A plurality of flare-affected pixels may be identified in the light-field image. The flare-affected pixels may have flare-affected pixel values. Flare-corrected pixel values may be generated for the flare-affected pixels. Relative to the flare-affected pixel values, the flare-corrected pixel values may at least partially remove the lens flare effects. The flare-corrected pixel values may be used to generate a corrected light-field image in which the lens flare effects are at least partially corrected. The corrected light-field image may be displayed on a display screen.

Abstract translation: 根据各种实施例，本文所公开的系统和方法处理光场图像数据，以减轻透镜闪光效应。 可以用具有微透镜阵列并在数据存储中接收的光场图像捕获装置捕获光场图像。 可以在光场图像中识别多个受耀斑影响的像素。 受耀斑影响的像素可能具有耀斑影响的像素值。 可以为受耀斑影响的像素产生闪光校正的像素值。 相对于受闪耀影响的像素值，闪光校正的像素值可以至少部分地去除透镜闪光效果。 闪光校正的像素值可以用于生成其中至少部分校正透镜闪光效果的校正的光场图像。 校正的光场图像可以显示在显示屏幕上。

公开(公告)号：US20160353082A1

公开(公告)日：2016-12-01

申请号：US15142565

申请日：2016-04-29

Applicant: Lytro, Inc.

Inventor： Colvin Pitts ,  Chia-Kai Liang ,  Kurt Akeley

IPC: H04N13/02 ,  G02B3/00 ,  H04N13/00

CPC classification number: H04N13/229 ,  G02B3/005 ,  G02B3/0056 ,  H04N13/218 ,  H04N13/232

Abstract: A light-field camera may generate four-dimensional light-field data indicative of incoming light. The light-field camera may have an aperture configured to receive the incoming light, an image sensor, and a microlens array configured to redirect the incoming light at the image sensor. The image sensor may receive the incoming light and, based on the incoming light, generate the four-dimensional light-field data, which may have first and second spatial dimensions and first and second angular dimensions. The first angular dimension may have a first resolution higher than a second resolution of the second angular dimension.

Abstract translation: 光场照相机可以产生指示入射光的四维光场数据。 光场摄像机可以具有被配置为接收入射光的孔径，图像传感器和被配置为将入射光重定向在图像传感器处的微透镜阵列。 图像传感器可以接收入射光，并且基于入射光产生可以具有第一和第二空间维度以及第一和第二角度尺寸的四维光场数据。 第一角度尺寸可以具有比第二角度尺寸的第二分辨率高的第一分辨率。

公开(公告)号：US09420276B2

公开(公告)日：2016-08-16

申请号：US14854493

申请日：2015-09-15

Applicant: Lytro, Inc.

Inventor： Chia-Kai Liang ,  Zejing Wang

IPC: G06K7/00 ,  H04N17/00 ,  G06T7/00 ,  G02B3/00 ,  H04N5/225 ,  G02B27/00

CPC classification number: H04N17/002 ,  G02B3/0056 ,  G02B27/0075 ,  G06T7/80 ,  H04N5/2254

Abstract: Microlens positions for a light-field capture device may be calibrated. A calibration light-field image may be captured, with a microlens portion corresponding to each microlens of the light-field capture device. Interstitial spaces between the microlens portions may be identified and used to locate one or more center locations of the microlens portions. The center locations may be used to generate a model that indicates the microlens positions. Additionally or alternatively, the calibration light field image may be used to select one or more contour samples from among multiple contour samples of the microlens portions. The contour sample may be fitted to a circle centered at a center location of a microlens portion to identify the center location, which may then be used to generate a model that indicates the microlens positions. Multiple iterations may be used to enhance the accuracy of the models.

Abstract translation: 可以校准用于光场捕获装置的微透镜位置。 可以捕获校准光场图像，其中微透镜部分对应于光场捕获装置的每个微透镜。 可以识别微透镜部分之间的间隙空间，并用于定位微透镜部分的一个或多个中心位置。 中心位置可用于产生指示微透镜位置的模型。 附加地或替代地，校准光场图像可以用于从微透镜部分的多个轮廓样本中选择一个或多个轮廓样本。 轮廓样本可以装配到以微透镜部分的中心位置为中心的圆圈，以识别中心位置，然后可以使用该中心位置来产生指示微透镜位置的模型。 可以使用多次迭代来提高模型的准确性。