公开(公告)号：US09686539B1

公开(公告)日：2017-06-20

申请号：US15180048

申请日：2016-06-12

Applicant: Apple Inc.

Inventor： Marco Zuliani ,  Claus Molgaard ,  Paul M. Hubel

IPC: H04N17/00 ,  H04N17/02 ,  H04N5/247 ,  G06T7/00

CPC classification number: H04N17/002 ,  G06T7/004 ,  G06T7/85 ,  G06T2207/10012 ,  G06T2207/30168 ,  G06T2207/30201 ,  H04N5/247 ,  H04N13/239 ,  H04N13/246

Abstract: Systems, methods, and computer readable media for calibrating two cameras (image capture units) using a non-standard, and initially unknown, calibration object are described. More particularly, an iterative approach to determine the structure and pose of an target object in an unconstrained environment are disclosed. The target object may be any of a number of predetermined objects such as a specific three dimensional (3D) shape, a specific type of animal (e.g., dogs), or the face of an arbitrary human. Virtually any object whose structure may be expressed in terms of a relatively low dimensional parametrized model may be used as a target object. The identified object (i.e., its pose and shape) may be used as input to a bundle adjustment operation resulting in camera calibration.

公开(公告)号：US09626745B2

公开(公告)日：2017-04-18

申请号：US14872104

申请日：2015-09-30

Applicant: Apple Inc.

Inventor： Farhan A. Baqai ,  Fabio Riccardi ,  Russell A. Pflughaupt ,  Claus Molgaard ,  Gijesh Varghese

IPC: G06K9/40 ,  G06T5/00 ,  G06T5/50 ,  G06T5/20 ,  H04N5/217

CPC classification number: H04N9/77 ,  G06K9/40 ,  G06T5/002 ,  G06T5/009 ,  G06T5/20 ,  G06T5/50 ,  G06T2207/10024 ,  G06T2207/20016 ,  G06T2207/20182 ,  G06T2207/20208 ,  G06T2207/20221 ,  H04N5/208 ,  H04N5/217 ,  H04N9/646 ,  H04N9/67 ,  H04N9/69 ,  H04N9/73

Abstract: Systems, methods, and computer readable media to fuse digital images are described. In general, techniques are disclosed that use multi-band noise reduction techniques to represent input and reference images as pyramids. Once decomposed in this manner, images may be fused using novel low-level (noise dependent) similarity measures. In some implementations similarity measures may be based on intra-level comparisons between reference and input images. In other implementations, similarity measures may be based on inter-level comparisons. In still other implementations, mid-level semantic features such as black-level may be used to inform the similarity measure. In yet other implementations, high-level semantic features such as color or a specified type of region (e.g., moving, stationary, or having a face or other specified shape) may be used to inform the similarity measure.

公开(公告)号：US09596423B1

公开(公告)日：2017-03-14

申请号：US14503322

申请日：2014-09-30

Applicant: Apple Inc.

Inventor： Claus Molgaard

IPC: H04N5/372

CPC classification number: H04N9/045 ,  H04N5/347 ,  H04N2209/045

Abstract: An image sensor includes a pixel array having a plurality of pixels. Pixels can be summed or binned diagonally in the pixel array in a first diagonal direction and in a different second diagonal direction. The locations of the first and second diagonal summed pairs can be distributed across the pixel array.

Abstract translation: 图像传感器包括具有多个像素的像素阵列。 像素可以在第一对角线方向和不同的第二对角线方向上在像素阵列中对角地相加或合并。 第一和第二对角线求和对的位置可以分布在像素阵列上。

公开(公告)号：US20160065795A1

公开(公告)日：2016-03-03

申请号：US14474103

申请日：2014-08-30

Applicant: Apple Inc.

Inventor： Farhan A. Baqai ,  Claus Molgaard ,  Fabio Riccardi ,  Xuemei Zhang

IPC: H04N5/217 ,  H04N9/77 ,  G06T5/00 ,  H04N9/04

CPC classification number: H04N5/217 ,  G06T5/002 ,  G06T5/008 ,  G06T2207/20012 ,  G06T2207/20016 ,  G06T2207/20048 ,  H04N9/045

Abstract: Image enhancement is achieved by separating image signals, e.g. YCbCr image signals, into a series of frequency bands and performing locally-adaptive noise reduction on bands below a given frequency but not on bands above that frequency. The bands are summed to develop the image enhanced signals. The YCbCr, multi-band locally-adaptive approach to denoising is able to operate independently—and in an optimized fashion—on both luma and chroma channels. Noise reduction is done based on models developed for both luma and chroma channels by measurements taken for multiple frequency bands, in multiple patches on the ColorChecker chart, and at multiple gain levels, in order to develop a simple yet robust set of models that may be tuned off-line a single time for each camera and then applied to images taken by such cameras in real-time without excessive processing requirements and with satisfactory results across illuminant types and lighting conditions.

Abstract translation: 图像增强通过分离图像信号来实现。 YCbCr图像信号转换成一系列频带，并在给定频率以下的频带上执行局部自适应噪声降低，但不在高于该频率的频带上进行。 将频带相加以开发图像增强信号。 YCbCr多频域局部自适应去噪方法能够在亮度和色度通道上独立运行，并以优化方式运行。 基于通过对ColorChecker图表上的多个频段进行测量，在ColorChecker图表上以多个增益水平测量的亮度和色度通道开发的模型以及多个增益水平来进行降噪，以便开发一个简单但可靠的模型集 为每台摄像机单独调整离线时间，然后实时应用于摄像机所拍摄的图像，而无需过多的处理要求，并且在各种照明类型和照明条件下都能获得令人满意的结果。

公开(公告)号：US20160065794A1

公开(公告)日：2016-03-03

申请号：US14474100

申请日：2014-08-30

Applicant: Apple Inc.

Inventor： Farhan A. Baqai ,  Claus Molgaard ,  Fabio Riccardi ,  Russell Pflughaupt

IPC: H04N5/217 ,  G06T5/00 ,  H04N9/077

CPC classification number: H04N5/217 ,  G06T5/002 ,  G06T5/10 ,  G06T2207/10024 ,  G06T2207/20016 ,  G06T2207/20048 ,  H04N5/21 ,  H04N9/077 ,  H04N9/646

Abstract: Image enhancement is achieved by separating image signals, e.g. YCbCr image signals, into a series of frequency bands and performing noise reduction on bands below a given frequency but not on bands above that frequency. The bands are summed to develop the image enhanced signals. The YCbCr, multi-band approach to denoising is able to operate independently—and in an optimized fashion—on both luma and chroma channels. Noise reduction is done based on models developed for both luma and chroma channels by measurements taken for multiple frequency bands, in multiple patches on the ColorChecker chart, and at multiple gain levels, in order to develop a simple yet robust set of models that may be tuned off-line a single time for each camera and then applied to images taken by such cameras in real-time without excessive processing requirements and with satisfactory results across illuminant types and lighting conditions.

Abstract translation: 图像增强通过分离图像信号来实现。 YCbCr图像信号转换成一系列频带，并在低于给定频率但不超过该频率的频带上进行降噪。 将频带相加以开发图像增强信号。 YCbCr多频去噪方法能够在亮度和色度通道上独立运行，并以优化方式运行。 基于通过对ColorChecker图表上的多个频段进行测量，在ColorChecker图表上以多个增益水平测量的亮度和色度通道开发的模型以及多个增益水平来进行降噪，以便开发一个简单但可靠的模型集 为每台摄像机单独调整离线时间，然后实时应用于摄像机所拍摄的图像，而无需过多的处理要求，并且在各种照明类型和照明条件下都能获得令人满意的结果。

公开(公告)号：US10284835B2

公开(公告)日：2019-05-07

申请号：US14864650

申请日：2015-09-24

Applicant: Apple Inc.

Inventor： Thomas E. Bishop ,  Alexander Lindskog ,  Claus Molgaard ,  Frank Doepke

IPC: H04N13/106 ,  G06T5/00 ,  H04N5/235 ,  H04N5/232 ,  G06T7/593

Abstract: Generating an image with a selected level of background blur includes capturing, by a first image capture device, a plurality of frames of a scene, wherein each of the plurality of frames has a different focus depth, obtaining a depth map of the scene, determining a target object and a background in the scene based on the depth map, determining a goal blur for the background, and selecting, for each pixel in an output image, a corresponding pixel from the focus stack.

公开(公告)号：US20160007008A1

公开(公告)日：2016-01-07

申请号：US14788386

申请日：2015-06-30

Applicant: APPLE INC.

Inventor： Claus Molgaard ,  Iain A. McAllister

IPC: H04N13/02 ,  H04N3/14 ,  G03B13/34

CPC classification number: G03B13/34 ,  H04N3/1593 ,  H04N5/2258 ,  H04N5/2259 ,  H04N5/23212 ,  H04N5/23216 ,  H04N5/23293 ,  H04N5/23296 ,  H04N5/2628

Abstract: Some embodiments include methods and/or systems for using multiple cameras to provide optical zoom to a user. Some embodiments include a first camera unit of a multifunction device capturing a first image of a first visual field. A second camera unit of the multifunction device simultaneously captures a second image of a second visual field. In some embodiments, the first camera unit includes a first optical package with a first focal length. In some embodiments, the second camera unit includes a second optical package with a second focal length. In some embodiments, the first focal length is different from the second focal length, and the first visual field is a subset of the second visual field. In some embodiments, the first image and the second image are preserved to a storage medium as separate data structures.

Abstract translation: 一些实施例包括用于使用多个相机向用户提供光学变焦的方法和/或系统。 一些实施例包括捕获第一视野的第一图像的多功能装置的第一相机单元。 多功能设备的第二摄像机单元同时捕获第二视野的第二图像。 在一些实施例中，第一相机单元包括具有第一焦距的第一光学封装。 在一些实施例中，第二相机单元包括具有第二焦距的第二光学封装。 在一些实施例中，第一焦距与第二焦距不同，第一视野是第二视野的子集。 在一些实施例中，将第一图像和第二图像作为单独的数据结构保存到存储介质。

公开(公告)号：US20150350575A1

公开(公告)日：2015-12-03

申请号：US14292599

申请日：2014-05-30

Applicant: Apple Inc.

Inventor： Gennadiy A. Agranov ,  Claus Molgaard ,  Ashirwad Bahukhandi ,  Chiajen Lee ,  Xiangli Li

IPC: H04N5/345

CPC classification number: H04N9/04511 ,  H04N5/347 ,  H04N5/37457 ,  H04N9/045

Abstract: Pixel binning is performed by summing charge from some pixels positioned diagonally in a pixel array. Pixel signals output from pixels positioned diagonally in the pixel array may be combined on the output lines. A signal representing summed charge produces a binned 2×1 cluster. A signal representing combined voltage signals produces a binned 2×1 cluster. A signal representing summed charge and a signal representing combined pixel signals can be combined digitally to produce a binned 2×2 pixel. Orthogonal binning may be performed on other pixels in the pixel array by summing charge on respective common sense regions and then then combining the voltage signals that represent the summed charge on respective output lines.

Abstract translation: 通过对来自像素阵列中的对角定位的一些像素的电荷求和来执行像素合并。 从位于像素阵列中对角线的像素输出的像素信号可以组合在输出线上。 表示相加电荷的信号产生一个二进制的2×1簇。 表示组合电压信号的信号产生一个二进制的2×1簇。 表示合并的电荷的信号和表示组合的像素信号的信号可以数字地组合以产生二进制的2×2像素。 可以通过对相应的公共感测区域上的电荷进行相加，然后将表示相加的输出线上的相加电荷的电压信号合并，来对像素阵列中的其他像素执行正交分样。

公开(公告)号：US09092853B2

公开(公告)日：2015-07-28

申请号：US14513046

申请日：2014-10-13

Applicant: Apple Inc.

Inventor： Todd S. Sachs ,  Kevin L. Hunter ,  Marco Zuliani ,  Claus Molgaard

IPC: H04N17/00 ,  G06T7/00 ,  H04N5/232

CPC classification number: G06T7/0018 ,  G06T7/80 ,  G06T2207/10004 ,  H04N5/23216 ,  H04N5/23293 ,  H04N17/002

Abstract: A method for dynamically calibrating rotational offset in a device includes obtaining an image captured by a camera of the device. Orientation information of the device at the time of image capture may be associated with the image. Pixel data of the image may be analyzed to determine an image orientation angle for the image. A device orientation angle may be determined from the orientation information. A rotational offset, based on the image orientation angle and the device orientation angle, may be determined. The rotational offset is relative to the camera or orientation sensor. A rotational bias may be determined from statistical analysis of numerous rotational offsets from numerous respective images. In some embodiments, various thresholds and predetermined ranges may be used to exclude some rotational offsets from the statistical analysis or to discontinue processing for that image.

Abstract translation: 用于动态校准装置中的旋转偏移的方法包括获得由装置的照相机拍摄的图像。 图像拍摄时设备的方向信息可能与图像相关联。 可以分析图像的像素数据以确定图像的图像取向角度。 可以根据取向信息来确定设备取向角度。 可以确定基于图像取向角度和装置取向角度的旋转偏移。 旋转偏移量相对于相机或方位传感器。 旋转偏差可以根据来自大量各自图像的许多旋转偏移的统计分析来确定。 在一些实施例中，可以使用各种阈值和预定范围来从统计分析中排除一些旋转偏移或者中断对该图像的处理。

公开(公告)号：US20140362256A1

公开(公告)日：2014-12-11

申请号：US13911873

申请日：2013-06-06

Applicant: Apple Inc.

Inventor： Anita Nariani Schulze ,  Rolf Toft ,  Paul M. Hubel ,  Marius Tico ,  Jianping Zhou ,  Ralph Brunner ,  Claus Molgaard

IPC: H04N5/232

CPC classification number: H04N5/23277

Abstract: Systems, methods, and computer readable media to improve image stabilization operations are described. A novel combination of image quality and commonality metrics are used to identify a reference frame from a set of commonly captured images which, when the set's other images are combined with it, results in a quality stabilized image. The disclosed image quality and commonality metrics may also be used to optimize the use of a limited amount of image buffer memory during image capture sequences that return more images that the memory may accommodate at one time. Image quality and commonality metrics may also be used to effect the combination of multiple relatively long-exposure images which, when combined with a one or more final (relatively) short-exposure images, yields images exhibiting motion-induced blurring in interesting and visually pleasing ways.

Abstract translation: 描述了用于改善图像稳定操作的系统，方法和计算机可读介质。 使用图像质量和共性度量的新颖组合来从一组共同拍摄的图像中识别参考帧，当集合的其他图像与其组合时，其导致质量稳定的图像。 公开的图像质量和通用性度量还可以用于在图像捕获序列期间优化使用有限量的图像缓冲存储器，从而可以一次返回存储器可以容纳的更多图像。 图像质量和共性度量也可用于影响多个相对较长曝光图像的组合，当与一个或多个最终（相对）短曝光图像组合时，产生显示运动诱导的模糊的图像，其在有趣和视觉上令人满意 方法。