公开(公告)号：US20170069060A1

公开(公告)日：2017-03-09

申请号：US14872104

申请日：2015-09-30

Applicant: Apple Inc.

Inventor： Farhan A. Baqai ,  Fabio Riccardi ,  Russell A. Pflughaupt ,  Claus Molgaard ,  Gijesh Varghese

IPC: G06T5/00 ,  G06T5/20 ,  G06T5/50

CPC classification number: H04N9/77 ,  G06K9/40 ,  G06T5/002 ,  G06T5/009 ,  G06T5/20 ,  G06T5/50 ,  G06T2207/10024 ,  G06T2207/20016 ,  G06T2207/20182 ,  G06T2207/20208 ,  G06T2207/20221 ,  H04N5/208 ,  H04N5/217 ,  H04N9/646 ,  H04N9/67 ,  H04N9/69 ,  H04N9/73

Abstract: Systems, methods, and computer readable media to fuse digital images are described. In general, techniques are disclosed that use multi-band noise reduction techniques to represent input and reference images as pyramids. Once decomposed in this manner, images may be fused using novel low-level (noise dependent) similarity measures. In some implementations similarity measures may be based on intra-level comparisons between reference and input images. In other implementations, similarity measures may be based on inter-level comparisons. In still other implementations, mid-level semantic features such as black-level may be used to inform the similarity measure. In yet other implementations, high-level semantic features such as color or a specified type of region (e.g., moving, stationary, or having a face or other specified shape) may be used to inform the similarity measure.

Abstract translation: 描述了融合数字图像的系统，方法和计算机可读介质。 通常，公开了使用多频带降噪技术将输入和参考图像表示为金字塔的技术。 一旦以这种方式分解，可以使用新颖的低级（噪声依赖）相似性度量来融合图像。 在一些实现中，相似性度量可以基于参考和输入图像之间的层内比较。 在其他实现中，相似性度量可以基于层间比较。 在其他实现中，可以使用诸如黑色级别的中级语义特征来通知相似性度量。 在其他实现中，可以使用诸如颜色或指定类型的区域（例如，移动，静止或具有面部或其他指定形状）的高级语义特征来通知相似性度量。

公开(公告)号：US09344638B2

公开(公告)日：2016-05-17

申请号：US14292792

申请日：2014-05-30

Applicant: Apple Inc.

Inventor： Xuemei Zhang ,  Marius Tico ,  Yingjun Bai ,  Farhan A. Baqai ,  Fabio Riccardi ,  Juanqing Chen ,  Paul M. Hubel ,  George E. Williams ,  Todd S. Sachs ,  Yongfang Liang

IPC: H04N5/225 ,  H04N5/235 ,  G03B7/00 ,  H04N3/14 ,  H04N5/335 ,  H04N5/228 ,  H04N5/91 ,  G06K9/46 ,  G06T5/00 ,  H04N5/265 ,  H04N5/232 ,  G06T5/40

CPC classification number: H04N5/2355 ,  G06K9/4642 ,  G06K9/4661 ,  G06T5/001 ,  G06T5/009 ,  G06T5/40 ,  G06T2207/10144 ,  G06T2207/20016 ,  G06T2207/20208 ,  G06T2207/20221 ,  H04N5/23245 ,  H04N5/265 ,  H04N5/91

Abstract: Systems, methods, and computer readable media to capture and process high dynamic range (HDR) images when appropriate for a scene are disclosed. When appropriate, multiple images at a single—slightly underexposed—exposure value are captured (making a constant bracket HDR capture sequence) and local tone mapping (LTM) applied to each image. Local tone map and histogram information can be used to generate a noise-amplification mask which can be used during fusion operations. Images obtained and fused in the disclosed manner provide high dynamic range with improved noise and de-ghosting characteristics.

Abstract translation: 公开了适用于场景时捕获和处理高动态范围（HDR）图像的系统，方法和计算机可读介质。 在适当的情况下，捕获单个稍微曝光不足的曝光值的多个图像（使恒定的支架HDR捕获序列）和本地色调映射（LTM）应用于每个图像。 本地色调图和直方图信息可用于生成可在融合操作期间使用的噪声放大掩模。 以所公开的方式获得和融合的图像提供高动态范围，具有改进的噪声和去重影特性。

公开(公告)号：US09667842B2

公开(公告)日：2017-05-30

申请号：US14474103

申请日：2014-08-30

Applicant: Apple Inc.

Inventor： Farhan A. Baqai ,  Claus Molgaard ,  Fabio Riccardi ,  Xuemei Zhang

IPC: H04N5/217 ,  H04N9/04 ,  H04N9/77 ,  G06T5/00

CPC classification number: H04N5/217 ,  G06T5/002 ,  G06T5/008 ,  G06T2207/20012 ,  G06T2207/20016 ,  G06T2207/20048 ,  H04N9/045

Abstract: Image enhancement is achieved by separating image signals, e.g. YCbCr image signals, into a series of frequency bands and performing locally-adaptive noise reduction on bands below a given frequency but not on bands above that frequency. The bands are summed to develop the image enhanced signals. The YCbCr, multi-band locally-adaptive approach to denoising is able to operate independently—and in an optimized fashion—on both luma and chroma channels. Noise reduction is done based on models developed for both luma and chroma channels by measurements taken for multiple frequency bands, in multiple patches on the ColorChecker chart, and at multiple gain levels, in order to develop a simple yet robust set of models that may be tuned off-line a single time for each camera and then applied to images taken by such cameras in real-time without excessive processing requirements and with satisfactory results across illuminant types and lighting conditions.

公开(公告)号：US09641820B2

公开(公告)日：2017-05-02

申请号：US14872098

申请日：2015-09-30

Applicant: Apple Inc.

Inventor： Farhan A. Baqai ,  Fabio Riccardi ,  Russell A. Pflughaupt ,  Claus Molgaard ,  Gijesh Varghese

IPC: H04N9/77 ,  H04N9/64 ,  H04N9/67 ,  H04N9/73 ,  H04N9/69

CPC classification number: H04N9/77 ,  G06K9/40 ,  G06T5/002 ,  G06T5/009 ,  G06T5/20 ,  G06T5/50 ,  G06T2207/10024 ,  G06T2207/20016 ,  G06T2207/20182 ,  G06T2207/20208 ,  G06T2207/20221 ,  H04N5/208 ,  H04N5/217 ,  H04N9/646 ,  H04N9/67 ,  H04N9/69 ,  H04N9/73

Abstract: Techniques for de-noising a digital image using a multi-band noise filter and a unique combination of texture and chroma metrics are described. A novel texture metric may be used during multi-band filter operations on an image's luma channel to determine if a given pixel is associated with a textured/smooth region of the image. A novel chroma metric may be used during the same multi-band filter operation to determine if the same pixel is associated with a blue/not-blue region of the image. Pixels identified as being associated with a smooth blue region may be aggressively de-noised and conservatively sharpened. Pixels identified as being associated with a textured blue region may be conservatively de-noised and aggressively sharpened. By coupling texture and chroma constraints it has been shown possible to mitigate noise in an image's smooth blue regions without affecting the edges/texture in other blue objects.

公开(公告)号：US09525804B2

公开(公告)日：2016-12-20

申请号：US14474100

申请日：2014-08-30

Applicant: Apple Inc.

Inventor： Farhan A. Baqai ,  Claus Molgaard ,  Fabio Riccardi ,  Russell Pflughaupt

IPC: H04N9/077 ,  H04N5/217 ,  G06T5/00 ,  H04N9/64 ,  G06T5/10 ,  H04N5/21

CPC classification number: H04N5/217 ,  G06T5/002 ,  G06T5/10 ,  G06T2207/10024 ,  G06T2207/20016 ,  G06T2207/20048 ,  H04N5/21 ,  H04N9/077 ,  H04N9/646

Abstract: Image enhancement is achieved by separating image signals, e.g. YCbCr image signals, into a series of frequency bands and performing noise reduction on bands below a given frequency but not on bands above that frequency. The bands are summed to develop the image enhanced signals. The YCbCr, multi-band approach to denoising is able to operate independently—and in an optimized fashion—on both luma and chroma channels. Noise reduction is done based on models developed for both luma and chroma channels by measurements taken for multiple frequency bands, in multiple patches on the ColorChecker chart, and at multiple gain levels, in order to develop a simple yet robust set of models that may be tuned off-line a single time for each camera and then applied to images taken by such cameras in real-time without excessive processing requirements and with satisfactory results across illuminant types and lighting conditions.

Abstract translation: 图像增强通过分离图像信号来实现。 YCbCr图像信号转换成一系列频带，并在低于给定频率但不超过该频率的频带上进行降噪。 将频带相加以开发图像增强信号。 YCbCr多频去噪方法能够在亮度和色度通道上独立运行，并以优化方式运行。 基于通过对ColorChecker图表上的多个频段进行测量，在ColorChecker图表上以多个增益水平测量的亮度和色度通道开发的模型以及多个增益水平来进行降噪，以便开发一个简单但可靠的模型集 为每台摄像机单独调整离线时间，然后实时应用于摄像机所拍摄的图像，而无需过多的处理要求，并且在各种照明类型和照明条件下都能获得令人满意的结果。

公开(公告)号：US09626745B2

公开(公告)日：2017-04-18

申请号：US14872104

申请日：2015-09-30

Applicant: Apple Inc.

Inventor： Farhan A. Baqai ,  Fabio Riccardi ,  Russell A. Pflughaupt ,  Claus Molgaard ,  Gijesh Varghese

IPC: G06K9/40 ,  G06T5/00 ,  G06T5/50 ,  G06T5/20 ,  H04N5/217

CPC classification number: H04N9/77 ,  G06K9/40 ,  G06T5/002 ,  G06T5/009 ,  G06T5/20 ,  G06T5/50 ,  G06T2207/10024 ,  G06T2207/20016 ,  G06T2207/20182 ,  G06T2207/20208 ,  G06T2207/20221 ,  H04N5/208 ,  H04N5/217 ,  H04N9/646 ,  H04N9/67 ,  H04N9/69 ,  H04N9/73

Abstract: Systems, methods, and computer readable media to fuse digital images are described. In general, techniques are disclosed that use multi-band noise reduction techniques to represent input and reference images as pyramids. Once decomposed in this manner, images may be fused using novel low-level (noise dependent) similarity measures. In some implementations similarity measures may be based on intra-level comparisons between reference and input images. In other implementations, similarity measures may be based on inter-level comparisons. In still other implementations, mid-level semantic features such as black-level may be used to inform the similarity measure. In yet other implementations, high-level semantic features such as color or a specified type of region (e.g., moving, stationary, or having a face or other specified shape) may be used to inform the similarity measure.

公开(公告)号：US20160065795A1

公开(公告)日：2016-03-03

申请号：US14474103

申请日：2014-08-30

Applicant: Apple Inc.

Inventor： Farhan A. Baqai ,  Claus Molgaard ,  Fabio Riccardi ,  Xuemei Zhang

IPC: H04N5/217 ,  H04N9/77 ,  G06T5/00 ,  H04N9/04

CPC classification number: H04N5/217 ,  G06T5/002 ,  G06T5/008 ,  G06T2207/20012 ,  G06T2207/20016 ,  G06T2207/20048 ,  H04N9/045

Abstract: Image enhancement is achieved by separating image signals, e.g. YCbCr image signals, into a series of frequency bands and performing locally-adaptive noise reduction on bands below a given frequency but not on bands above that frequency. The bands are summed to develop the image enhanced signals. The YCbCr, multi-band locally-adaptive approach to denoising is able to operate independently—and in an optimized fashion—on both luma and chroma channels. Noise reduction is done based on models developed for both luma and chroma channels by measurements taken for multiple frequency bands, in multiple patches on the ColorChecker chart, and at multiple gain levels, in order to develop a simple yet robust set of models that may be tuned off-line a single time for each camera and then applied to images taken by such cameras in real-time without excessive processing requirements and with satisfactory results across illuminant types and lighting conditions.

Abstract translation: 图像增强通过分离图像信号来实现。 YCbCr图像信号转换成一系列频带，并在给定频率以下的频带上执行局部自适应噪声降低，但不在高于该频率的频带上进行。 将频带相加以开发图像增强信号。 YCbCr多频域局部自适应去噪方法能够在亮度和色度通道上独立运行，并以优化方式运行。 基于通过对ColorChecker图表上的多个频段进行测量，在ColorChecker图表上以多个增益水平测量的亮度和色度通道开发的模型以及多个增益水平来进行降噪，以便开发一个简单但可靠的模型集 为每台摄像机单独调整离线时间，然后实时应用于摄像机所拍摄的图像，而无需过多的处理要求，并且在各种照明类型和照明条件下都能获得令人满意的结果。

公开(公告)号：US20160065794A1

公开(公告)日：2016-03-03

申请号：US14474100

申请日：2014-08-30

Applicant: Apple Inc.

Inventor： Farhan A. Baqai ,  Claus Molgaard ,  Fabio Riccardi ,  Russell Pflughaupt

IPC: H04N5/217 ,  G06T5/00 ,  H04N9/077

CPC classification number: H04N5/217 ,  G06T5/002 ,  G06T5/10 ,  G06T2207/10024 ,  G06T2207/20016 ,  G06T2207/20048 ,  H04N5/21 ,  H04N9/077 ,  H04N9/646

Abstract: Image enhancement is achieved by separating image signals, e.g. YCbCr image signals, into a series of frequency bands and performing noise reduction on bands below a given frequency but not on bands above that frequency. The bands are summed to develop the image enhanced signals. The YCbCr, multi-band approach to denoising is able to operate independently—and in an optimized fashion—on both luma and chroma channels. Noise reduction is done based on models developed for both luma and chroma channels by measurements taken for multiple frequency bands, in multiple patches on the ColorChecker chart, and at multiple gain levels, in order to develop a simple yet robust set of models that may be tuned off-line a single time for each camera and then applied to images taken by such cameras in real-time without excessive processing requirements and with satisfactory results across illuminant types and lighting conditions.

Abstract translation: 图像增强通过分离图像信号来实现。 YCbCr图像信号转换成一系列频带，并在低于给定频率但不超过该频率的频带上进行降噪。 将频带相加以开发图像增强信号。 YCbCr多频去噪方法能够在亮度和色度通道上独立运行，并以优化方式运行。 基于通过对ColorChecker图表上的多个频段进行测量，在ColorChecker图表上以多个增益水平测量的亮度和色度通道开发的模型以及多个增益水平来进行降噪，以便开发一个简单但可靠的模型集 为每台摄像机单独调整离线时间，然后实时应用于摄像机所拍摄的图像，而无需过多的处理要求，并且在各种照明类型和照明条件下都能获得令人满意的结果。

公开(公告)号：US20150350513A1

公开(公告)日：2015-12-03

申请号：US14292792

申请日：2014-05-30

Applicant: Apple Inc.

Inventor： Xuemei Zhang ,  Marius Tico ,  Yingjun Bai ,  Farhan A. Baqai ,  Fabio Riccardi ,  Juanqing Chen ,  Paul M. Hubel ,  George E. Williams ,  Todd S. Sachs ,  Yongfang Liang

IPC: H04N5/235 ,  H04N5/232 ,  G06T5/00 ,  H04N5/265 ,  H04N5/91 ,  G06K9/46

CPC classification number: H04N5/2355 ,  G06K9/4642 ,  G06K9/4661 ,  G06T5/001 ,  G06T5/009 ,  G06T5/40 ,  G06T2207/10144 ,  G06T2207/20016 ,  G06T2207/20208 ,  G06T2207/20221 ,  H04N5/23245 ,  H04N5/265 ,  H04N5/91

Abstract: Systems, methods, and computer readable media to capture and process high dynamic range (HDR) images when appropriate for a scene are disclosed. When appropriate, multiple images at a single—slightly underexposed—exposure value are captured (making a constant bracket HDR capture sequence) and local tone mapping (LTM) applied to each image. Local tone map and histogram information can be used to generate a noise-amplification mask which can be used during fusion operations. Images obtained and fused in the disclosed manner provide high dynamic range with improved noise and de-ghosting characteristics.

Abstract translation: 公开了适用于场景时捕获和处理高动态范围（HDR）图像的系统，方法和计算机可读介质。 在适当的情况下，捕获单个稍微曝光不足的曝光值的多个图像（使恒定的支架HDR捕获序列）和本地色调映射（LTM）应用于每个图像。 本地色调图和直方图信息可用于生成可在融合操作期间使用的噪声放大掩模。 以所公开的方式获得和融合的图像提供高动态范围，具有改进的噪声和去重影特性。

公开(公告)号：US10666874B2

公开(公告)日：2020-05-26

申请号：US15979259

申请日：2018-05-14

Applicant: Apple Inc.

Inventor： Hao Sun ,  Fabio Riccardi ,  Garrett M. Johnson ,  Farhan A. Baqai

IPC: H04N5/235 ,  H04N5/232 ,  H04N5/355

Abstract: Techniques of reducing or eliminating artifact pixels in high dynamic range (HDR) imaging are described. One embodiment includes obtaining a first image of a scene at a first time with first exposure settings and obtaining a second image of the scene at a second time with second exposure settings that differ from the first exposure settings. The obtained images may be downsampled. The images may be compared to each other to assist with determining a number of potential artifact pixels in the scene. Depending on a relationship between the number of potential artifact pixels and a threshold value, the first image or second image may be selected as a reference image for registering the images with each other. A type of registration performed between the images may depend on which of the two images is the selected reference image. The registered images may be used to generate an HDR image.