公开(公告)号：US20170070692A1

公开(公告)日：2017-03-09

申请号：US14845659

申请日：2015-09-04

Applicant: Apple Inc.

Inventor： Sheng Lin ,  D. Amnon Silverstein ,  David R. Pope ,  Suk Hwan Lim

IPC: H04N5/367 ,  H04N5/217

Abstract: An image signal processor may include a pixel defect correction component that tracks defect history for frames captured by an image sensor and applies the history when identifying and correcting defective pixels in a frame. The component maintains a defect pixel location table that includes a defect confidence value for pixels of the image sensor. The component identifies defective pixels in a frame, for example by comparing each pixel's value to the values of its neighbor pixels. If a pixel is detected as defective, its defect confidence value may be incremented. Otherwise, the value may be decremented. If a pixel's defect confidence value is over a defect confidence threshold, the pixel is considered defective and thus may be corrected. If a pixel's defect confidence value is under the threshold, the pixel is considered not defective and thus may not be corrected even if the pixel was detected as defective.

Abstract translation: 图像信号处理器可以包括跟踪由图像传感器捕获的帧的缺陷历史的像素缺陷校正组件，并且在识别和校正帧中的缺陷像素时应用历史。 该组件保持缺陷像素位置表，其包括图像传感器的像素的缺陷置信度值。 组件识别帧中的缺陷像素，例如通过将每个像素的值与其相邻像素的值进行比较。 如果像素被检测为缺陷，则其缺陷置信度值可以增加。 否则，该值可能会减少。 如果像素的缺陷置信度值超过缺陷置信度阈值，则该像素被认为是有缺陷的，因此可被校正。 如果像素的缺陷置信度值低于阈值，则认为像素不是有缺陷的，因此即使像素被检测为有缺陷，也可能不被校正。

公开(公告)号：US20170061584A1

公开(公告)日：2017-03-02

申请号：US14841384

申请日：2015-08-31

Applicant: Apple Inc.

Inventor： Suk Hwan Lim ,  D. Amnon Silverstein

IPC: G06T5/00 ,  H04N9/04 ,  G06T1/20

CPC classification number: G06T5/002 ,  G06T1/20 ,  G06T7/41 ,  G06T2207/10024 ,  G06T2207/20012 ,  H04N5/217 ,  H04N9/045

Abstract: An image processing pipeline may dynamically determine filtering strengths for noise filtering of image data. Statistics may be collected for an image at an image processing pipeline. The statistics may be accessed and evaluated to generate a filter strength model that maps respective filtering strengths to different portions of the image. A noise filter may determine a filtering strength for image data received at the noise filter according to the filter strength model. The noise filter may then apply a filtering technique according to the determined filtering strength.

Abstract translation: 图像处理流水线可以动态地确定图像数据的噪声滤波的滤波强度。 可以在图像处理流水线上为图像收集统计数据。 可以访问和评估统计信息以产生将各自的过滤强度映射到图像的不同部分的过滤器强度模型。 噪声滤波器可以根据滤波强度模型确定在噪声滤波器处接收的图像数据的滤波强度。 然后，噪声滤波器可以根据确定的滤波强度来应用滤波技术。

公开(公告)号：US09369684B2

公开(公告)日：2016-06-14

申请号：US14623332

申请日：2015-02-16

Applicant: Apple Inc.

Inventor： Suk Hwan Lim ,  D. Amnon Silverstein

IPC: H04N9/64 ,  G06T5/40 ,  G06T5/00

CPC classification number: H04N9/646 ,  G06T5/008 ,  G06T5/40 ,  G06T2207/10016 ,  G06T2207/20012 ,  G06T2207/20016 ,  G06T2207/20021 ,  G06T2207/20208

Abstract: Image tone adjustment using local tone curve computation may be utilized to adjust luminance ranges for images. Image tone adjustment using local tone curve computation may reduce the overall contrast of an image, while maintaining local contrast in smaller areas, such as in images capturing brightly lit scenes where the difference in intensity between brightest and darkest areas is large. A desired brightness representation of the image may be generated including target luminance values for corresponding blocks of the image. For each block, one or more tone adjustment values may be computed, that when jointly applied to the respective histograms for the block and neighboring blocks results in the luminance values that match corresponding target values. The tone adjustment values may be determined by solving an under-constrained optimization problem such that optimization constraints are minimized. The image may then be adjusted according to the computed tone adjustment values.

公开(公告)号：US20160110843A1

公开(公告)日：2016-04-21

申请号：US14977384

申请日：2015-12-21

Applicant: Apple Inc.

Inventor： Christopher L. Mills ,  Sheng Lin ,  David R. Pope ,  D. Amnon Silverstein ,  Suk Hwan Lim

IPC: G06T3/40 ,  H04N9/64 ,  H04N9/07

CPC classification number: H04N9/04511 ,  G06T3/4015 ,  H04N5/2628 ,  H04N9/07 ,  H04N9/646 ,  H04N2209/046

Abstract: An input rescale module that performs cross-color correlated downscaling of sensor data in the horizontal and vertical dimensions. The module may perform a first-pass demosaic of sensor data, apply horizontal and vertical scalers to resample and downsize the data in the horizontal and vertical dimensions, and then remosaic the data to provide horizontally and vertically downscaled sensor data as output for additional image processing. The module may, for example, act as a front end scaler for an image signal processor (ISP). The demosaic performed by the module may be a relatively simple demosaic, for example a demosaic function that works on 3×3 blocks of pixels. The front end of module may receive and process sensor data at two pixels per clock (ppc); the horizontal filter component reduces the sensor data down to one ppc for downstream components of the input rescale module and for the ISP pipeline.

Abstract translation: 在水平和垂直维度上执行传感器数据的交叉色相关缩小的输入重定标模块。 模块可以执行传感器数据的第一遍去马赛克，应用水平和垂直缩放器对水平和垂直尺寸的数据进行重新取样和缩小，然后重新绘制数据，以提供水平和垂直缩小的传感器数据作为附加图像处理的输出 。 该模块可以例如用作图像信号处理器（ISP）的前端缩放器。 由模块执行的去镶嵌可以是相对简单的去马赛克，例如在3×3像素块上工作的去马赛克功能。 模块的前端可以以每个时钟两个像素（ppc）接收和处理传感器数据; 水平滤波器组件将传感器数据减少到输入重定标模块的下游组件和ISP管线的一ppc。

公开(公告)号：US20150350547A1

公开(公告)日：2015-12-03

申请号：US14502128

申请日：2014-09-30

Applicant: Apple Inc.

Inventor： Anita Nariani-Schulze ,  Benjamin M. Olson ,  Ralph Brunner ,  Suk Hwan Lim ,  Frank Doepke

IPC: H04N5/232

CPC classification number: H04N5/23254 ,  G06T7/215 ,  G06T2207/20016 ,  G06T2207/20076 ,  G06T2207/20104

Abstract: Techniques to detect subject and camera motion in a set of consecutively captured image frames are disclosed. More particularly, techniques disclosed herein temporally track two sets of downscaled images to detect motion. One set may contain higher resolution and the other set lower resolution of the same images. For each set, a coefficient of variation may be computed across the set of images for each sample in the downscaled image to detect motion and generate a change mask. The information in the change mask can be used for various applications, including determining how to capture a next image in the sequence.

Abstract translation: 公开了在一组连续拍摄的图像帧中检测主体和相机运动的技术。 更具体地，本文公开的技术暂时跟踪两组缩小图像以检测运动。 一组可能包含较高的分辨率，另一组则设置相同图像的较低分辨率。 对于每个集合，可以在缩小图像中的每个样本的图像集上计算变化系数，以检测运动并生成改变掩模。 更改掩码中的信息可用于各种应用，包括确定如何捕获序列中的下一个图像。

公开(公告)号：US20140010480A1

公开(公告)日：2014-01-09

申请号：US14023359

申请日：2013-09-10

Applicant: Apple Inc.

Inventor： Guy Cote ,  D. Amnon Silverstein ,  Suk Hwan Lim

IPC: G06T1/20

CPC classification number: G06T1/20 ,  H04N17/002

Abstract: Systems and methods are provided for selectively performing image statistics processing based at least partly on whether a pixel has been clipped. In one example, an image signal processor may include statistics collection logic. The statistics collection logic may include statistics image processing logic and a statistics core. The statistics image processing logic may perform initial image processing on image pixels, at least occasionally causing some of the image pixels to become clipped. The statistics core may obtain image statistics from the image pixels. The statistics core may obtain at least one of the image statistics using only pixels that have not been clipped and excluding pixels that have been clipped.

Abstract translation: 系统和方法被提供用于至少部分地基于是否已经剪切像素来选择性地执行图像统计处理。 在一个示例中，图像信号处理器可以包括统计信息收集逻辑。 统计信息采集逻辑可以包括统计图像处理逻辑和统计核心。 统计图像处理逻辑可以对图像像素执行初始图像处理，至少偶尔导致一些图像像素被削波。 统计核心可以从图像像素获取图像统计。 统计核心可以仅使用未被裁剪的像素并且排除已经被裁剪的像素来获得至少一个图像统计信息。

公开(公告)号：US11689826B2

公开(公告)日：2023-06-27

申请号：US17377308

申请日：2021-07-15

Applicant: Apple Inc.

Inventor： Guy Cote ,  D. Amnon Silverstein ,  Suk Hwan Lim ,  Sheng Lin ,  Haitao Guo

IPC: H04N5/217 ,  H04N25/67

CPC classification number: H04N25/67

Abstract: The present disclosure generally relates to systems and methods for image data processing. In certain embodiments, an image processing pipeline may be configured to receive a frame of the image data having a plurality of pixels acquired using a digital image sensor. The image processing pipeline may then be configured to determine a first plurality of correction factors that may correct each pixel in the plurality of pixels for fixed pattern noise. The first plurality of correction factors may be determined based at least in part on fixed pattern noise statistics that correspond to the frame of the image data. After determining the first plurality of correction factors, the image processing pipeline may be configured to configured to apply the first plurality of correction factors to the plurality of pixels, thereby reducing the fixed pattern noise present in the plurality of pixels.

公开(公告)号：US10586128B2

公开(公告)日：2020-03-10

申请号：US15954103

申请日：2018-04-16

Applicant: Apple Inc.

Inventor： Suk Hwan Lim ,  Jaewon Shin

IPC: G06K9/00 ,  G06K9/62 ,  G06K9/46

Abstract: Embodiments relate to a histogram-of-oriented gradients (HOG) module. The HOG module is implemented in hardware rather than software. The HOG module applies an algorithm to an image to identify gradient orientation in localized portions of the image. The HOG module creates a histogram-of orientation gradients based on the identified gradient orientations.

公开(公告)号：US10467496B2

公开(公告)日：2019-11-05

申请号：US14841412

申请日：2015-08-31

Applicant: Apple Inc.

Inventor： Suk Hwan Lim ,  D. Amnon Silverstein ,  Sheng Lin

IPC: G06K9/40 ,  G06K9/62 ,  G06T5/00

Abstract: An image processing pipeline may perform temporal filtering on independent color channels in image data. A filter weight may be determined for a given pixel received at a temporal filter. The filter weight may be determined for blending a value of a channel in a full color encoding of the given pixel with a value of the same channel for a corresponding pixel in a previously filtered reference image frame. In some embodiments, the filtering strength for the channel may be determined independent from the filtering strength of another channel in the full color encoding of the given pixel. Spatial filtering may be applied to a filtered version of the given pixel prior to storing the given pixel as part of a new reference image frame.

公开(公告)号：US20190244330A1

公开(公告)日：2019-08-08

申请号：US16389785

申请日：2019-04-19

Applicant: Apple Inc.

Inventor： Suk Hwan Lim ,  D. Amnon Silverstein

IPC: G06T5/00 ,  G06T7/41 ,  H04N5/217 ,  G06T1/20

CPC classification number: G06T5/002 ,  G06T1/20 ,  G06T7/41 ,  G06T2207/10024 ,  G06T2207/20012 ,  H04N5/217 ,  H04N9/045

Abstract: An image processing pipeline may dynamically determine filtering strengths for noise filtering of image data. Statistics may be collected for an image at an image processing pipeline. The statistics may be accessed and evaluated to generate a filter strength model that maps respective filtering strengths to different portions of the image. A noise filter may determine a filtering strength for image data received at the noise filter according to the filter strength model. The noise filter may then apply a filtering technique according to the determined filtering strength.