公开(公告)号：US09122368B2

公开(公告)日：2015-09-01

申请号：US12722280

申请日：2010-03-11

Applicant: Richard S. Szeliski ,  Blaise H. Aguera y Arcas ,  Tomasz S. M. Kasperkiewicz

Inventor： Richard S. Szeliski ,  Blaise H. Aguera y Arcas ,  Tomasz S. M. Kasperkiewicz

IPC: G06F17/00 ,  G06F3/0481 ,  G06F17/30 ,  G06T19/00

CPC classification number: G06F3/04815 ,  G06F17/30274 ,  G06T19/003

Abstract: Images are analyzed within a 3D environment that is generated based on spatial relationships of the images and that allows users to experience the images in the 3D environment. Image analysis may include ranking images based on user viewing information, such as the number of users who have viewed an image and how long an image was viewed. Image analysis may further include analyzing the spatial density of images within a 3D environment to determine points of user interest.

Abstract translation: 在基于图像的空间关系生成的3D环境中分析图像，并允许用户体验3D环境中的图像。 图像分析可以包括基于用户观看信息来排序图像，诸如已经观看图像的用户的数量以及观看多长时间的图像。 图像分析还可以包括分析3D环境中的图像的空间密度以确定用户感兴趣的点。

公开(公告)号：US07590265B2

公开(公告)日：2009-09-15

申请号：US11242204

申请日：2005-10-01

Applicant: Richard S. Szeliski ,  Shmuel Avidan ,  Padmanabhan Anandan

Inventor： Richard S. Szeliski ,  Shmuel Avidan ,  Padmanabhan Anandan

IPC: G06K9/00

CPC classification number: H04N19/53 ,  G06T7/215

Abstract: The present invention is embodied in a system and method for extracting structure from multiple images of a scene by representing the scene as a group of image layers, including reflection and transparency layers. In general, the present invention performs layer extraction from multiple images containing reflections and transparencies. The present invention includes an optimal approach for recovering layer images and their associated motions from an arbitrary number of composite images. The present invention includes image formation equations, the constrained least squares technique used to recover the component images, a novel method to estimate upper and lower bounds on the solution using min- and max-composites, and a motion refinement method.

Abstract translation: 本发明体现在一种用于通过将场景表示为包括反射和透明层的图像层的组来从场景的多个图像提取结构的系统和方法。 通常，本发明从包含反射和透明度的多个图像中执行层提取。 本发明包括从任意数量的合成图像中恢复层图像及其相关运动的最佳方法。 本发明包括图像形成方程，用于恢复分量图像的约束最小二乘法技术，使用最小和最大复合材料估计解的上限和下限的新方法，以及运动细化方法。

公开(公告)号：US07587099B2

公开(公告)日：2009-09-08

申请号：US11340969

申请日：2006-01-27

Applicant: Richard S. Szeliski ,  Sing Bing Kang ,  Ce Liu ,  Charles L. Zitnick

Inventor： Richard S. Szeliski ,  Sing Bing Kang ,  Ce Liu ,  Charles L. Zitnick

IPC: G06K9/40 ,  G06K9/00

CPC classification number: G06T5/003 ,  G06T5/002 ,  G06T2207/20076

Abstract: An “Image Denoiser” provides a probabilistic process for denoising color images by segmenting an input image into regions, estimating statistics within each region, and then estimating a clean (or denoised) image using a probabilistic model of image formation. In one embodiment, estimated blur between each region is used to reduce artificial sharpening of region boundaries resulting from denoising the input image. In further embodiments, the estimated blur is used for additional purposes, including sharpening edges between one or more regions, and selectively blurring or sharpening one or more specific regions of the image (i.e., “selective focus”) while maintaining the original blurring between the various regions.

Abstract translation: “图像去噪器”提供了通过将输入图像分割成区域来去除彩色图像的概率过程，估计每个区域内的统计量，然后使用图像形成的概率模型来估计干净（或去噪）图像。 在一个实施例中，使用每个区域之间的估计模糊来减少由于对输入图像的去噪而导致的区域边界的人造锐化。 在另外的实施例中，估计的模糊被用于额外的目的，包括在一个或多个区域之间锐化边缘，以及选择性地模糊或锐化图像的一个或多个特定区域（即，“选择性聚焦”），同时保持原始模糊 各地区。

公开(公告)号：US06788333B1

公开(公告)日：2004-09-07

申请号：US09611646

申请日：2000-07-07

Applicant: Matthew T. Uyttendaele ,  Richard S. Szeliski

Inventor： Matthew T. Uyttendaele ,  Richard S. Szeliski

IPC: H04N700

CPC classification number: H04N5/23238 ,  G03B37/04 ,  G06T7/30 ,  G06T7/80 ,  G06T2200/32 ,  H04N17/002

Abstract: A system and process for generating a panoramic video. Essentially, the panoramic video is created by first acquiring multiple videos of the scene being depicted. Preferably, these videos collectively depict a full 360 degree view of the surrounding scene and are captured using a multiple camera rig. The acquisition phase also includes a calibration procedure that provides information about the camera rig used to capture the videos that is used in the next phase for creating the panoramic video. This next phase, which is referred to as the authoring phase, involves mosaicing or stitching individual frames of the videos, which were captured at approximately the same moment in time, to form each frame of the panoramic video. A series of texture maps are then constructed for each frame of the panoramic video. Each texture map coincides with a portion of a prescribed environment model of the scene. The texture map representations of each frame of the panoramic video are encoded so as to facilitate their transfer and viewing. This can include compressing the panoramic video frames. Such a procedure is useful in applications where the panoramic video is to be transferred over a network, such as the Internet.

Abstract translation: 用于生成全景视频的系统和过程。 本质上，通过首先获取所描绘的场景的多个视频来创建全景视频。 优选地，这些视频共同地描绘了周围场景的完整360度视图，并且使用多个相机钻机来捕获。 采集阶段还包括一个校准程序，该程序提供有关用于捕获下一阶段中用于创建全景视频的视频的摄像机的信息。 这个下一个阶段（被称为创作阶段）涉及拼接或缝合在大约相同的时刻被捕获的视频的各个帧，以形成全景视频的每个帧。 然后为全景视频的每个帧构建一系列纹理贴图。 每个纹理贴图与场景的规定环境模型的一部分重合。 对全景视频的每个帧的纹理映射表示进行编码，以便于它们的传送和观看。 这可以包括压缩全景视频帧。 这样的程序在通过诸如因特网的网络传输全景视频的应用中是有用的。

公开(公告)号：US06687400B1

公开(公告)日：2004-02-03

申请号：US09334860

申请日：1999-06-16

Applicant: Richard S. Szeliski

Inventor： Richard S. Szeliski

IPC: G06K900

CPC classification number: H04N5/235

Abstract: A system and method for manipulating a set of images of a static scene captured at different exposures (i.e., “bracketed” images) to yield a composite image with improved uniformity in exposure and tone. In general, the aforementioned goal can be achieved by analyzing a set of bracketed images using a multi-dimensional histogram and merging the images via an approach that projects pixels onto a curve that fits the data. However, it has been found that the desired composite image can be produced in a simpler manner by summing the pixel brightness levels across the multiple images, followed by an equalization process. One possible equalization process involves simply averaging the summed pixel brightness values by dividing the summed value of each pixel set (i.e., groups of corresponding pixels from the bracketed images) by the number of bracketed images. An even better result can be achieved using a histogram equalization process. In essence, this histogram equalization involves creating a count of the number of pixels sets having the same summed brightness level. From this count, a cumulative distribution function is computed and normalized to a maximum value corresponding to the maximum summed brightness level. The cumulative distribution function is then used to determine new pixel brightness levels for use in generating the composite image.

Abstract translation: 一种用于操纵在不同曝光（即，“括号”的图像）处拍摄的静态场景的一组图像以产生具有改善的曝光和色调均匀性的合成图像的系统和方法。 通常，可以通过使用多维直方图分析一组括号图像并通过将像素投影到适合数据的曲线上的方法来合并图像来实现上述目标。 然而，已经发现，可以通过对多个图像之间的像素亮度等级进行求和，然后进行均衡处理，以更简单的方式产生期望的合成图像。 一个可能的均衡处理涉及通过将每个像素组（即，来自括号的图像的相应像素的组）的相加值除以包围的图像的数量来简单地对相加的像素亮度值进行平均。 使用直方图均衡过程可以获得更好的结果。 实质上，该直方图均衡涉及产生具有相同的相加亮度水平的像素集合的数量的计数。 从该计数，计算累积分布函数并将其归一化为对应于最大相加亮度水平的最大值。 然后累积分布函数用于确定用于生成合成图像的新像素亮度级别。

公开(公告)号：US6023523A

公开(公告)日：2000-02-08

申请号：US885259

申请日：1997-06-30

Applicant: Michael F. Cohen ,  Steven Gortler ,  Richard S. Szeliski

Inventor： Michael F. Cohen ,  Steven Gortler ,  Richard S. Szeliski

IPC: G06T15/06 ,  G06T15/50 ,  G06T15/00 ,  G06T17/00

CPC classification number: G06T15/50 ,  G06T15/06

Abstract: A computer-based method and system for digital 3-dimensional imaging of an object which allows for viewing images of the object from arbitrary vantage points. The system, referred to as the Lumigraph system, collects a complete appearance of either a synthetic or real object (or a scene), stores a representation of the appearance, and uses the representation to render images of the object from any vantage point. The appearance of an object is a collection of light rays that emanate from the object in all directions. The system stores the representation of the appearance as a set of coefficients of a 4-dimensional function, referred to as the Lumigraph function. From the Lumigraph function with these coefficients, the Lumigraph system can generate 2-dimensional images of the object from any vantage point. The Lumigraph system generates an image by evaluating the Lumigraph function to identify the intensity values of light rays that would emanate from the object to form the image. The Lumigraph system then combines these intensity values to form the image.

Abstract translation: 一种用于对象的数字三维成像的基于计算机的方法和系统，其允许从任意有利位置观看对象的图像。 称为Lumigraph系统的系统收集合成或实物（或场景）的完整外观，存储外观的表示，并使用该表示从任何优势点呈现对象的图像。 物体的外观是从各个方向从物体发出的光线的集合。 该系统将外观的表示存储为称为Lumigraph函数的四维函数的一组系数。 从具有这些系数的Lumigraph函数，Lumigraph系统可以从任何有利位置生成对象的二维图像。 Lumigraph系统通过评估Lumigraph函数来生成图像，以识别从物体发出的形成图像的光线的强度值。 然后，Lumigraph系统将这些强度值组合以形成图像。

公开(公告)号：US20110109755A1

公开(公告)日：2011-05-12

申请号：US12616782

申请日：2009-11-12

Applicant: Neel S. Joshi ,  Sing Bing Kang ,  Charles L. Zitnick, III ,  Richard S. Szeliski

Inventor： Neel S. Joshi ,  Sing Bing Kang ,  Charles L. Zitnick, III ,  Richard S. Szeliski

IPC: G06T7/20 ,  G06K9/40 ,  H04N5/228

CPC classification number: H04N5/23248 ,  H04N5/23258 ,  H04N5/23267

Abstract: The described implementations relate to deblurring images. One system includes an imaging device configured to capture an image, a linear motion detector and a rotational motion detector. This system also includes a controller configured to receive a signal from the imaging device relating to capture of the image and to responsively cause the linear motion detector and the rotational motion detector to detect motion-related information. Finally, this particular system includes a motion calculator configured to recover camera motion associated with the image based upon the detected motion-related information and to infer imaging device motion induced blur of the image and an image deblurring component configured to reduce imaging device induced blur from the image utilizing the inferred camera motion induced blur.

Abstract translation: 所描述的实现涉及去模糊图像。 一个系统包括被配置为捕获图像的成像装置，线性运动检测器和旋转运动检测器。 该系统还包括控制器，其被配置为从成像装置接收与捕获图像有关的信号，并且响应地使线性运动检测器和旋转运动检测器检测运动相关信息。 最后，该特定系统包括运动计算器，该运动计算器被配置为基于检测到的运动相关信息来恢复与图像相关联的摄像机运动，并且推断图像的成像装置运动引起的模糊，以及被配置成减少成像装置引起的模糊的图像去模糊部件 该图像利用推测的相机运动引起的模糊。

公开(公告)号：US07903883B2

公开(公告)日：2011-03-08

申请号：US11694938

申请日：2007-03-30

Applicant: Charles Lawrence Zitnick, III ,  Xiangyang Lan ,  Richard S. Szeliski

Inventor： Charles Lawrence Zitnick, III ,  Xiangyang Lan ,  Richard S. Szeliski

IPC: G06K9/00

CPC classification number: G06K9/468 ,  G06K9/6296

Abstract: A local bi-gram model object recognition system and method for constructing a local bi-gram model and using the model to recognize objects in a query image. In a learning phase, the local bi-gram model is constructed that represents objects found in a set of training images. The local bi-gram model is a local spatial model that only models the relationship of neighboring features without any knowledge of their global context. Object recognition is performed by finding a set of matching primitives in the query image. A tree structure of matching primitives is generated and a search is performed to find a tree structure of matching primitives that obeys the local bi-gram model. The local bi-gram model can be found using unsupervised learning. The system and method also can be used to recognize objects unsupervised that are undergoing non-rigid transformations for both object instance recognition and category recognition.

Abstract translation: 一种局部双向模型对象识别系统和方法，用于构建局部双向模型，并使用该模型来识别查询图像中的对象。 在学习阶段，构建了表示在一组训练图像中发现的对象的局部双语模型。 当地的双语模型是一种局部空间模型，它只对相邻特征的关系进行建模，而无需了解其全局环境。 通过在查询图像中找到一组匹配的基元来执行对象识别。 生成匹配原语的树形结构，并执行搜索以找到符合本地双语模型的匹配原语的树结构。 可以使用无监督学习找到当地的双语模型。 系统和方法也可用于识别无监督的对象实例识别和类别识别正在进行非刚性转换的对象。

公开(公告)号：US20100302280A1

公开(公告)日：2010-12-02

申请号：US12476810

申请日：2009-06-02

Applicant: Richard S. Szeliski ,  Johannes P. Kopf ,  Michael F. Cohen ,  Eric J. Stollnitz

Inventor： Richard S. Szeliski ,  Johannes P. Kopf ,  Michael F. Cohen ,  Eric J. Stollnitz

IPC: G09G5/00

CPC classification number: G06T19/003 ,  G06T7/33 ,  G06T13/80 ,  G06T15/30 ,  G06T2200/32 ,  G06T2207/10016 ,  G06T2207/30196

Abstract: Techniques and systems are disclosed for navigating human scale image data using aligned perspective images. A consecutive sequence of digital images is stacked together by aligning consecutive images laterally with an image offset between edges of consecutive images corresponding to a distance between respective view windows of the consecutive images. A view window of an image in the sequence is rendered, where the view window of the image corresponds to a desired location. Offset portions of the view window of a desired number of images in the sequence are rendered, for example, alongside the full view of the image at the desired location.

Abstract translation: 公开了用于使用对准的透视图导航人体尺度图像数据的技术和系统。 通过将连续图像横向对准连续图像的边缘之间的图像偏移，对应于连续图像的各个视图窗口之间的距离，将连续的数字图像序列堆叠在一起。 呈现序列中的图像的视窗，其中图像的视图窗口对应于期望的位置。 该序列中所需数量的图像的视窗的偏移部分，例如，在所需位置处的图像的全视图旁边被渲染。

公开(公告)号：US07706609B2

公开(公告)日：2010-04-27

申请号：US11343581

申请日：2006-01-30

Applicant: Eric P. Bennett ,  Matthew T. Uyttendaele ,  Charles L. Zitnick ,  Sing Bing Kang ,  Richard S. Szeliski

Inventor： Eric P. Bennett ,  Matthew T. Uyttendaele ,  Charles L. Zitnick ,  Sing Bing Kang ,  Richard S. Szeliski

IPC: G06K9/00 ,  G06K9/40 ,  G06K9/32

CPC classification number: H04N9/646 ,  H04N1/58 ,  H04N9/045

Abstract: A Bayesian two-color image demosaicer and method for processing a digital color image to demosaic the image in such a way as to reduce image artifacts. The method and system are an improvement on and an enhancement to previous demosaicing techniques. A preliminary demosaicing pass is performed on the image to assign each pixel a fully specified RGB triple color value. The final color value of pixel in the processed image is restricted to be a linear combination of two colors. Fully-specified RGB triple color values for each pixel in an image used to find two clusters represented favored two colors. The amount of contribution from these favored two colors on the final color value then is determined. The method and system also can process multiple images to improve the demosaicing results. When using multiple images, sampling can be performed at a finer resolution, known as super resolution.

Abstract translation: 一种贝叶斯双色图像拆分器和用于处理数字彩色图像以使图像去马赛克的方法，以减少图像伪像。 该方法和系统是对以前的去马赛克技术的改进和增强。 对图像执行初步去马赛克通行，为每个像素分配完全指定的RGB三色值。 处理图像中的像素的最终色值被限制为两种颜色的线性组合。 用于查找两个聚类的图像中每个像素的完全指定的RGB三色值代表有利于两种颜色。 然后确定这些有利的两种颜色对最终颜色值的贡献量。 该方法和系统还可以处理多个图像以改善去马赛克结果。 当使用多个图像时，可以以更精细的分辨率（称为超分辨率）进行采样。