公开(公告)号：US09153025B2

公开(公告)日：2015-10-06

申请号：US13551603

申请日：2012-07-17

申请人： Hailin Jin ,  Zihan Zhou

发明人： Hailin Jin ,  Zihan Zhou

IPC分类号： G06K9/00 ,  G06T7/00 ,  G06T7/20

CPC分类号： G06T7/80 ,  G06T7/174 ,  G06T7/20 ,  G06T2207/10016 ,  G06T2207/30241 ,  H04N13/20

摘要： Plane detection and tracking algorithms are described that may take point trajectories as input and provide as output a set of inter-image homographies. The inter-image homographies may, for example, be used to generate estimates for 3D camera motion, camera intrinsic parameters, and plane normals using a plane-based self-calibration algorithm. A plane detection and tracking algorithm may obtain a set of point trajectories for a set of images (e.g., a video sequence, or a set of still photographs). A 2D plane may be detected from the trajectories, and trajectories that follow the 2D plane through the images may be identified. The identified trajectories may be used to compute a set of inter-image homographies for the images as output.

摘要翻译： 描述了平面检测和跟踪算法，其可以将点轨迹作为输入，并且作为输出提供一组图像间同画。 例如，可以使用图像间同画来使用基于平面的自校准算法来生成用于3D相机运动，相机内在参数和平面法线的估计。 平面检测和跟踪算法可以获得一组图像（例如，视频序列或一组静态照片）的一组点轨迹。 可以从轨迹检测2D平面，并且可以识别通过图像在2D平面之后的轨迹。 所识别的轨迹可以用于计算图像的图像间同画作为输出。

公开(公告)号：US08929610B2

公开(公告)日：2015-01-06

申请号：US13368279

申请日：2012-02-07

申请人： Hailin Jin ,  Aseem O. Agarwala ,  Jue Wang

发明人： Hailin Jin ,  Aseem O. Agarwala ,  Jue Wang

IPC分类号： G06K9/00 ,  G06K9/36 ,  H04N7/18 ,  H04N5/225 ,  G06T7/00 ,  G06T7/20

CPC分类号： G06T7/33 ,  G06T7/246 ,  G06T2207/10016 ,  G06T2207/20028 ,  G06T2207/30241

摘要： Methods and apparatus for robust video stabilization. A video stabilization technique applies a feature tracking technique to an input video sequence to generate feature trajectories. The technique applies a video partitioning technique to segment the input video sequence into factorization windows and transition windows. The technique smoothes the trajectories in each of the windows, in sequence. For factorization windows, a subspace-based optimization technique may be used. For transition windows, a direct track optimization technique that uses a similarity motion model may be used. The technique then determines and applies warping models to the frames in the video sequence. In at least some embodiments, the warping models may include a content-preserving warping model, a homography model, a similarity transform model, and a whole-frame translation model. The warped frames may then be cropped according to a cropping technique.

摘要翻译： 稳健视频稳定的方法和装置。 视频稳定技术将特征跟踪技术应用于输入视频序列以产生特征轨迹。 该技术应用视频分割技术将输入视频序列分解为分解窗口和转换窗口。 该技术顺序地平滑每个窗口中的轨迹。 对于分解窗口，可以使用基于子空间的优化技术。 对于转换窗口，可以使用使用相似度运动模型的直接跟踪优化技术。 该技术然后确定并应用翘曲模型到视频序列中的帧。 在至少一些实施例中，翘曲模型可以包括内容保留翘曲模型，单应性模型，相似度变换模型和全帧翻译模型。 然后可以根据裁剪技术裁剪翘曲的框架。

公开(公告)号：US08675918B2

公开(公告)日：2014-03-18

申请号：US13368282

申请日：2012-02-07

申请人： Hailin Jin ,  Aseem O. Agarwala ,  Jue Wang

发明人： Hailin Jin ,  Aseem O. Agarwala ,  Jue Wang

IPC分类号： G06K9/00 ,  G06K9/36 ,  G06K9/46 ,  H04N5/225

CPC分类号： G06T7/33 ,  G06T7/246 ,  G06T2207/10016 ,  G06T2207/20028 ,  G06T2207/30241

摘要： Methods and apparatus for robust video stabilization. A video stabilization technique applies a feature tracking technique to an input video sequence to generate feature trajectories. The technique applies a video partitioning technique to segment the input video sequence into factorization windows and transition windows. The technique smoothes the trajectories in each of the windows, in sequence. For factorization windows, a subspace-based optimization technique may be used. For transition windows, a direct track optimization technique that uses a similarity motion model may be used. The technique then determines and applies warping models to the frames in the video sequence. In at least some embodiments, the warping models may include a content-preserving warping model, a homography model, a similarity transform model, and a whole-frame translation model. The warped frames may then be cropped according to a cropping technique.

公开(公告)号：US08509566B2

公开(公告)日：2013-08-13

申请号：US13486882

申请日：2012-06-01

申请人： Hailin Jin ,  Jen-Chan Chien

发明人： Hailin Jin ,  Jen-Chan Chien

IPC分类号： G06K9/36

CPC分类号： G06T3/4038 ,  G06T7/30

摘要： Systems, methods, and apparatuses, including computer program products, are provided for re-layout of composite images. In some implementations, a method includes identifying geometric transformations corresponding to multiple images from a collection of images, where a geometric transformation reorients a corresponding image in relation to a common reference frame when applied and identifying a reference image for the multiple images in the collection of images. The method also includes determining overlapping image regions for the multiple images starting from the reference image, the determining based on the identified geometric transformations, determining additional transformations of a specified type for the multiple images based on the overlapping image regions, where an additional transformation lays out a corresponding image in relation to the reference image when applied, and making the additional transformations available for further processing and output with respect to the collection of images.

摘要翻译： 提供了包括计算机程序产品在内的系统，方法和装置用于重新布局合成图像。 在一些实施方式中，一种方法包括从图像集合中识别对应于多个图像的几何变换，其中几何变换在应用时重新定位相对于公共参考帧的对应图像，并且在集合中识别多个图像的参考图像 图片。 该方法还包括确定从参考图像开始的多个图像的重叠图像区域，基于所识别的几何变换的确定，基于重叠图像区域确定多个图像的指定类型的额外变换，其中附加变换 在应用时相对于参考图像输出相应的图像，并且使得附加变换可用于进一步处理和输出关于图像的收集。

公开(公告)号：US20130129154A1

公开(公告)日：2013-05-23

申请号：US13300277

申请日：2011-11-18

申请人： Hailin Jin

发明人： Hailin Jin

IPC分类号： G06K9/00

CPC分类号： G06K9/00664 ,  G06T7/579 ,  G06T2207/30244

摘要： Methods and apparatus for detecting and removing poorly conditioned points in bundle adjustment. A detection technique may be used to detect poorly conditioned points during the bundle adjustment process or portion of a feature-based 3D reconstruction pipeline. Once the poorly conditioned points are detected, the technique removes the points from the bundle adjustment process. The technique detects and removes the poorly conditioned points before the optimization is performed, and therefore ensures that the optimization will be successful.

摘要翻译： 用于检测和去除束调节中不良条件点的方法和装置。 可以使用检测技术来检测束调整过程中的不良条件点或基于特征的3D重建管线的一部分。 一旦检测到不良条件的点，该技术从束调整过程中去除点。 该技术在执行优化之前检测和去除不良条件的点，因此确保优化成功。

公开(公告)号：US20130128065A1

公开(公告)日：2013-05-23

申请号：US13368279

申请日：2012-02-07

申请人： Hailin Jin ,  Aseem O. Agarwala ,  Jue Wang

发明人： Hailin Jin ,  Aseem O. Agarwala ,  Jue Wang

IPC分类号： H04N5/228

CPC分类号： G06T7/33 ,  G06T7/246 ,  G06T2207/10016 ,  G06T2207/20028 ,  G06T2207/30241

摘要： Methods and apparatus for robust video stabilization. A video stabilization technique applies a feature tracking technique to an input video sequence to generate feature trajectories. The technique applies a video partitioning technique to segment the input video sequence into factorization windows and transition windows. The technique smoothes the trajectories in each of the windows, in sequence. For factorization windows, a subspace-based optimization technique may be used. For transition windows, a direct track optimization technique that uses a similarity motion model may be used. The technique then determines and applies warping models to the frames in the video sequence. In at least some embodiments, the warping models may include a content-preserving warping model, a homography model, a similarity transform model, and a whole-frame translation model. The warped frames may then be cropped according to a cropping technique.

公开(公告)号：US20130127847A1

公开(公告)日：2013-05-23

申请号：US12954014

申请日：2010-11-24

申请人： Hailin Jin ,  James L. Andrews

发明人： Hailin Jin ,  James L. Andrews

IPC分类号： G06T17/05

CPC分类号： G06T7/564 ,  G06T17/20 ,  G06T19/20 ,  G06T2200/08 ,  G06T2200/24 ,  G06T2207/20072 ,  G06T2207/20096 ,  G06T2210/12 ,  G06T2219/2021

摘要： Various embodiments of an interactive system and methods for generating an image-based 3D model of an object from multiple images captured from different viewpoints of the object are described. A user may identify feature curves for the object by outlining the feature curves on a display of the images. The user may outline a feature curve in multiple images which depict the feature curve from different viewpoints. 3D shape constraints for the model may be generated dependent on the identified feature curves and camera parameters for the images. The user may also outline multiple single-view feature curves, such as silhouette curves, in single images. A 3D surface approximation for the model may be generated from the single-view feature curves. The 3D shape constraints and the 3D surface approximation may be used to generate a polygonal mesh which represents the surface of the 3D model of the object.

摘要翻译： 描述了用于从对象的不同视点捕获的多个图像生成对象的基于图像的3D模型的交互式系统和方法的各种实施例。 用户可以通过概述图像显示器上的特征曲线来识别对象的特征曲线。 用户可以在多个图像中描绘特征曲线，其从不同的视点描绘特征曲线。 可以根据所识别的特征曲线和图像的相机参数来生成模型的3D形状约束。 用户还可以在单​​个图像中概述多个单视图特征曲线，例如剪影曲线。 可以从单视图特征曲线生成模型的3D表面近似。 3D形状约束和3D表面近似可以用于生成表示对象的3D模型的表面的多边形网格。

公开(公告)号：US08411961B1

公开(公告)日：2013-04-02

申请号：US13286033

申请日：2011-10-31

申请人： Hailin Jin

发明人： Hailin Jin

IPC分类号： G06K9/46

CPC分类号： G06K9/6211 ,  G06K2009/2045 ,  G06T3/4038 ,  G06T7/33 ,  G06T2207/20092 ,  H04N5/23238

摘要： Method and apparatus for image feature matching in automatic image stitching processes. Embodiments may provide a computer-implemented method for performing a portion of an automatic image stitching process where feature correspondences are established between pairs of images. In embodiments, a computer-implemented image feature matching component may use a combination of one or more of heuristic techniques, information obtained from the user, file information related to the component images, and/or information obtained from previous feature matching iterations to narrow the number of images that are in a subset of component images to be compared for any given component image, and thus to narrow the number of pairs of component images on which image feature comparisons are performed.

摘要翻译： 自动图像拼接过程中图像特征匹配的方法和装置。 实施例可以提供一种用于执行自动图像拼接处理的一部分的计算机实现的方法，其中在图像对之间建立特征对应。 在实施例中，计算机实现的图像特征匹配部件可以使用启发式技术中的一个或多个，从用户获得的信息，与分量图像相关的文件信息和/或从先前的特征匹配迭代获得的信息的组合来缩小 在要对任何给定分量图像进行比较的分量图像的子集中的图像数量，从而缩小执行图像特征比较的分量图像对的数量。

公开(公告)号：US08368773B1

公开(公告)日：2013-02-05

申请号：US12201824

申请日：2008-08-29

申请人： Hailin Jin

发明人： Hailin Jin

IPC分类号： H04N5/228

CPC分类号： G06K9/3275 ,  H04N5/23238 ,  H04N5/247 ,  H04N5/3572

摘要： A method for aligning and unwarping distorted images in which lens profiles for a variety of lens and camera combinations are precomputed. Metadata stored with images is used to automatically determine if a set of component images include an excessive amount of distortion, and if so the metadata is used to determine an appropriate lens profile and initial unwarping function. The initial unwarping function is applied to the coordinates of feature points of the component images to generate substantially rectilinear feature points, which are used to estimate focal lengths, centers, and relative rotations for pairs of the images. A global nonlinear optimization is applied to the initial unwarping function(s) and the relative rotations to generate optimized unwarping functions and rotations for the component images. The optimized unwarping functions and rotations may be used to render a panoramic image.

摘要翻译： 一种用于对准和扭曲失真图像的方法，其中预先计算用于各种镜头和相机组合的透镜轮廓。 使用存储有图像的元数据来自动确定一组分量图像是否包含过多的失真，如果是，则使用元数据来确定适当的透镜轮廓和初始不正确的功能。 将初始不正确函数应用于分量图像的特征点的坐标，以生成基本上直线的特征点，其用于估计图像对的焦距，中心和相对旋转。 全局非线性优化被应用于初始不平衡函数和相对旋转以产生用于分量图像的优化的不正确函数和旋转。 优化的不正确功能和旋转可用于渲染全景图像。

公开(公告)号：US08073259B1

公开(公告)日：2011-12-06

申请号：US11843043

申请日：2007-08-22

申请人： Hailin Jin

发明人： Hailin Jin

IPC分类号： G06K9/46 ,  G06K9/66

CPC分类号： G06K9/6211 ,  G06K2009/2045 ,  G06T3/4038 ,  G06T7/33 ,  G06T2207/20092 ,  H04N5/23238

摘要： Method and apparatus for image feature matching in automatic image stitching processes. Embodiments may provide a computer-implemented method for performing a portion of an automatic image stitching process where feature correspondences are established between pairs of images. In embodiments, a computer-implemented image feature matching component may use a combination of one or more of heuristic techniques, information obtained from the user, file information related to the component images, and/or information obtained from previous feature matching iterations to narrow the number of images that are in a subset of component images to be compared for any given component image, and thus to narrow the number of pairs of component images on which image feature comparisons are performed.

摘要翻译： 自动图像拼接过程中图像特征匹配的方法和装置。 实施例可以提供一种用于执行自动图像拼接处理的一部分的计算机实现的方法，其中在图像对之间建立特征对应。 在实施例中，计算机实现的图像特征匹配部件可以使用启发式技术中的一个或多个，从用户获得的信息，与分量图像相关的文件信息和/或从先前的特征匹配迭代获得的信息的组合来缩小 在要对任何给定分量图像进行比较的分量图像的子集中的图像数量，从而缩小执行图像特征比较的分量图像对的数量。