公开(公告)号：US11699264B2

公开(公告)日：2023-07-11

申请号：US16881104

申请日：2020-05-22

Applicant: INDUSTRIAL TECHNOLOGY RESEARCH INSTITUTE

Inventor： Te-Mei Wang

IPC: G06T17/20 ,  G06T7/90 ,  G06T7/73 ,  G06T17/00 ,  G06T7/11 ,  G06T7/50 ,  G06T7/40 ,  G06T7/12

CPC classification number: G06T17/205 ,  G06T7/11 ,  G06T7/40 ,  G06T7/50 ,  G06T7/73 ,  G06T7/90 ,  G06T17/00 ,  G06T17/20 ,  G06T7/12 ,  G06T2207/10016 ,  G06T2207/10024 ,  G06T2207/10028 ,  G06T2210/56

Abstract: A method, a system, and a computing device for reconstructing three-dimensional planes are provided. The method includes the following steps: obtaining a series of color information, depth information and pose information of a dynamic scene by a sensing device; extracting a plurality of feature points according to the color information and the depth information, and marking part of the feature points as non-planar objects including dynamic objects and fragmentary objects; computing point cloud according to the unmarked feature points and the pose information, and instantly converting the point cloud to a three-dimensional mesh; and growing the three-dimensional mesh to fill vacancy corresponding to the non-planar objects according to the information of the three-dimensional mesh surrounding or adjacent to the non-planar objects.

公开(公告)号：US10742961B2

公开(公告)日：2020-08-11

申请号：US15255164

申请日：2016-09-02

Applicant: Industrial Technology Research Institute

Inventor： Te-Mei Wang ,  Zen-Chung Shih

IPC: H04N17/00 ,  H04N13/246 ,  H04N5/222 ,  G01B11/25 ,  H04N13/239

Abstract: A depth sensing apparatus with self-calibration and a self-calibration method thereof are provided. The depth sensing apparatus includes a projection apparatus, an image capturing apparatus and a calibration module. The projection apparatus projects a calibration pattern and a depth computation pattern to a reference plane based on a predefined calibration pattern and a predefined depth computation pattern. The image capturing apparatus captures an image including the calibration pattern and the depth computation pattern. The calibration module coupled to the image capturing apparatus adjusts apparatus parameters of the depth sensing apparatus to calibrate a depth computation deviation according to the calibration pattern of the image, the predefined calibration pattern and a predefined lookup table corresponding to the predefined calibration pattern.

公开(公告)号：US12075024B2

公开(公告)日：2024-08-27

申请号：US17556401

申请日：2021-12-20

Applicant: INDUSTRIAL TECHNOLOGY RESEARCH INSTITUTE

Inventor： Te-Mei Wang

IPC: G06T7/80 ,  G06T1/00 ,  G06T3/02 ,  H04N17/00

CPC classification number: H04N17/002 ,  G06T1/0007 ,  G06T3/02 ,  G06T7/80

Abstract: A system and a method for computing the relative rotation and the relative translation of back-to-back cameras comprise: a fixed mode and an adjustment mode. In the fixed mode, a reversible external camera is installed next to the back-to-back cameras, the external camera is flipped to shoot a calibration pattern from different positions and angles, the intrinsic and extrinsic parameters of each camera are computed, and then a reverse calculation is performed on the extrinsic parameters, as such the amount of translation and rotation of the back-to-back cameras are obtained. When the relative angle of the back-to-back cameras changes, the angle is automatically detected and the extrinsic parameters of the camera computed at the fixed mode is modified, and then the reverse calculation is performed on the extrinsic parameters to adjust the amount of translation and rotation of the back-to-back cameras.

公开(公告)号：US20200184684A1

公开(公告)日：2020-06-11

申请号：US16233262

申请日：2018-12-27

Applicant: INDUSTRIAL TECHNOLOGY RESEARCH INSTITUTE

Inventor： Tien-Yan MA ,  Te-Mei Wang ,  Ping-Chang Shih

IPC: G06T7/80 ,  G06T7/55 ,  G06T7/174 ,  G06T7/12

Abstract: A depth camera calibration device and a method thereof are provided. The method comprises: disabling a projection function of a camera and photographing surface planes to obtain a first image; enabling the projection function of the camera and photographing the surface planes to obtain a second image, and a positional relationship between the camera and a calibration plate assembly remaining unchanged when obtaining the first image and the second image; and obtaining parameters of the camera by cropping an image boundary, calculating positions of characteristic points, reading a projection pattern, calculating positions of corresponding points and excluding abnormal points according to the first image and the second image.

公开(公告)号：US20160163091A1

公开(公告)日：2016-06-09

申请号：US14576163

申请日：2014-12-18

Applicant: Industrial Technology Research Institute

Inventor： Te-Mei Wang ,  Chen-Hao Wei ,  Hsiao-Wei Chen

IPC: G06T15/20 ,  G06T17/05 ,  G06T7/20 ,  G06T5/00 ,  G06T7/00 ,  H04N9/04 ,  G06T17/10 ,  G06T19/20

CPC classification number: G06T15/20 ,  G06T5/002 ,  G06T7/73 ,  G06T17/00 ,  G06T17/05 ,  G06T17/10 ,  G06T19/20 ,  G06T2200/04 ,  G06T2200/08 ,  G06T2200/21 ,  G06T2207/10004 ,  G06T2207/10032 ,  G06T2207/20182 ,  G06T2207/30181 ,  G06T2207/30244 ,  G06T2210/61 ,  H04N5/23222 ,  H04N5/2353 ,  H04N9/045

Abstract: An electronic apparatus and a method for incremental pose estimation and photographing are provided. In the method, at least two images of a 3D object are captured at different positions encircling the 3D object. Displacements and angular displacements are detected when capturing the images. Features of the 3D object in the images, the displacements and the angular displacements are used to estimate a central position of the 3D object and a distance between the electronic apparatus and the 3D object. A circumference suitable for capturing the images of the 3D object is estimated based on the distance and is divided into several segments, and a timing interval of a timer is adjusted based on a length of the segments. A camera of the electronic apparatus is triggered at intervals set by the timer to capture the images of the 3D object.

Abstract translation: 提供了一种用于增量姿势估计和拍摄的电子设备和方法。 在该方法中，在围绕3D对象的不同位置捕获3D对象的至少两个图像。 捕获图像时会检测位移和角位移。 图像中3D对象的特征，位移和角位移用于估计3D对象的中心位置和电子设备与3D对象之间的距离。 基于距离来估计适于捕获3D对象的图像的圆周，并且被划分成若干段，并且基于段的长度调整定时器的定时间隔。 电子设备的相机以由定时器设置的间隔触发以捕获3D对象的图像。

公开(公告)号：US09443349B2

公开(公告)日：2016-09-13

申请号：US14576163

申请日：2014-12-18

Applicant: Industrial Technology Research Institute

Inventor： Te-Mei Wang ,  Chen-Hao Wei ,  Hsiao-Wei Chen

IPC: G06K9/00 ,  G06T15/20 ,  G06T17/10 ,  G06T17/05 ,  G06T19/20 ,  G06T5/00 ,  G06T7/00 ,  H04N9/04 ,  G06T7/20

CPC classification number: G06T15/20 ,  G06T5/002 ,  G06T7/73 ,  G06T17/00 ,  G06T17/05 ,  G06T17/10 ,  G06T19/20 ,  G06T2200/04 ,  G06T2200/08 ,  G06T2200/21 ,  G06T2207/10004 ,  G06T2207/10032 ,  G06T2207/20182 ,  G06T2207/30181 ,  G06T2207/30244 ,  G06T2210/61 ,  H04N5/23222 ,  H04N5/2353 ,  H04N9/045

Abstract: An electronic apparatus and a method for incremental pose estimation and photographing are provided. In the method, at least two images of a 3D object are captured at different positions encircling the 3D object. Displacements and angular displacements are detected when capturing the images. Features of the 3D object in the images, the displacements and the angular displacements are used to estimate a central position of the 3D object and a distance between the electronic apparatus and the 3D object. A circumference suitable for capturing the images of the 3D object is estimated based on the distance and is divided into several segments, and a timing interval of a timer is adjusted based on a length of the segments. A camera of the electronic apparatus is triggered at intervals set by the timer to capture the images of the 3D object.

Abstract translation: 提供了一种用于增量姿势估计和拍摄的电子设备和方法。 在该方法中，在围绕3D对象的不同位置捕获3D对象的至少两个图像。 捕获图像时会检测位移和角位移。 图像中3D对象的特征，位移和角位移用于估计3D对象的中心位置和电子设备与3D对象之间的距离。 基于距离来估计适于捕获3D对象的图像的圆周，并且被划分成若干段，并且基于段的长度调整定时器的定时间隔。 电子设备的相机以由定时器设置的间隔触发以捕获3D对象的图像。

公开(公告)号：US09076249B2

公开(公告)日：2015-07-07

申请号：US13789693

申请日：2013-03-08

Applicant: Industrial Technology Research Institute

Inventor： Hsiao-Wei Chen ,  Te-Mei Wang ,  Wei-Yi Lee

IPC: G06T15/00 ,  H04N13/00

CPC classification number: G06T15/00 ,  H04N13/00 ,  H04N13/111

Abstract: A hole filling method for multi-view disparity maps is provided. At least one disparity map is respectively captured as a plurality of known views among a plurality of views for capturing an object. As for a plurality of virtual views among the views excluding the at least one known view, disparity maps of the virtual views are synthesized by sequentially using the disparity maps of the known views according to a distance of a virtual camera position or a transformed angle between each virtual view and each known view. Hole filling information of the disparity maps of other virtual views having the distances or the transformed angles smaller than that of the virtual view is used to fill holes in the synthesized disparity maps of the virtual views.

Abstract translation: 提供了一种用于多视角视差图的孔填充方法。 在用于捕获对象的多个视图中，分别捕获至少一个视差图作为多个已知视图。 对于除了至少一个已知视图之外的视图中的多个虚拟视图，虚拟视图的视差图是通过依照虚拟相机位置的距离或者虚拟相机位置的距离或者在 每个虚拟视图和每个已知视图。 使用具有小于虚拟视图的距离或变换角度的其他虚拟视图的视差图的填孔信息来填充虚拟视图的合成视差图中的空洞。

公开(公告)号：US20130251240A1

公开(公告)日：2013-09-26

申请号：US13726821

申请日：2012-12-26

Applicant: INDUSTRIAL TECHNOLOGY RESEARCH INSTITUTE

Inventor： Te-Mei Wang ,  Hsiao-Wei Chen

IPC: G06K9/36

CPC classification number: G06K9/36 ,  G06T5/005 ,  G06T2200/04 ,  G06T2207/10028 ,  H04N13/111

Abstract: An image processing apparatus includes a determination unit, a search unit, a weight assignment unit and a filling unit. The determination unit determines whether a hole is surrounded by the foreground in a disparity map or a depth map. The search unit searches for multiple relative backgrounds along multiple directions when the hole is surrounded by the foreground. The weight assignment unit respectively assigns weights to the relative backgrounds. The filling unit selects an extremum from the weights, and fills the hole according to the relative background corresponding to the extremum.

Abstract translation: 图像处理装置包括确定单元，搜索单元，重量分配单元和填充单元。 确定单元确定孔在视差图或深度图中是否被前景包围。 当孔被前景包围时，搜索单元沿着多个方向搜索多个相对背景。 权重分配单元分配权重给相对背景。 填充单元从重物中选择极值，并根据与极值相对应的相对背景填充孔。

公开(公告)号：US10977829B2

公开(公告)日：2021-04-13

申请号：US16233262

申请日：2018-12-27

Applicant: INDUSTRIAL TECHNOLOGY RESEARCH INSTITUTE

Inventor： Tien-Yan Ma ,  Te-Mei Wang ,  Ping-Chang Shih

IPC: G06T7/80 ,  G06T7/55 ,  G06T7/12 ,  G06T7/174

Abstract: A depth camera calibration device and a method thereof are provided. The method comprises: disabling a projection function of a camera and photographing surface planes to obtain a first image; enabling the projection function of the camera and photographing the surface planes to obtain a second image, and a positional relationship between the camera and a calibration plate assembly remaining unchanged when obtaining the first image and the second image; and obtaining parameters of the camera by cropping an image boundary, calculating positions of characteristic points, reading a projection pattern, calculating positions of corresponding points and excluding abnormal points according to the first image and the second image.

公开(公告)号：US09998719B2

公开(公告)日：2018-06-12

申请号：US15610174

申请日：2017-05-31

Applicant: INDUSTRIAL TECHNOLOGY RESEARCH INSTITUTE

Inventor： Te-Mei Wang ,  Zen-Chung Shih

IPC: H04N9/31 ,  G06T7/50

CPC classification number: H04N9/3194 ,  G06T7/50 ,  G06T7/521 ,  G06T7/80 ,  G06T2207/10024 ,  G06T2207/10152 ,  G06T2207/30204 ,  H04N9/3185 ,  H04N9/3188

Abstract: A non-planar surface projecting system, an auto-calibration method thereof and an auto-calibration device thereof are provided. In an embodiment of the non-planar surface auto-calibration method, a depth map of a projection surface is measured, a corresponding area corresponding to a projecting area and a depth sensing area is calibrated; and in the corresponding area, an interactive content is adjusted according to the depth map of the projection surface.