公开(公告)号：US20220137226A1

公开(公告)日：2022-05-05

申请号：US17578232

申请日：2022-01-18

Applicant: SAMSUNG ELECTRONICS CO., LTD.

Inventor： Kwang Oh KIM ,  Yibing Michelle WANG ,  Radwanul Hasan SIDDIQUE ,  Lilong SHI ,  Chunji WANG

IPC: G01S17/894 ,  G06T7/55 ,  G06T7/521 ,  H04N13/156 ,  H04N13/254 ,  H04N5/369 ,  G06T7/20 ,  G06V20/64

Abstract: Provided is a 3D depth sensing system and method of providing an image based on a hybrid sensing array. The 3D sensing system including a light source configured to emit light, a hybrid sensing array comprising a 2D sensing region configured to detect ambient light reflected from an object and a 3D depth sensing region configured to detect the light emitted by the light source and reflected from the object, a metalens on the hybrid sensing array, the metalens being configured to direct the ambient light reflected from the object towards the 2D sensing region, and to direct the light emitted by the light source and reflected from the object towards the 3D depth sensing region, and a processing circuit configured to combine 2D image information provided by the 2D sensing region and 3D information provided by the 3D depth sensing region to generate a combined 3D image.

公开(公告)号：US20200303437A1

公开(公告)日：2020-09-24

申请号：US16409767

申请日：2019-05-10

Applicant: Samsung Electronics Co., Ltd.

Inventor： Yibing Michelle WANG ,  Lilong SHI ,  Kwang Oh KIM ,  Chunji WANG

IPC: H01L27/146 ,  G01S17/10

Abstract: A pixel of an image sensor includes a pinned photodiode (PPD), a switching device and an output circuit. A first terminal of the switching device is coupled to the PPD. A second terminal of the switching device is coupled to a floating diffusion (FD). A third terminal of the switching device is coupled to a first enable signal and a second enable signal. The switching device is responsive to the first enable signal to transfer a first charge on the PPD to the FD, and responsive to the second enable signal to transfer a second charge on the PPD to the FD. The output circuit outputs a first voltage based on the first charge and outputs a second voltage based on the second charge in which the first voltage corresponds to a time of flight of one or more detected photons and the second voltage corresponds to a greyscale image.

公开(公告)号：US20200242789A1

公开(公告)日：2020-07-30

申请号：US16851093

申请日：2020-04-16

Applicant: Samsung Electronics Co., Ltd.

Inventor： Lilong SHI ,  Seunghoon HAN

IPC: G06T7/521 ,  G06K9/62 ,  H04N13/271 ,  H04N13/254 ,  G01B11/25

Abstract: A system and a method are disclosed for a structured-light system to estimate depth in an image. An image is received in which the image is of a scene onto which a reference light pattern has been projected. The projection of the reference light pattern includes a predetermined number of particular sub-patterns. A patch of the received image and a sub-pattern of the reference light pattern are matched based on either a hardcode template matching technique or a probability that the patch corresponds to the sub-pattern. If a lookup table is used, the table may be a probability matrix, may contain precomputed correlations scores or may contain precomputed class IDs. An estimate of depth of the patch is determined based on a disparity between the patch and the sub-pattern.

公开(公告)号：US20200043196A1

公开(公告)日：2020-02-06

申请号：US16597846

申请日：2019-10-09

Applicant: Samsung Electronics Co., Ltd.

Inventor： Zhengping JI ,  Lilong SHI ,  Yibing Michelle WANG ,  Hyun Surk RYU ,  Ilia OVSIANNIKOV

IPC: G06T7/73

Abstract: A Dynamic Vision Sensor (DVS) pose-estimation system includes a DVS, a transformation estimator, an inertial measurement unit (IMU) and a camera-pose estimator based on sensor fusion. The DVS detects DVS events and shapes frames based on a number of accumulated DVS events. The transformation estimator estimates a 3D transformation of the DVS camera based on an estimated depth and matches confidence-level values within a camera-projection model such that at least one of a plurality of DVS events detected during a first frame corresponds to a DVS event detected during a second subsequent frame. The IMU detects inertial movements of the DVS with respect to world coordinates between the first and second frames. The camera-pose estimator combines information from a change in a pose of the camera-projection model between the first frame and the second frame based on the estimated transformation and the detected inertial movements of the DVS.

公开(公告)号：US20200027228A1

公开(公告)日：2020-01-23

申请号：US16186463

申请日：2018-11-09

Applicant: Samsung Electronics Co., Ltd.

Inventor： Lilong SHI ,  Yibing Michelle WANG

IPC: G06T7/521 ,  H04N13/254 ,  H04N13/271

Abstract: A method of reconstructing a three dimensional image using a structured light pattern system is provided as follows. A class identifier of an observed pixel on a captured image by a camera is extracted. The observed pixel has a coordinate (x, y) on the captured image. A first relative position of the x coordinate of the observed pixel in a tile domain of the captured image is calculated. A second relative position of one of a plurality of dots in a tile domain of a reference image using the extracted class identifier is calculated. A disparity of the observed pixel using the first relative position and the second relative position is calculated.

公开(公告)号：US20190020864A1

公开(公告)日：2019-01-17

申请号：US16134954

申请日：2018-09-18

Applicant: Samsung Electronics Co., Ltd.

Inventor： Yibing Michelle WANG ,  Lilong SHI ,  Ilia OVSIANNIKOV

IPC: H04N13/254 ,  G01S7/486 ,  G01S17/10 ,  H04N5/3745 ,  H04N5/369 ,  H04N5/378

CPC classification number: H04N13/254 ,  G01S7/4863 ,  G01S17/10 ,  G01S17/89 ,  H04N5/3696 ,  H04N5/37455 ,  H04N5/378

Abstract: A Time-of-Flight (TOF) technique is combined with analog amplitude modulation within each pixel in a pixel array using multiple Single Photon Avalanche Diodes (SPADs) in conjunction with a single Pinned Photo Diode (PPD) in each pixel. A SPAD may be shared among multiple neighboring pixels. The TOF information is added to the received light signal by the analog domain-based single-ended to differential converter inside the pixel itself. The spatial-temporal correlation among outputs of multiple, adjacent SPADs in a pixel is used to control the operation of the PPD to facilitate recording of TOF values and range of an object. Erroneous range measurements due to ambient light are prevented by stopping the charge transfer from the PPD—and, hence, recording a TOF value—only when two or more SPADs in the pixel are triggered within a pre-defined time interval. An autonomous navigation system with multi-SPAD pixels provides improved vision for drivers under difficult driving conditions.

公开(公告)号：US20220146683A1

公开(公告)日：2022-05-12

申请号：US17583190

申请日：2022-01-24

Applicant: Samsung Electronics Co., Ltd.

Inventor： Yibing Michelle WANG ,  Lilong SHI ,  Ilia OVSIANNIKOV

IPC: G01S17/894 ,  H04N5/225 ,  H04N5/232 ,  G01N21/55 ,  G01S7/4865 ,  G01S7/481

Abstract: An image sensor includes a time-resolving sensor and a processor. The time-resolving sensor outputs a first signal and a second signal pair in response detecting one or more photons that have been reflected from an object. A first ratio of a magnitude of the first signal to a sum of the magnitude of the first signal and a magnitude of the second signal is proportional to a time of flight of the one or more detected photons. A second ratio of the magnitude of the second signal to the sum of the magnitude of the first signal and the magnitude of the second signal is proportional to the time of flight of the one or more detected photons. The processor determines a surface reflectance of the object where the light pulse has been reflected based on the first signal and the second signal pair and may generate a grayscale image.

公开(公告)号：US20210341284A1

公开(公告)日：2021-11-04

申请号：US17374982

申请日：2021-07-13

Applicant: Samsung Electronics Co., Ltd.

Inventor： Lilong SHI ,  Yibing Michelle WANG

IPC: G01B11/25 ,  G06T7/50

Abstract: A structured-light pattern for a structured-light system includes a base light pattern that includes a row of a plurality of sub-patterns extending in a first direction. Each sub-pattern is adjacent to at least one other sub-pattern, and each sub-pattern is different from each other sub-pattern. Each sub-pattern includes a first number of portions in a sub-row and a second number of portions in a sub-column. Each sub-row extends in the first direction and each sub-column extends in a second direction that is substantially orthogonal to the first direction. Each portion may be a first-type portion or a second-type portion. A size of a first-type portion is larger in the first direction and in the second direction than a size of a second-type portion in the first direction and in the second direction. In one embodiment, a first-type portion is a black portion and the second-type portion is a white portion.

公开(公告)号：US20210325541A1

公开(公告)日：2021-10-21

申请号：US16940308

申请日：2020-07-27

Applicant: SAMSUNG ELECTRONICS CO., LTD.

Inventor： Kwang Oh KIM ,  Yibing Michelle WANG ,  Radwanul Hasan Siddique ,  Lilong SHI ,  Chunji WANG

IPC: G01S17/894 ,  H04N13/254 ,  H04N13/156 ,  H04N5/369 ,  G06K9/00 ,  G06T7/20 ,  G06T7/521 ,  G06T7/55

Abstract: Provided is a 3D depth sensing system and method of providing an image based on a hybrid sensing array. The 3D sensing system including a light source configured to emit light, a hybrid sensing array comprising a 2D sensing region configured to detect ambient light reflected from an object and a 3D depth sensing region configured to detect the light emitted by the light source and reflected from the object, a metalens on the hybrid sensing array, the metalens being configured to direct the ambient light reflected from the object towards the 2D sensing region, and to direct the light emitted by the light source and reflected from the object towards the 3D depth sensing region, and a processing circuit configured to combine 2D image information provided by the 2D sensing region and 3D information provided by the 3D depth sensing region to generate a combined 3D image.

公开(公告)号：US20150022869A1

公开(公告)日：2015-01-22

申请号：US13944859

申请日：2013-07-17

Applicant: Samsung Electronics Co., Ltd.

Inventor： Lilong SHI ,  Ilia OVSIANNIKOV

IPC: H04N1/40 ,  H04N1/028

CPC classification number: H04N1/40012 ,  H04N1/648 ,  H04N9/045 ,  H04N2209/046 ,  H04N2209/047

Abstract: According to one general aspect, an apparatus may include an image sensor, a monochromatic pixel reconstruction engine, and a demosaicing engine. The image sensor may be configured to capture, at least in part, an image. The image sensor may include a plurality of first sensors configured to generate captured monochromatic pixel values, and a plurality of second sensors configured to not generate captured monochromatic pixel values. The plurality of second sensors may be dispersed amongst the plurality of first sensors such that portions of the image are not captured by the image sensor. The monochromatic pixel reconstruction engine may be configured to estimate an estimated monochromatic pixel value for each second sensor. The demosaicing engine may be configured to, via a color selective adaptive technique, generate a polychromatic image based upon the captured monochromatic pixel values and the estimated monochromatic pixel values.

Abstract translation: 根据一个一般方面，装置可以包括图像传感器，单色像素重构引擎和去马赛克引擎。 图像传感器可以被配置为至少部分地捕获图像。 图像传感器可以包括被配置为生成捕获的单色像素值的多个第一传感器，以及被配置为不生成捕获的单色像素值的多个第二传感器。 多个第二传感器可以分散在多个第一传感器之间，使得图像的一部分不被图像传感器捕获。 单色像素重建引擎可以被配置为估计每个第二传感器的估计单色像素值。 去马赛克引擎可以被配置为经由颜色选择自适应技术，基于所捕获的单色像素值和估计的单色像素值来生成多色图像。