公开(公告)号：US12200398B2

公开(公告)日：2025-01-14

申请号：US17591040

申请日：2022-02-02

Applicant: Samsung Electronics Co., Ltd.

Inventor： Gyeongmin Choe ,  Yingmao Li ,  John Seokjun Lee ,  Hamid R. Sheikh ,  Michael O. Polley

IPC: H04N7/01 ,  G06T1/00 ,  G06T3/18 ,  G06T5/50 ,  G06T7/207 ,  H04N1/32 ,  H04N19/139 ,  H04N19/577

Abstract: An apparatus includes at least one processing device configured to obtain input frames from a video. The at least one processing device is also configured to generate a forward flow from a first input frame to a second input frame and a backward flow from the second input frame to the first input frame. The at least one processing device is further configured to generate an occlusion map at an interpolated frame coordinate using the forward flow and the backward flow. The at least one processing device is also configured to generate a consistency map at the interpolated frame coordinate using the forward flow and the backward flow. In addition, the at least one processing device is configured to perform blending using the occlusion map and the consistency map to generate an interpolated frame at the interpolated frame coordinate.

公开(公告)号：US11869118B2

公开(公告)日：2024-01-09

申请号：US17588024

申请日：2022-01-28

Applicant: Samsung Electronics Co., Ltd.

Inventor： Pavan Chennagiri ,  John Seokjun Lee ,  Hamid R. Sheikh

IPC: G06T11/00 ,  G06T19/20 ,  G06T7/90 ,  G06T5/00 ,  G06N20/00 ,  G06T17/00

CPC classification number: G06T11/001 ,  G06N20/00 ,  G06T5/002 ,  G06T7/90 ,  G06T17/00 ,  G06T19/20 ,  G06T2207/10012 ,  G06T2207/10024 ,  G06T2207/20081 ,  G06T2219/2012

Abstract: An apparatus includes at least one memory configured to store an AI network and at least one processor. The at least one processor is configured to generate a dead leaves model. The at least one processor is also configured to capture a ground truth frame from the dead leaves. The at least one processor is further configured to apply a mathematical noise model to the ground truth frame to produce a noisy frame. In addition, the at least one processor is configured to train the AI network using the ground truth frame and the noisy frame.

公开(公告)号：US20230252608A1

公开(公告)日：2023-08-10

申请号：US17666166

申请日：2022-02-07

Applicant: Samsung Electronics Co., Ltd.

Inventor： Yingmao Li ,  Hamid R. Sheikh ,  John Seokjun Lee ,  Youngmin Kim ,  Jun Ki Cho ,  Seung-Chul Jeon

IPC: G06T5/00

CPC classification number: G06T5/005 ,  G06T2207/20081 ,  G06T2207/10144 ,  G06T2207/20084

Abstract: A method includes obtaining, using a stationary sensor of an electronic device, multiple image frames including first and second image frames. The method also includes generating, using multiple previously generated motion vectors, a first motion-distorted image frame using the first image frame and a second motion-distorted image frame using the second image frame. The method further includes adding noise to the motion-distorted image frames to generate first and second noisy motion-distorted image frames. The method also includes performing (i) a first multi-frame processing (MFP) operation to generate a ground truth image using the motion-distorted image frames and (ii) a second MFP operation to generate an input image using the noisy motion-distorted image frames. In addition, the method includes storing the ground truth and input images as an image pair for training an artificial intelligence/machine learning (AI/ML)-based image processing operation for removing image distortions caused by handheld image capture.

公开(公告)号：US20220301184A1

公开(公告)日：2022-09-22

申请号：US17591350

申请日：2022-02-02

Applicant: Samsung Electronics Co., Ltd.

Inventor： Gyeongmin Choe ,  Yingmao Li ,  John Seokjun Lee ,  Hamid R. Sheikh ,  Michael O. Polley

IPC: G06T7/207 ,  H04N19/577 ,  H04N19/139

Abstract: A method includes obtaining multiple video frames. The method also includes determining whether a bi-directional optical flow between the multiple video frames satisfies an image quality criterion for bi-directional consistency. The method further includes identifying a non-linear curve based on pixel coordinate values from at least two of the video frames. The at least two video frames include first and second video frames. The method also includes generating interpolated video frames between the first and second video frames by applying non-linear interpolation based on the non-linear curve. In addition, the method includes outputting the interpolated video frames for presentation.

公开(公告)号：US12192673B2

公开(公告)日：2025-01-07

申请号：US17591350

申请日：2022-02-02

Applicant: Samsung Electronics Co., Ltd.

Inventor： Gyeongmin Choe ,  Yingmao Li ,  John Seokjun Lee ,  Hamid R. Sheikh ,  Michael O. Polley

IPC: H04N7/01 ,  G06T1/00 ,  G06T3/18 ,  G06T5/50 ,  G06T7/207 ,  H04N1/32 ,  H04N19/139 ,  H04N19/577

Abstract: A method includes obtaining multiple video frames. The method also includes determining whether a bi-directional optical flow between the multiple video frames satisfies an image quality criterion for bi-directional consistency. The method further includes identifying a non-linear curve based on pixel coordinate values from at least two of the video frames. The at least two video frames include first and second video frames. The method also includes generating interpolated video frames between the first and second video frames by applying non-linear interpolation based on the non-linear curve. In addition, the method includes outputting the interpolated video frames for presentation.

公开(公告)号：US20240135673A1

公开(公告)日：2024-04-25

申请号：US18049213

申请日：2022-10-23

Applicant: Samsung Electronics Co., Ltd.

Inventor： Yibo Xu ,  Weidi Liu ,  Hamid R. Sheikh ,  John Seokjun Lee

IPC: G06V10/70 ,  G06T5/00 ,  G06V10/60

CPC classification number: G06V10/70 ,  G06T5/003 ,  G06V10/60

Abstract: A method includes obtaining an under-display camera (UDC) image captured using a camera located under a display. The method also includes processing, using at least one processing device of an electronic device, the UDC image based on a machine learning model to restore the UDC image. The method further includes displaying or storing the restored image corresponding to the UDC image. The machine learning model is trained using (i) a ground truth image and (ii) a synthetic image generated using the ground truth image and a point spread function that is based on an optical transmission model of the display.

公开(公告)号：US20240062342A1

公开(公告)日：2024-02-22

申请号：US17820795

申请日：2022-08-18

Applicant: Samsung Electronics Co., Ltd.

Inventor： Devendra K. Jangid ,  John Seokjun Lee ,  Hamid R. Sheikh

IPC: G06T5/00

CPC classification number: G06T5/002 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: A method includes obtaining an input image that contains blur. The method also includes providing the input image to a trained machine learning model, where the trained machine learning model includes (i) a shallow feature extractor configured to extract one or more feature maps from the input image and (ii) a deep feature extractor configured to extract deep features from the one or more feature maps. The method further includes using the trained machine learning model to generate a sharpened output image. The trained machine learning model is trained using ground truth training images and input training images, where the input training images include versions of the ground truth training images with blur created using demosaic and noise filtering operations.

公开(公告)号：US20230252770A1

公开(公告)日：2023-08-10

申请号：US18045558

申请日：2022-10-11

Applicant: Samsung Electronics Co., Ltd.

Inventor： Tyler Luu ,  John W. Glotzbach ,  Hamid R. Sheikh ,  John Seokjun Lee ,  Youngmin Kim ,  Jun Ki Cho ,  Seung-Chul Jeon

IPC: G06V10/774 ,  G06T5/00 ,  G06T5/50 ,  G06V10/24

CPC classification number: G06V10/7747 ,  G06T5/002 ,  G06T5/006 ,  G06T5/50 ,  G06V10/24 ,  G06T2207/10144 ,  G06T2207/20081 ,  G06T2207/20221

Abstract: A method for training data generation includes obtaining a first set of image frames of a scene and a second set of image frames of the scene using multiple exposure settings. The method also includes generating an alignment map, a blending map, and an input image using the first set of image frames. The method further includes generating a ground truth image using the alignment map, the blending map, and the second set of image frames. In addition, the method includes using the ground truth image and the input image as an image pair in a training dataset when training a machine learning model to reduce image distortion and noise.

公开(公告)号：US11720782B2

公开(公告)日：2023-08-08

申请号：US17135573

申请日：2020-12-28

Applicant: Samsung Electronics Co., Ltd.

Inventor： Chenchi Luo ,  Gyeongmin Choe ,  Yingmao Li ,  Zeeshan Nadir ,  Hamid R. Sheikh ,  John Seokjun Lee ,  Youngjun Yoo

IPC: G06N3/00 ,  G06N3/045 ,  G06T7/60 ,  G06T7/44 ,  G06N3/08 ,  G06N3/042

CPC classification number: G06N3/045 ,  G06N3/042 ,  G06N3/08 ,  G06T7/44 ,  G06T7/60 ,  G06T2207/10024 ,  G06T2207/20081

Abstract: A method includes obtaining, using at least one processor of an electronic device, multiple calibration parameters associated with multiple sensors of a selected mobile device. The method also includes obtaining, using the at least one processor, an identification of multiple imaging tasks. The method further includes obtaining, using the at least one processor, multiple synthetically-generated scene images. In addition, the method includes generating, using the at least one processor, multiple training images and corresponding meta information based on the calibration parameters, the identification of the imaging tasks, and the scene images. The training images and corresponding meta information are generated concurrently, different ones of the training images correspond to different ones of the sensors, and different pieces of the meta information correspond to different ones of the imaging tasks.

公开(公告)号：US20250037237A1

公开(公告)日：2025-01-30

申请号：US18360536

申请日：2023-07-27

Applicant: Samsung Electronics Co., Ltd.

Inventor： Devendra Kumar Jangid ,  Abhiram Gnanasambandam ,  John W. Glotzbach ,  John Seokjun Lee ,  Hamid R. Sheikh

IPC: G06T3/40 ,  H04N23/80 ,  H04N23/84

Abstract: A method includes extracting multiple shallow features from a low-resolution image using a shallow feature extractor that includes a quaternion convolutional network. The method also includes extracting multiple deep features from the multiple shallow features using a deep feature extractor that includes multiple quaternion residual distillation blocks (QRDBs), where each QRDB includes a quaternion self-attention module. The method further includes reconstructing the multiple deep features into a high-resolution image. Each QRDB may further include a quaternion gated deconvolutional feed forward network (QGDFN) configured to suppress one or more of the multiple deep features.