Reference-Based Super-Resolution for Image and Video Enhancement

    公开(公告)号:US20230098437A1

    公开(公告)日:2023-03-30

    申请号:US17658706

    申请日:2022-04-11

    Applicant: Apple Inc.

    Abstract: Devices, methods, and computer readable media to provide enhanced images in multi-camera systems, e.g., by using images captured by cameras with different optical properties and/or sensors. In one embodiment, the techniques comprise reference-based image super-resolution techniques for producing, with a first neural network employing robust feature aggregation techniques (e.g., techniques able to blend between single-image enhancement and feature aggregation, when appropriate), an enhanced output image that attempts to match the quality characteristics of each of region in a lower quality (e.g., shorter focal length, larger field of view (FOV)) input image with the quality characteristics of the region's determined guidance region from at least a second, i.e., higher quality (e.g., longer focal length, smaller FOV image) input image. The guidance regions from the higher quality image that are determined for each region from the lower quality input image may be determined by performing homographic mapping and/or semantic feature matching techniques.

    Efficient unwanted reflection artifact mitigation in videos for embedded devices

    公开(公告)号:US12177567B2

    公开(公告)日:2024-12-24

    申请号:US17805399

    申请日:2022-06-03

    Applicant: Apple Inc.

    Abstract: Devices, methods, and non-transitory computer readable media are disclosed herein to repair or mitigate the appearance of unwanted reflection artifacts in captured video image streams. These unwanted reflection artifacts often present themselves as brightly-colored spots, circles, rings, or halos that reflect the shape of a bright light source in the captured image. These artifacts, also referred to herein as “ghosts” or “green ghosts” (due to often having a greenish tint), are typically located in regions of the captured images where there is not actually a bright light source located in the image. In fact, such unwanted reflection artifacts often present themselves on the image sensor across the principal point of the lens from where the actual bright light source in the captured image is located. Such devices, methods and computer readable media may be configured to detect, track, and repair such unwanted reflection artifacts in an intelligent and efficient fashion.

    Systems and methods for multi-sensor image enhancement

    公开(公告)号:US11688100B1

    公开(公告)日:2023-06-27

    申请号:US17351869

    申请日:2021-06-18

    Applicant: Apple Inc.

    CPC classification number: G06T7/80 G06N3/04 G06T9/002 H04N23/62

    Abstract: Devices, methods, and non-transitory program storage devices are disclosed to provide enhanced images in multi-camera systems, e.g., by using information from images captured by cameras with different properties in terms of optics and/or sensors. In one embodiment, the techniques comprise: obtaining a first image from a first image capture device, wherein the first image has a first field of view (FOV) and a first set of quality characteristics; obtaining a second image from a second image capture device, wherein the second image has a second FOV and a second set of quality characteristics, and wherein the second FOV partially overlaps the first FOV; obtaining a neural network that produces a modified second image having a modified second set of quality characteristics determined by the neural network attempting to match the first set of quality characteristics; and generating an output image based, at least in part, on the modified second image.

    Efficient Unwanted Reflection Artifact Mitigation in Videos for Embedded Devices

    公开(公告)号:US20230396883A1

    公开(公告)日:2023-12-07

    申请号:US17805399

    申请日:2022-06-03

    Applicant: Apple Inc.

    Abstract: Devices, methods, and non-transitory computer readable media are disclosed herein to repair or mitigate the appearance of unwanted reflection artifacts in captured video image streams. These unwanted reflection artifacts often present themselves as brightly-colored spots, circles, rings, or halos that reflect the shape of a bright light source in the captured image. These artifacts, also referred to herein as “ghosts” or “green ghosts” (due to often having a greenish tint), are typically located in regions of the captured images where there is not actually a bright light source located in the image. In fact, such unwanted reflection artifacts often present themselves on the image sensor across the principal point of the lens from where the actual bright light source in the captured image is located. Such devices, methods and computer readable media may be configured to detect, track, and repair such unwanted reflection artifacts in an intelligent and efficient fashion.

    Reference-based super-resolution for image and video enhancement

    公开(公告)号:US12094079B2

    公开(公告)日:2024-09-17

    申请号:US17658706

    申请日:2022-04-11

    Applicant: Apple Inc.

    CPC classification number: G06T3/4046 G06T7/30 G06T2207/20212

    Abstract: Devices, methods, and computer readable media to provide enhanced images in multi-camera systems, e.g., by using images captured by cameras with different optical properties and/or sensors. In one embodiment, the techniques comprise reference-based image super-resolution techniques for producing, with a first neural network employing robust feature aggregation techniques (e.g., techniques able to blend between single-image enhancement and feature aggregation, when appropriate), an enhanced output image that attempts to match the quality characteristics of each of region in a lower quality (e.g., shorter focal length, larger field of view (FOV)) input image with the quality characteristics of the region's determined guidance region from at least a second, i.e., higher quality (e.g., longer focal length, smaller FOV image) input image. The guidance regions from the higher quality image that are determined for each region from the lower quality input image may be determined by performing homographic mapping and/or semantic feature matching techniques.

Patent Agency Ranking