公开(公告)号：US20230048501A1

公开(公告)日：2023-02-16

申请号：US17880115

申请日：2022-08-03

Applicant: Apple Inc.

Inventor： Meghan C. Welles ,  Stacey L. Matthias ,  Daniel Ulbricht ,  Jim J. Tilander ,  Mariano Merchante ,  Sarune Baceviciute

IPC: G06T19/00 ,  G06V20/20 ,  G09B5/02

Abstract: An exemplary process obtains sensor data corresponding to a physical environment including one or more physical objects. A physical property of the one or more physical objects is determined based on the sensor data. A presentation mode associated with a knowledge domain is determined. An extended reality environment including a view of the physical environment and a visualization selected based on the determined presentation mode is provided. The visualization includes virtual content associated with the knowledge domain. The virtual content is provided based on display characteristics specified by the presentation mode that depend upon the physical property of the one or more objects.

公开(公告)号：US20220345849A1

公开(公告)日：2022-10-27

申请号：US17861167

申请日：2022-07-08

Applicant: Apple Inc.

Inventor： Robert William Mayor ,  Isaac T. Miller ,  Adam S. Howell ,  Vinay R. Majjigi ,  Oliver Ruepp ,  Daniel Ulbricht ,  Oleg Naroditsky ,  Christian Lipski ,  Sean P. Cier ,  Hyojoon Bae ,  Saurabh Godha ,  Patrick J. Coleman

IPC: H04W4/024 ,  G06T7/73 ,  H04M1/724 ,  H04W4/02 ,  G06V20/10

Abstract: Location mapping and navigation user interfaces may be generated and presented via mobile computing devices. A mobile device may detect its location and orientation using internal systems, and may capture image data using a device camera. The mobile device also may retrieve map information from a map server corresponding to the current location of the device. Using the image data captured at the device, the current location data, and the corresponding local map information, the mobile device may determine or update a current orientation reading for the device. Location errors and updated location data also may be determined for the device, and a map user interface may be generated and displayed on the mobile device using the updated device orientation and/or location data.

公开(公告)号：US11468275B1

公开(公告)日：2022-10-11

申请号：US16744425

申请日：2020-01-16

Applicant: Apple Inc.

Inventor： Angela Blechschmidt ,  Daniel Ulbricht ,  Mohammad Haris Baig

IPC: G06K9/62 ,  G06N20/00 ,  G06V10/20 ,  G06N3/08

Abstract: A machine learning (ML) model is trained and used to produce a probability distribution associated with a computer vision task. The ML model uses a prior probability distribution associated with a particular image capture condition determined based on sensor data. For example, given that an image was captured by an image capture device at a particular height above the floor and angle relative to the vertical world axis, a prior probability distribution for that particular image capture device condition can be used in performing a computer vision task on the image. Accordingly, the machine learning model is given the image as input as well as the prior probability distribution for the particular image capture device condition. The use of the prior probability distribution can improve the accuracy, efficiency, or effectiveness of the ML learning model for the computer vison task.

公开(公告)号：US11430238B1

公开(公告)日：2022-08-30

申请号：US16838934

申请日：2020-04-02

Applicant: Apple Inc.

Inventor： Angela Blechschmidt ,  Mohammad Haris Baig ,  Daniel Ulbricht

IPC: G06V30/262 ,  G06K9/62 ,  G06V10/40 ,  G06N3/04 ,  G06T7/60 ,  G06N3/08

Abstract: In one implementation, a method of generating a confidence value for a result from a primary task is performed at an image processing system. The method includes obtaining, by a feature extractor portion of the neural network, a set of feature maps for an image data frame; generating a contextual information vector associated with the image data frame based on results from one or more auxiliary tasks performed on the set of feature maps by an auxiliary task sub-network portion of the neural network; performing, by a primary task sub-network portion of the neural network, a primary task on the set of feature maps for the image data frame in order to generate a primary task result; and generating a confidence value based on the contextual information vector, wherein the confidence value corresponds to a reliability metric for the primary task result.

公开(公告)号：US11308347B2

公开(公告)日：2022-04-19

申请号：US17002137

申请日：2020-08-25

Applicant: Apple Inc.

Inventor： Daniel Ulbricht ,  Thomas Olszamowski

IPC: G06K9/62 ,  G06T7/33 ,  G06T7/70 ,  G06T7/579 ,  G06T7/20 ,  G06T19/20 ,  H04N5/247

Abstract: The invention is related to a method of determining a similarity transformation between first coordinates and second coordinates of 3D features, comprising providing a first plurality of 3D features having first coordinates in a first coordinate system which is associated with a first geometrical model of a first real object, wherein the first plurality of 3D features describes physical 3D features of the first real object, providing a second coordinate system, providing image information associated with a plurality of images captured by at least one camera, for each respective 3D feature of at least part of the first plurality of 3D features, wherein the respective 3D feature is captured by at least two of the plurality of images, determining camera poses of the at least one camera in the second coordinate system while the at least two of the plurality of images are captured, determining for the respective 3D feature a second coordinate in the second coordinate system according to the at least two of the plurality of images and the camera poses, and the method further comprising determining a similarity transformation between the first coordinates and the second coordinates of the at least part of the first plurality of 3D features, wherein the similarity transformation includes at least one translation, at least one rotation, at least one scale and/or their combinations in 3D space.

公开(公告)号：US11188787B1

公开(公告)日：2021-11-30

申请号：US16582722

申请日：2019-09-25

Applicant: Apple Inc.

Inventor： Daniel Ulbricht ,  Chen-Yu Lee ,  Amit Kumar K C

IPC: G06K9/62 ,  G06T11/60 ,  G06K9/46

Abstract: Systems, methods, and computer readable media to implementing an end-to-end room layout estimation are described. A room layout estimation engine performs feature extraction on an image frame to generate a first set of coefficients for a first room layout class and a second set of coefficients for a second room layout class. Afterwards, the room layout estimation engine generates a first set of planes according to the first set of coefficients and a second set of planes according to the second set of coefficients. The room layout estimation engine generates a first prediction plane according to the first set of planes and a second prediction plane according to the second set of planes. Afterwards, the room layout estimation engine merges the first prediction plane and the second prediction plane to generate a predicted room layout for the room.

公开(公告)号：US10896548B1

公开(公告)日：2021-01-19

申请号：US16580176

申请日：2019-09-24

Applicant: Apple Inc.

Inventor： Daniel Ulbricht ,  Amit Kumar K C ,  Angela Blechschmidt ,  Chen-Yu Lee ,  Eshan Verma ,  Mohammad Haris Baig ,  Tanmay Batra

IPC: G06T19/00 ,  G06F3/01 ,  A63F13/825 ,  G02B27/01 ,  A63F13/212 ,  G06F3/03

Abstract: In one implementation, a method of including a person in a CGR experience or excluding the person from the CGR experience is performed by a device including one or more processors, non-transitory memory, and a scene camera. The method includes, while presenting a CGR experience, capturing an image of scene; detecting, in the image of the scene, a person; and determining an identity of the person. The method includes determining, based on the identity of the person, whether to include the person in the CGR experience or exclude the person from the CGR experience. The method includes presenting the CGR experience based on the determination.

公开(公告)号：US10373018B2

公开(公告)日：2019-08-06

申请号：US15028364

申请日：2013-10-08

Applicant: Apple Inc.

Inventor： Daniel Ulbricht ,  Thomas Olszamowski

IPC: G06K9/62 ,  G06T7/20 ,  G06T7/33 ,  G06T7/579 ,  G06T19/20 ,  H04N5/247 ,  G06T7/70

Abstract: The invention is related to a method of determining a similarity transformation between first coordinates and second coordinates of 3D features, comprising providing a first plurality of 3D features having first coordinates in a first coordinate system which is associated with a first geometrical model of a first real object, wherein the first plurality of 3D features describes physical 3D features of the first real object, providing a second coordinate system, providing image information associated with a plurality of images captured by at least one camera, for each respective 3D feature of at least part of the first plurality of 3D features, wherein the respective 3D feature is captured by at least two of the plurality of images, determining camera poses of the at least one camera in the second coordinate system while the at least two of the plurality of images are captured, determining for the respective 3D feature a second coordinate in the second coordinate system according to the at least two of the plurality of images and the camera poses, and the method further comprising determining a similarity transformation between the first coordinates and the second coordinates of the at least part of the first plurality of 3D features, wherein the similarity transformation includes at least one translation, at least one rotation, at least one scale and/or their combinations in 3D space.