公开(公告)号：US12033058B1

公开(公告)日：2024-07-09

申请号：US16421606

申请日：2019-05-24

Applicant: Apple Inc.

Inventor： Peter Meier ,  Tanmay Batra

IPC: G06N3/045 ,  G06F18/2115 ,  G06F18/214 ,  G06N3/082

CPC classification number: G06N3/045 ,  G06F18/2115 ,  G06F18/2148 ,  G06N3/082

Abstract: In some implementations initially training a first neural network includes inputting the training inputs and corresponding training labels into the first neural network to produce output labels, comparing the output labels to the corresponding training labels using a second neural network that learns and applies a comparison metric, and adjusting parameters of the first neural network based on the comparing. The device then inputs additional inputs into the first neural network to produce additional output labels and corresponding confidence values from the second neural network. The device selects, based on the confidence values, an automatically-labeled training set of data including a subset of the additional inputs and a corresponding subset of the additional output labels. During a second training stage, the device trains the first neural network and the second neural network using the automatically-labeled training set of data.

公开(公告)号：US20240185539A1

公开(公告)日：2024-06-06

申请号：US18438214

申请日：2024-02-09

Applicant: Apple Inc.

Inventor： Patrick S. Piemonte ,  Daniel De Rocha Rosario ,  Jason D. Gosnell ,  Peter Meier

IPC: G06T19/00 ,  G01C21/26 ,  G01C21/36 ,  G02B27/01 ,  G06T15/20 ,  G06T17/00 ,  G06T19/20

CPC classification number: G06T19/006 ,  G01C21/26 ,  G01C21/365 ,  G02B27/017 ,  G06T15/205 ,  G06T17/00 ,  G06T19/20 ,  G02B2027/0141 ,  G06T2210/56

Abstract: An AR system that leverages a pre-generated 3D model of the world to improve rendering of 3D graphics content for AR views of a scene, for example an AR view of the world in front of a moving vehicle. By leveraging the pre-generated 3D model, the AR system may use a variety of techniques to enhance the rendering capabilities of the system. The AR system may obtain pre-generated 3D data (e.g., 3D tiles) from a remote source (e.g., cloud-based storage), and may use this pre-generated 3D data (e.g., a combination of 3D mesh, textures, and other geometry information) to augment local data (e.g., a point cloud of data collected by vehicle sensors) to determine much more information about a scene, including information about occluded or distant regions of the scene, than is available from the local data.

公开(公告)号：US20240127561A1

公开(公告)日：2024-04-18

申请号：US18538656

申请日：2023-12-13

Applicant: Apple Inc.

Inventor： Peter Meier ,  Michael Kuhn ,  Frank Angermann

IPC: G06T19/00 ,  G06T7/73

CPC classification number: G06T19/006 ,  G06T7/74 ,  G06T19/00 ,  G01S19/01

Abstract: A method for representing virtual information in a view of a real environment comprises providing a virtual object having a global position and orientation with respect to a geographic global coordinate system, with first pose data on the global position and orientation of the virtual object, in a database of a server, taking an image of a real environment by a mobile device and providing second pose data as to at which position and with which orientation with respect to the geographic global coordinate system the image was taken. The method further includes displaying the image on a display of the mobile device, accessing the virtual object in the database and positioning the virtual object in the image on the basis of the first and second pose data, manipulating the virtual object or adding a further virtual object, and providing the manipulated virtual object with modified first pose data or the further virtual object with third pose data in the database.

公开(公告)号：US11954881B2

公开(公告)日：2024-04-09

申请号：US16513991

申请日：2019-07-17

Applicant: Apple Inc.

Inventor： Peter Meier ,  Tanmay Batra

IPC: G06K9/62 ,  G06F18/2115 ,  G06F18/214 ,  G06F18/23213 ,  G06F18/25 ,  G06N3/045 ,  G06N3/084 ,  G06T7/20 ,  G06T7/70 ,  G06T7/73 ,  G06V10/70 ,  G06V10/774 ,  G06V10/82 ,  G06V40/10 ,  G06V40/16 ,  G06V40/20 ,  G10L15/16

CPC classification number: G06T7/73 ,  G06F18/2115 ,  G06F18/214 ,  G06F18/23213 ,  G06F18/251 ,  G06N3/045 ,  G06N3/084 ,  G06T7/20 ,  G06T7/70 ,  G06V10/70 ,  G06V10/774 ,  G06V10/7753 ,  G06V10/82 ,  G06V40/10 ,  G06V40/11 ,  G06V40/171 ,  G06V40/20 ,  G10L15/16 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: In some implementations a neural network is trained to perform a main task using a clustering constraint, for example, using both a main task training loss and a clustering training loss. Training inputs are inputted into a main task neural network to produce output labels predicting locations of the parts of the objects in the training inputs. Data from pooled layers of the main task neural network is inputted into a clustering neural network. The main task neural network and the clustering neural network are trained based on a main task loss from the main task neural network and a clustering loss from the clustering neural network. The main task loss is determined by comparing differences between the output labels and the training labels. The clustering loss encourages the clustering network to learn to label the parts of the objects individually, e.g., to learn groups corresponding to the object parts.

公开(公告)号：US11859982B2

公开(公告)日：2024-01-02

申请号：US16938559

申请日：2020-07-24

Applicant: Apple Inc.

Inventor： Peter Meier ,  Christian Lipski

IPC: G01C21/20 ,  H04W4/33 ,  G05D1/02 ,  G06V20/10 ,  G06V20/00

CPC classification number: G01C21/206 ,  G01C21/20 ,  G05D1/0231 ,  G06V20/10 ,  G06V20/36 ,  H04W4/33

Abstract: An apparatus, method, and computer readable medium related to determining the position of a device, for example determining the pose of a camera. Varying embodiments discuss the use of sensors and captured images to construct an environment property map, which provides reference information in the form of environment properties that are associated with positions, such as camera poses. Embodiments of the disclosure discuss using the environment property map online (e.g. in real time) in order to determine the position of a device. In some embodiments, the environment property map provides a coarse position that is used to refine or limit the necessary work for determining a more precise position.

公开(公告)号：US20220375123A1

公开(公告)日：2022-11-24

申请号：US17818255

申请日：2022-08-08

Applicant: Apple Inc.

Inventor： Peter Meier ,  Michael Kuhn ,  Frank Angermann

IPC: G06T7/73 ,  G01S19/01

Abstract: A method for representing virtual information in a view of a real environment comprises providing a virtual object having a global position and orientation with respect to a geographic global coordinate system, with first pose data on the global position and orientation of the virtual object, in a database of a server, taking an image of a real environment by a mobile device and providing second pose data as to at which position and with which orientation with respect to the geographic global coordinate system the image was taken. The method further includes displaying the image on a display of the mobile device, accessing the virtual object in the database and positioning the virtual object in the image on the basis of the first and second pose data, manipulating the virtual object or adding a further virtual object, and providing the manipulated virtual object with modified first pose data or the further virtual object with third pose data in the database.

公开(公告)号：US20220270335A1

公开(公告)日：2022-08-25

申请号：US17740052

申请日：2022-05-09

Applicant: Apple Inc.

Inventor： Bo Morgan ,  Mark E. Drummond ,  Peter Meier ,  Cameron J. Dunn ,  John Christopher Russell ,  Siva Chandra Mouli Sivapurapu ,  Ian M. Richter

IPC: G06T19/00 ,  A63F13/55 ,  G06F40/14 ,  G06F3/01

Abstract: In various implementations, a device surveys a scene and presents, within the scene, a extended reality (XR) environment including one or more assets that evolve over time (e.g., change location or age). Modeling such an XR environment at various timescales can be computationally intensive, particularly when modeling the XR environment over larger timescales. Accordingly, in various implementations, different models are used to determine the environment state of the XR environment when presenting the XR environment at different timescales.

公开(公告)号：US11092690B1

公开(公告)日：2021-08-17

申请号：US15711982

申请日：2017-09-21

Applicant: Apple Inc.

Inventor： Peter Meier ,  Abhishek Sharma

IPC: G01S17/931 ,  G06N20/00

Abstract: A vehicle including one or more sensors, a light detection and ranging (lidar) sensor and a lidar prediction system. The one or more sensors include an optical sensor, a radar sensor, or both, configured to capture sensor data of a particular view. The lidar sensor is configured to capture lidar data of the particular view. The lidar prediction system includes a predictive model. The lidar prediction system is configured to generate a predicted lidar frame comprising applying the predictive model to the sensor data and send the predicted lidar frame to an external system.

公开(公告)号：US20210201594A1

公开(公告)日：2021-07-01

申请号：US17203393

申请日：2021-03-16

Applicant: Apple Inc.

Inventor： Bo Morgan ,  Mark E. Drummond ,  Peter Meier ,  Cameron J. Dunn ,  John Christopher Russell ,  Siva Chandra Mouli Sivapurapu ,  Ian M. Richter

IPC: G06T19/00 ,  G06F40/14 ,  G06F3/01 ,  A63F13/55

Abstract: In various implementations, a device surveys a scene and presents, within the scene, a extended reality (XR) environment including one or more assets that evolve over time (e.g., change location or age). Modeling such an XR environment at various timescales can be computationally intensive, particularly when modeling the XR environment over larger timescales. Accordingly, in various implementations, different models are used to determine the environment state of the XR environment when presenting the XR environment at different timescales.

公开(公告)号：US20210201108A1

公开(公告)日：2021-07-01

申请号：US17203374

申请日：2021-03-16

Applicant: Apple Inc.

Inventor： Bo Morgan ,  Mark E. Drummond ,  Peter Meier ,  Cameron J. Dunn ,  John Christopher Russell ,  Siva Chandra Mouli Sivapurapu

IPC: G06N3/00 ,  G06T19/00 ,  G06T17/10

Abstract: In one implementation, a method of generating an environment state is performed by a device including one or more processors and non-transitory memory. The method includes obtaining a first environment state of an environment, wherein the first environment state indicates the inclusion in the environment of a first asset associated with a first timescale value and a second asset associated with a second timescale value, wherein the first environment state further indicates that the first asset has a first state of the first asset and the second asset has a first state of the second asset. The method includes determining a second state of the first asset and the second asset based on the first and second timescale value. The method includes determining a second environment state that indicates that the first asset has the second state and the second asset has the second state.