公开(公告)号：US20200258249A1

公开(公告)日：2020-08-13

申请号：US16861441

申请日：2020-04-29

Applicant: Google LLC

Inventor： Anelia Angelova ,  Martin Wicke ,  Reza Mahjourian

IPC: G06T7/579 ,  G06N3/08 ,  G06N3/04 ,  G06T7/285

Abstract: A system includes a neural network implemented by one or more computers, in which the neural network includes an image depth prediction neural network and a camera motion estimation neural network. The neural network is configured to receive a sequence of images. The neural network is configured to process each image in the sequence of images using the image depth prediction neural network to generate, for each image, a respective depth output that characterizes a depth of the image, and to process a subset of images in the sequence of images using the camera motion estimation neural network to generate a camera motion output that characterizes the motion of a camera between the images in the subset. The image depth prediction neural network and the camera motion estimation neural network have been jointly trained using an unsupervised learning technique.

公开(公告)号：US11790549B2

公开(公告)日：2023-10-17

申请号：US17826849

申请日：2022-05-27

Applicant: Google LLC

Inventor： Reza Mahjourian ,  Martin Wicke ,  Anelia Angelova

IPC: G06V10/00 ,  G06T7/579 ,  G06T7/285 ,  G06N3/084 ,  G06N3/045

CPC classification number: G06T7/579 ,  G06N3/045 ,  G06N3/084 ,  G06T7/285 ,  G06T2207/20081

Abstract: A system includes a neural network implemented by one or more computers, in which the neural network includes an image depth prediction neural network and a camera motion estimation neural network. The neural network is configured to receive a sequence of images. The neural network is configured to process each image in the sequence of images using the image depth prediction neural network to generate, for each image, a respective depth output that characterizes a depth of the image, and to process a subset of images in the sequence of images using the camera motion estimation neural network to generate a camera motion output that characterizes the motion of a camera between the images in the subset. The image depth prediction neural network and the camera motion estimation neural network have been jointly trained using an unsupervised learning technique.

公开(公告)号：US11783500B2

公开(公告)日：2023-10-10

申请号：US17272419

申请日：2019-09-05

Applicant: Google LLC

Inventor： Vincent Michael Casser ,  Soeren Pirk ,  Reza Mahjourian ,  Anelia Angelova

IPC: G06T7/00 ,  G06T7/55 ,  G06T7/246 ,  G06N3/088 ,  G06T3/00 ,  G06N3/045

CPC classification number: G06T7/55 ,  G06N3/045 ,  G06N3/088 ,  G06T3/0093 ,  G06T7/248 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: A system for generating a depth output for an image is described. The system receives input images that depict the same scene, each input image including one or more potential objects. The system generates, for each input image, a respective background image and processes the background images to generate a camera motion output that characterizes the motion of the camera between the input images. For each potential object, the system generates a respective object motion output for the potential object based on the input images and the camera motion output. The system processes a particular input image of the input images using a depth prediction neural network (NN) to generate a depth output for the particular input image, and updates the current values of parameters of the depth prediction NN based on the particular depth output, the camera motion output, and the object motion outputs for the potential objects.

公开(公告)号：US11100646B2

公开(公告)日：2021-08-24

申请号：US16562819

申请日：2019-09-06

Applicant: Google LLC

Inventor： Suhani Vora ,  Reza Mahjourian ,  Soeren Pirk ,  Anelia Angelova

IPC: G06T7/00 ,  G06T7/11 ,  G06T7/246 ,  G06T7/55 ,  G06K9/00 ,  G06N3/08

Abstract: A method for generating a predicted segmentation map for potential objects in a future scene depicted in a future image is described. The method includes receiving input images that depict a same scene; processing a current input image to generate a segmentation map for potential objects in the current input image and a respective depth map; generating a point cloud for the current input image; processing the input images to generate, for each pair of two input images in the sequence, a respective ego-motion output that characterizes motion of the camera between the two input images; processing the ego-motion outputs to generate a future ego-motion output; processing the point cloud of the current input image and the future ego-motion output to generate a future point cloud; and processing the future point cloud to generate the predicted segmentation map for potential objects in the future scene depicted in the future image.

公开(公告)号：US20210233265A1

公开(公告)日：2021-07-29

申请号：US17150291

申请日：2021-01-15

Applicant: Google LLC

Inventor： Anelia Angelova ,  Martin Wicke ,  Reza Mahjourian

IPC: G06T7/55 ,  G06T3/40 ,  G06N3/04 ,  G06N3/08 ,  G06T15/20

Abstract: A system includes an image depth prediction neural network implemented by one or more computers. The image depth prediction neural network is a recurrent neural network that is configured to receive a sequence of images and, for each image in the sequence: process the image in accordance with a current internal state of the recurrent neural network to (i) update the current internal state and (ii) generate a depth output that characterizes a predicted depth of a future image in the sequence.