公开(公告)号：US20210347051A1

公开(公告)日：2021-11-11

申请号：US17385349

申请日：2021-07-26

Applicant: MUJIN, INC.

Inventor： Xutao YE ,  Puttichai LERTKULTANON ,  Rosen Nikolaev DIANKOV

IPC: B25J9/16 ,  B25J19/02 ,  G06T7/73 ,  H04N5/232 ,  B25J13/08 ,  B25J15/00 ,  B65G59/02 ,  G05B19/4155 ,  G06K9/62 ,  G06T7/60 ,  H04N5/225

Abstract: A system and method for motion planning is presented. The system is configured, when an object is or has been in a camera field of view of a camera, to receive first image information that is generated when the camera has a first camera pose. The system is further configured to determine, based on the first image information, a first estimate of the object structure, and to identify, based on the first estimate of the object structure or based on the first image information, an object corner. The system is further configured to cause an end effector apparatus to move the camera to a second camera pose, and to receive second image information for representing the object's structure. The system is configured to determine a second estimate of the object's structure based on the second image information, and to generate a motion plan based on at least the second estimate.

公开(公告)号：US20210312664A1

公开(公告)日：2021-10-07

申请号：US17350006

申请日：2021-06-17

Applicant: MUJIN, INC.

Inventor： Russell ISLAM ,  Xutao YE

IPC: G06T7/80 ,  G06T7/73 ,  H04N13/246

Abstract: A system and method for performing automatic camera calibration is presented. The system communicates with a first camera and a second camera, wherein a transparent platform is disposed between the two cameras. When a 3D calibration pattern is disposed on the platform, the system receives a first set of calibration images from the first camera, and a second set of calibration images from the second camera. The system determines, based on the first set of calibration images, a first set of coordinates for corners of the polyhedron. The system further determines, based on the second set of calibration images, a second set of coordinates for the corners. The system determines, based on the coordinates, a spatial relationship between the first camera and the second camera. The system further uses a description of the spatial relationship to generate a 3D model of an object other than the 3D calibration pattern.

公开(公告)号：US20210258478A1

公开(公告)日：2021-08-19

申请号：US17228783

申请日：2021-04-13

Applicant: Mujin, Inc.

Inventor： Jinze YU ,  Jose Jeronimo MOREIRA RODRIGUES ,  Xutao YE

IPC: H04N5/232 ,  H04N5/217

Abstract: A system and method for determining occlusion are presented. The system receives camera data generated by at least one camera, which includes a first camera having a first camera field of view. The camera data is generated when a stack having a plurality of objects is in the first camera field of view, and describes a stack structure formed from at least an object structure for a first object of the plurality of objects. The system identifies a target feature of or disposed on the object structure, and determines a 2D region that is co-planar with and surrounds the target feature. The system determines a 3D region defined by connecting a location of the first camera and the 2D region. The system determines, based on the camera data and the 3D region, a size of an occluding region, and determines a value of an object recognition confidence parameter.

公开(公告)号：US20210114222A1

公开(公告)日：2021-04-22

申请号：US17135261

申请日：2020-12-28

Applicant: MUJIN, INC.

Inventor： Russell ISLAM ,  Xutao YE ,  Rosen DIANKOV

IPC: B25J9/16 ,  G06T7/80

Abstract: A computing system and a method for calibration verification is presented. The computing system is configured to perform a first calibration operation, and to control a robot arm to move a verification symbol to a reference location. The robot control system further receives, from a camera, a reference image of the verification symbol, and determines a reference image coordinate for the verification symbol. The robot control system further controls the robot arm to move the verification symbol to the reference location again during an idle period, receives an additional image of the verification symbol, and determines a verification image coordinate. The robot control system determines a deviation parameter value based the reference image coordinate and the verification image coordinate, and whether the deviation parameter value exceeds a defined threshold, and performs a second calibration operation if the threshold is exceeded.

公开(公告)号：US20210019891A1

公开(公告)日：2021-01-21

申请号：US16909709

申请日：2020-06-23

Applicant: Mujin, Inc.

Inventor： Jinze YU ,  Jose Jeronimo MOREIRA RODRIGUES ,  Xutao YE

IPC: G06T7/181 ,  G06T7/174 ,  G06T7/13 ,  G06T7/33

Abstract: A system and method of detecting objects are provided. The method includes generating first edge information from first image data representing an object based on a first mode of image capture, generating second edge information from second image data representing the object based on a second mode of image capture, the second mode being different from the first mode, fusing the first edge information with the second edge information to generate fused edge information, generating an object detection hypothesis based on the fused edge information, and validating the object detection hypothesis based on the fused edge information, the first edge information, and/or the second edge information.

公开(公告)号：US20200324414A1

公开(公告)日：2020-10-15

申请号：US16749365

申请日：2020-01-22

Applicant: MUJIN, INC.

Inventor： Russell ISLAM ,  Xutao YE ,  Rosen DIANKOV

IPC: B25J9/16 ,  G06T7/80

Abstract: A robot control system and a method for updating camera calibration is presented. The method comprises the robot control system performing a first camera calibration to determine camera calibration information, and outputting a first movement command based on the camera calibration information for a robot operation. The method further comprises outputting, after the first camera calibration, a second movement command to move a calibration pattern within a camera field of view, receiving one or more calibration images, and adding the one or more calibration images to a captured image set. The method further comprises performing a second camera calibration based on calibration images in the captured image set to determine updated camera calibration information, determining whether a deviation between the camera calibration information and the updated camera calibration information exceeds a defined threshold, and outputting a notification signal if the deviation exceeds the defined threshold.