公开(公告)号：US11279022B2

公开(公告)日：2022-03-22

申请号：US16433162

申请日：2019-06-06

申请人： The Johns Hopkins University

发明人： Kelleher Guerin ,  Gregory D. Hager

IPC分类号： B25J9/16

摘要： System and methods to create an immersive virtual environment using a virtual reality system that receives parameters corresponding to a real-world robot. The real-world robot may be simulated to create a virtual robot based on the received parameters. The immersive virtual environment may be transmitted to a user. The user may supply input and interact with the virtual robot. Feedback such as the current state of the virtual robot or the real-world robot may be provided to the user. The user may train the virtual robot. The real-world robot may be programmed based on the virtual robot training.

公开(公告)号：US10807237B2

公开(公告)日：2020-10-20

申请号：US16008451

申请日：2018-06-14

申请人： THE JOHNS HOPKINS UNIVERSITY

发明人： Kelleher Guerin ,  Gregory D. Hager ,  Sebastian Riedel

IPC分类号： B25J9/16 ,  B25J13/00

摘要： Methods and systems for enabling human-machine collaborations include a generalizable framework that supports dynamic adaptation and reuse of robotic capability representations and human-machine collaborative behaviors. Specifically, a method of feedback-enabled user-robot collaboration includes obtaining a robot capability that models a robot's functionality for performing task actions, specializing the robot capability with an information kernel that encapsulates task-related parameters associated with the task actions, and providing an instance of the specialized robot capability as a robot capability element that controls the robot's functionality based on the task-related parameters. The method also includes obtaining, based on the robot capability element's user interaction requirements, user interaction capability elements, via which the robot capability element receives user input and provides user feedback, controlling, based on the task-related parameters, the robot's functionality to perform the task actions in collaboration with the user input; and providing the user feedback including task-related information generated by the robot capability element in association with the task actions.

公开(公告)号：US09652682B2

公开(公告)日：2017-05-16

申请号：US15225610

申请日：2016-08-01

申请人： The Johns Hopkins University

发明人： Gregory D. Hager ,  Nicolas Padoy

IPC分类号： G06K9/00 ,  G06K9/32 ,  G06T7/00 ,  G06T7/40 ,  A61B34/30 ,  A61B34/00 ,  B25J9/16 ,  A61B34/20

CPC分类号： G06K9/3241 ,  A61B34/30 ,  A61B34/70 ,  A61B2034/2057 ,  A61B2034/2065 ,  B25J9/1697 ,  G05B2219/39123 ,  G05B2219/4705 ,  G06T7/40 ,  G06T7/70 ,  Y10S901/46 ,  Y10S901/47

摘要： A system for detecting and tracking a curvilinear object in a three-dimensional space includes an image acquisition system including a video camera arranged to acquire a video image of the curvilinear object and output a corresponding video signal, the video image comprising a plurality n of image frames each at a respective time ti, where i=1, 2, . . . , n; and a data processing system adapted to communicate with the image acquisition system to receive the video signal. The data processing system is configured to determine a position, orientation and shape of the curvilinear object in the three-dimensional space at each time ti by forming a computational model of the curvilinear object at each time ti such that a projection of the computation model of the curvilinear object at each time ti onto a corresponding frame of the plurality of image frames of the video image matches a curvilinear image in the frame to a predetermined accuracy to thereby detect and track the curvilinear object from time t1 to time tn.

公开(公告)号：US20140160264A1

公开(公告)日：2014-06-12

申请号：US13710085

申请日：2012-12-10

申请人： THE JOHNS HOPKINS UNIVERSITY

发明人： Russell H. Taylor ,  Balazs Peter Vagvolgyi ,  Gregory D. Hager ,  Richa Rogerio

IPC分类号： G06T19/00

CPC分类号： G02B21/008 ,  A61B2090/3735 ,  A61F9/008 ,  G06T3/4038 ,  G06T7/337 ,  G06T11/00 ,  G06T2207/10056 ,  G06T2207/10101 ,  G06T2207/20221 ,  G06T2207/30041

摘要： An augmented field of view imaging system includes a microscope, an image sensor system arranged to receive images of a plurality of fields of view from the microscope as the microscope is moved across an object being viewed and to provide corresponding image signals, an image processing and data storage system configured to communicate with the image sensor system to receive the image signals and to provide augmented image signals, and at least one of an image injection system or an image display system configured to communicate with the image processing and data storage system to receive the augmented image signals and display an augmented field of view image. The image processing and data storage system is configured to track the plurality of fields of view in real time and register the plurality of fields of view to calculate a mosaic image. The augmented image signals from the image processing and data storage system provide the augmented image such that a live field of view from the microscope is composited with the mosaic image.

摘要翻译： 增强视野成像系统包括显微镜，图像传感器系统，其布置成当显微镜移动穿过正在观察的物体时从显微镜接收多个视野的图像，并提供相应的图像信号，图像处理和 数据存储系统，被配置为与图像传感器系统通信以接收图像信号并提供增强的图像信号，以及被配置为与图像处理和数据存储系统通信以接收的图像注入系统或图像显示系统中的至少一个 增强图像信号并显示增强视场图像。 图像处理和数据存储系统被配置为实时地跟踪多个视野并登记多个视场以计算马赛克图像。 来自图像处理和数据存储系统的增强图像信号提供增强图像，使得来自显微镜的实时视场与马赛克图像合成。

公开(公告)号：US20190283248A1

公开(公告)日：2019-09-19

申请号：US16433162

申请日：2019-06-06

申请人： The Johns Hopkins University

发明人： Kelleher Guerin ,  Gregory D. Hager

IPC分类号： B25J9/16

摘要： System and methods to create an immersive virtual environment using a virtual reality system that receives parameters corresponding to a real-world robot. The real-world robot may be simulated to create a virtual robot based on the received parameters. The immersive virtual environment may be transmitted to a user. The user may supply input and interact with the virtual robot. Feedback such as the current state of the virtual robot or the real-world robot may be provided to the user. The user may train the virtual robot. The real-world robot may be programmed based on the virtual robot training.

公开(公告)号：US10022870B2

公开(公告)日：2018-07-17

申请号：US15424369

申请日：2017-02-03

申请人： THE JOHNS HOPKINS UNIVERSITY

发明人： Kelleher Guerin ,  Gregory D. Hager ,  Sebastian Riedel

IPC分类号： G06F19/00 ,  G05B19/04 ,  G05B19/418 ,  G05B19/00 ,  B25J9/16

摘要： Methods and systems for enabling human-machine collaborations include a generalizable framework that supports dynamic adaptation and reuse of robotic capability representations and human-machine collaborative behaviors. Specifically, a method of feedback-enabled user-robot collaboration includes obtaining a robot capability that models a robot's functionality for performing task actions, specializing the robot capability with an information kernel that encapsulates task-related parameters associated with the task actions, and providing an instance of the specialized robot capability as a robot capability element that controls the robot's functionality based on the task-related parameters. The method also includes obtaining, based on the robot capability element's user interaction requirements, user interaction capability elements, via which the robot capability element receives user input and provides user feedback, controlling, based on the task-related parameters, the robot's functionality to perform the task actions in collaboration with the user input; and providing the user feedback including task-related information generated by the robot capability element in association with the task actions.

公开(公告)号：US20170203438A1

公开(公告)日：2017-07-20

申请号：US15480135

申请日：2017-04-05

申请人： The Johns Hopkins University

发明人： Kelleher Guerin ,  Gregory D. Hager

IPC分类号： B25J9/16

CPC分类号： B25J9/1605 ,  B25J9/1671 ,  G05B2219/40131 ,  G05B2219/45097 ,  Y10S901/05

摘要： System and methods to create an immersive virtual environment using a virtual reality system that receives parameters corresponding to a real-world robot. The real-world robot may be simulated to create a virtual robot based on the received parameters. The immersive virtual environment may be transmitted to a user. The user may supply input and interact with the virtual robot. Feedback such as the current state of the virtual robot or the real-world robot may be provided to the user. The user may train the virtual robot. The real-world robot may be programmed based on the virtual robot training.

公开(公告)号：US10350751B2

公开(公告)日：2019-07-16

申请号：US15480135

申请日：2017-04-05

申请人： The Johns Hopkins University

发明人： Kelleher Guerin ,  Gregory D. Hager

IPC分类号： B25J9/16

摘要： System and methods to create an immersive virtual environment using a virtual reality system that receives parameters corresponding to a real-world robot. The real-world robot may be simulated to create a virtual robot based on the received parameters. The immersive virtual environment may be transmitted to a user. The user may supply input and interact with the virtual robot. Feedback such as the current state of the virtual robot or the real-world robot may be provided to the user. The user may train the virtual robot. The real-world robot may be programmed based on the virtual robot training.

公开(公告)号：US09643314B2

公开(公告)日：2017-05-09

申请号：US14638973

申请日：2015-03-04

申请人： The Johns Hopkins University

发明人： Kelleher Guerin ,  Gregory D. Hager

IPC分类号： B25J9/16

CPC分类号： B25J9/1605 ,  B25J9/1671 ,  G05B2219/40131 ,  G05B2219/45097 ,  Y10S901/05

摘要： System and methods to create an immersive virtual environment using a virtual reality system that receives parameters corresponding to a real-world robot. The real-world robot may be simulated to create a virtual robot based on the received parameters. The immersive virtual environment may be transmitted to a user. The user may supply input and interact with the virtual robot. Feedback such as the current state of the virtual robot or the real-world robot may be provided to the user. The user may train the virtual robot. The real-world robot may be programmed based on the virtual robot training.

公开(公告)号：US09586315B2

公开(公告)日：2017-03-07

申请号：US15001003

申请日：2016-01-19

申请人： THE JOHNS HOPKINS UNIVERSITY

发明人： Kelleher Guerin ,  Gregory D. Hager ,  Sebastian Riedel

IPC分类号： G06F19/00 ,  G05B19/04 ,  B25J9/16 ,  B25J13/00

CPC分类号： B25J9/1664 ,  B25J9/1628 ,  B25J9/163 ,  B25J9/1661 ,  B25J13/00 ,  G05B2219/33051 ,  G05B2219/36422 ,  G05B2219/39365 ,  G05B2219/40118 ,  G05B2219/40122 ,  G05B2219/40131 ,  G05B2219/40391 ,  Y10S901/02 ,  Y10S901/04 ,  Y10S901/46

摘要： Methods and systems for enabling human-machine collaborations include a generalizable framework that supports dynamic adaptation and reuse of robotic capability representations and human-machine collaborative behaviors. Specifically, a method of enabling user-robot collaboration includes providing a composition of a robot capability that models a robot's functionality for performing a type of task action and user interaction capabilities; specializing the robot capability with an information kernel to provide a specialized robot capability, the information kernel encapsulating a set of task-related parameters associated with the type of task action; providing an instance of the specialized robot capability as a robot capability element that controls the robot's functionality based on the set of task-related parameters; providing instances of the user interaction capabilities as interaction capability elements; executing the robot capability element to receive user input via the user interaction capability elements; and controlling, based on the user input and the set of task-related parameters, the robot's functionality to perform a task action of the type of task action in collaboration with the user input.

摘要翻译： 用于实现人机协作的方法和系统包括支持机器人能力表示和人机协作行为的动态适应和重用的可概括的框架。 具体来说，启用用户 - 机器人协作的方法包括提供机器人能力的组合，其模拟机器人的功能以执行任务类型和用户交互能力; 通过信息内核专门提供机器人功能来提供专门的机器人功能，信息内核封装与任务动作类型相关联的一组任务相关参数; 提供专门的机器人能力的实例作为机器人能力元素，其基于任务相关参数的集合来控制机器人的功能; 提供用户交互能力的实例作为交互能力元素; 执行机器人能力元素以经由用户交互能力元素接收用户输入; 以及基于用户输入和一组任务相关参数来控制机器人的功能，以与用户输入协作地执行任务动作类型的任务动作。