公开(公告)号：US11602684B2

公开(公告)日：2023-03-14

申请号：US16751990

申请日：2020-01-24

Applicant: Sony Interactive Entertainment Inc.

Inventor： Sharwin Winesh Raghoebardajal

IPC: A63F13/213 ,  G06T7/73 ,  G06F3/039 ,  H04N5/225 ,  H04N5/235

Abstract: A system for tracking one or more peripherals operable by a user in a real environment includes: a camera, mountable upon the user, operable to capture images of the real environment, a feature identification unit operable to identify features in images belonging to a first set of images captured by the camera, a peripheral identification unit operable to identify the image position of one or more peripherals in one or more images belonging to a second set of images captured by the camera, and a position determining unit operable to generate first position information for the camera in the real environment in dependence upon one or more of the identified features, and to generate respective second position information for the one or more peripherals relative to the camera in dependence upon the identified image position of the one or more peripherals. The first set of images and the second set of images captured by the camera are interleaved such that groups of one or more images from each set are captured in turn, and the position determining unit is operable to generate third position information identifying the position of the one or more peripherals in the real environment by combining the second position information corresponding to a given group of images in the second set of images with a function of the first position information corresponding to a group of images in the first set of images preceding the given group of images and the first position information corresponding to a group of images in the first set of images following the given group of images.

公开(公告)号：US11590415B2

公开(公告)日：2023-02-28

申请号：US16604797

申请日：2018-04-26

Applicant: Sony Interactive Entertainment Inc.

Inventor： Sharwin Winesh Raghoebardajal ,  Simon Mark Benson

IPC: G06F3/16 ,  G06T19/00 ,  H04N13/332 ,  A63F13/26 ,  A63F13/213 ,  H04N13/128 ,  H04N13/15 ,  H04N13/271 ,  H04N13/344

Abstract: A system for displaying a mobile device screen comprises a head mounted display for displaying a first content to a user, a video camera mounted on the head mounted display, the video camera operable to capture a video image of a scene in front of the user, a region detection processor operable to detect a region of the captured video image comprising a mobile device screen, and an image processor operable to replace a corresponding region of the displayed first content in the head mounted display with the detected region of the captured video image comprising the mobile device screen.

公开(公告)号：US12223679B2

公开(公告)日：2025-02-11

申请号：US17609492

申请日：2020-04-24

Applicant: Sony Interactive Entertainment Inc.

Inventor： Sharwin Winesh Raghoebardajal ,  Takanori Minamino

IPC: G06T7/73 ,  G06T7/80 ,  G06T15/08 ,  H04N7/18

Abstract: A data processing device includes a plurality of cameras arranged according to a predetermined configuration, where each camera is configured to capture successive images and to detect a current location of one or more elements in the captured images with respect to a coordinate system of the camera, a processor to generate a common camera coordinate system in dependence upon the respective coordinate systems of the plurality of cameras, the common camera coordinate system comprising a set of locations in a three-dimensional space included within the field of view of the plurality of cameras, and a detector to detect a current location of a given element with respect to the common camera coordinate system based on images captured by the plurality of cameras, wherein for each camera, the processor is configured, in response to a detection by the detector that the current location of the given element corresponds to a location in the set of locations of the common camera coordinate system, to generate first data for that location in the set of locations indicative of a mapping between the coordinate system for the camera and the common camera coordinate system, and wherein in response to a detection of a location in the set of locations for which the first data is indicative of the mapping for two or more cameras with respect to the common camera coordinate system, the processor is configured to generate second data for that location indicative of a mapping between the coordinate systems for the two or more cameras.

公开(公告)号：US11229836B2

公开(公告)日：2022-01-25

申请号：US16620247

申请日：2018-05-22

Applicant: Sony Interactive Entertainment Inc.

Inventor： Simon Mark Benson ,  Sharwin Winesh Raghoebardajal ,  Patrick John Connor

IPC: A63F13/24 ,  A63F13/218

Abstract: A controller for determining a displacement of a control stick and an applied pressure on the control stick comprises at least a first control stick, one or more sensors configured to sense a pressure applied to at least one control stick and to generate corresponding pressure information, first circuitry configured to receive displacement information associated with a physical displacement of the control stick, second circuitry configured to receive from one or more of the sensors pressure information associated with the pressure applied to the control stick, and third circuitry configured to determine the displacement of the control stick and the applied pressure on the control stick; wherein at least a first sensor is configured to sense the pressure applied when the physical displacement of the control stick reaches a displacement limit.

公开(公告)号：US11033807B2

公开(公告)日：2021-06-15

申请号：US16469858

申请日：2017-12-19

Applicant: Sony Interactive Entertainment Inc.

Inventor： Sharwin Winesh Raghoebardajal

IPC: A63F13/213 ,  A63F13/211 ,  A63F13/235

Abstract: A system includes a data processing apparatus; and a position detector to detect a position of a trackable object at successive sampling time points, the position detector being configured to provide, by a first data communication path, position data indicative of the position of the trackable object to the data processing apparatus at each sampling time point along with data indicating a time of that sampling time point according to a clock signal at the position detector, and to send synchronisation data to the data processing apparatus by a second, different, data communication path indicative of the clock signal at the position detector; in which the data processing apparatus comprises: a detector to detect a difference between a clock signal at the data processing apparatus and the clock signal at the position detector in dependence upon the synchronisation data; and a processor to perform a data processing operation in dependence upon the position indicated by the position data, the time of a sampling time point associated with that position data, and the detected difference between the clock signal at the data processing apparatus and the clock signal at the position detector.

公开(公告)号：US20240179327A1

公开(公告)日：2024-05-30

申请号：US18513839

申请日：2023-11-20

Applicant: Sony Interactive Entertainment Inc.

Inventor： Sharwin Winesh Raghoebardajal ,  Jose Rubio

IPC: H04N19/167 ,  G01P15/02 ,  H04N19/46

CPC classification number: H04N19/167 ,  G01P15/02 ,  H04N19/46

Abstract: Methods and apparatus provide for capturing with a camera a wide field of view image having a field of view that is wider than a field of view subsequently displayed to the recipient, capturing at least orientation information of the camera as extrinsic camera parameters, and streaming successive images along with corresponding extrinsic camera parameters. Alternative provisions include receiving a stream comprising successive wide field of view images captured by a camera, and at least corresponding orientation information of the camera, receiving indications of at least an orientation of a head mounted display ‘HMD’ having a field of view smaller than the wide field of view images, aligning a respective predetermined main area of interest in the received images with a viewpoint corresponding at least to a predetermined orientation of the HMD, and displaying the result on the HMD, based on at least the received orientation information of the camera and the indicated orientation of the HMD.

公开(公告)号：US11367298B2

公开(公告)日：2022-06-21

申请号：US16828375

申请日：2020-03-24

Applicant: Sony Interactive Entertainment Inc.

Inventor： Sharwin Winesh Raghoebardajal

IPC: G06V40/10 ,  G02B27/00 ,  G02B27/01 ,  G06V40/20

Abstract: A system for tracking motion of a user includes one or more peripherals associated with the user, a first camera operable to be mounted upon the user's head, a second camera operable to capture images of the user and the first camera, a first tracking unit operable to determine the position of the one or more peripherals in images captured by the first camera, a second tracking unit operable to determine the position of the first camera and the one or more peripherals in images captured by the second camera, a location generating unit operable to identify the location of the one or more peripherals and the first camera in dependence upon the respective determinations by the first and second tracking units, a tracking validation unit operable to compare the locations of at least one peripheral as determined using the first and second tracking units and determine whether a difference in the locations exceeds a threshold value, and a location output unit operable to output location information for at least the one or more peripherals as identified by the location generating unit, where the location output unit is operable to output location information based upon information from the first tracking unit and information from the second tracking unit in varying contributions in dependence upon the difference in compared locations.

公开(公告)号：US11521366B2

公开(公告)日：2022-12-06

申请号：US16864413

申请日：2020-05-01

Applicant: Sony Interactive Entertainment Inc.

Inventor： Sharwin Winesh Raghoebardajal ,  Kenzo Nishikawa

IPC: G06V10/24 ,  G02B27/01 ,  G06T19/00 ,  G06V10/22 ,  G06V20/20

Abstract: A data processing device comprises an analyser to analyse successive images captured by a camera and to detect an optically detectable marker in the captured images, a first location detector to detect a location of the optically detectable marker with respect to a location of the camera according to a first detection mode and to generate a first detection result, a second location detector to detect the location of the optically detectable marker with respect to the location of the camera according to a second detection mode different to the first detection mode and to generate a second detection result, and a processor to select at least one of the first detection result and the second detection result and to generate data indicative of the location of the optically detectable marker with respect to the location of the camera based on the selection.

公开(公告)号：US11094109B2

公开(公告)日：2021-08-17

申请号：US16638613

申请日：2018-08-02

Applicant: Sony Interactive Entertainment Inc.

Inventor： Sharwin Winesh Raghoebardajal

IPC: G06T15/00 ,  G06T15/20 ,  G02B27/00 ,  G02B27/01 ,  G06F3/01 ,  G06F3/03 ,  G06F3/0346 ,  H04N5/232

Abstract: A data processing device includes a camera configured to capture successive images, each image being captured during a respective image capture period such that respective different portions of the captured image are captured at respective different capture times within the image capture period; a detector to detect, from images captured by the camera, information indicating a relative location of a remote marker with respect to the data processing device, and to associate a timestamp with a detected relative location indicating a time at which the relative location was detected; and a motion sensor to detect motion of the data processing device and to control operation of the detector in response to the detected motion.

公开(公告)号：US10996757B2

公开(公告)日：2021-05-04

申请号：US16488052

申请日：2018-02-20

Applicant: Sony Interactive Entertainment Inc.

Inventor： Sharwin Winesh Raghoebardajal ,  Simon Mark Benson

IPC: G09G5/00 ,  G06F3/01 ,  A63F13/211 ,  A63F13/212 ,  A63F13/285 ,  G06F3/16

Abstract: Virtual reality apparatus includes a display generator to generate images of a virtual environment, including a virtual representation of a display object and at least part of an avatar, for display to a user; a haptic interface including one or more actuators to provide a physical interaction with the user in response to a haptic interaction signal; a detector arrangement configured to detect two or more haptic detections applicable to a current configuration of the avatar relative to the object in the virtual environment; and a haptic generator to generate the haptic interaction signal in dependence upon the two or more haptic detections.