公开(公告)号：US10609308B2

公开(公告)日：2020-03-31

申请号：US16235582

申请日：2018-12-28

Applicant: Sony Interactive Entertainment Inc.

Inventor： Steven Osman

IPC: H04N5/262 ,  H04N5/272 ,  H04N21/8358 ,  H04N5/232 ,  H04N21/2389 ,  H04N21/414 ,  H04N21/4223 ,  H04N21/43 ,  H04N21/44

Abstract: Methods, systems, and devices are described for presenting non-video content through a mobile device that uses a video camera to track a video on another screen. In one embodiment, a system includes a video display, such as a TV, that displays video content. A mobile device with an integrated video camera captures video data from the TV and allows a user to select an area in the video in order to hear/feel/smell what is at that location in the video.

公开(公告)号：US10543430B2

公开(公告)日：2020-01-28

申请号：US15181266

申请日：2016-06-13

Applicant: Sony Interactive Entertainment Inc.

Inventor： Steven Osman ,  Dominic Mallinson

IPC: A63F13/86 ,  A63F13/355 ,  A63F13/525 ,  A63F13/26 ,  A63F13/213

Abstract: A method is provided: comprising: receiving, by a gaming server, game state data from a player client device, the game state data being defined from a game state of a first instance of a video game that is executed by the player client device; applying, by the gaming server, the received game state data and the view direction data to a second instance of the video game that is executed by the gaming server, to generate a wide field of view (FOV) spectator video; receiving, by a video server, view direction data from a spectator client device, the view direction data being determined from a tracked pose of a head-mounted display (HMD); applying, by the video server, the view direction data to define a portion of the wide FOV spectator video, the portion of the wide FOV spectator video identifying a sub-area of the wide FOV spectator video that is less than an entire area of the wide FOV spectator video; streaming the portion of the wide FOV spectator video to the spectator client device for rendering on the HMD.

公开(公告)号：US10376785B2

公开(公告)日：2019-08-13

申请号：US15199384

申请日：2016-06-30

Applicant: Sony Interactive Entertainment Inc.

Inventor： Gustavo Hernandez-Abrego ,  Xavier Menendez-Pidal ,  Steven Osman ,  Ruxin Chen ,  Rishi Deshpande ,  Care Michaud-Wideman ,  Richard Marks ,  Eric J. Larsen ,  Xiaodong Mao

IPC: A63F13/428 ,  G10L17/04 ,  A63F13/213 ,  A63F13/217 ,  G06F3/01 ,  G10L17/00 ,  G06F3/00

Abstract: Consumer electronic devices have been developed with enormous information processing capabilities, high quality audio and video outputs, large amounts of memory, and may also include wired and/or wireless networking capabilities. Additionally, relatively unsophisticated and inexpensive sensors, such as microphones, video camera, GPS or other position sensors, when coupled with devices having these enhanced capabilities, can be used to detect subtle features about users and their environments. A variety of audio, video, simulation and user interface paradigms have been developed to utilize the enhanced capabilities of these devices. These paradigms can be used separately or together in any combination. One paradigm automatically creating user identities using speaker identification. Another paradigm includes a control button with 3-axis pressure sensitivity for use with game controllers and other input devices.

公开(公告)号：US20190168122A1

公开(公告)日：2019-06-06

申请号：US16268384

申请日：2019-02-05

Applicant: Sony Interactive Entertainment Inc.

Inventor： Steven Osman ,  Jeffrey R. Stafford ,  Javier F. Rico ,  Michael G. Taylor ,  Todd S. Tokubo

IPC: A63F13/67 ,  A63F13/795 ,  A63F13/798 ,  A63F13/5375 ,  G06N5/04 ,  A63F13/335 ,  A63F13/85

Abstract: A method for assisting game play. The method includes monitoring game play of the user playing a gaming application, wherein the user has a defined task to accomplish, wherein the task is associated with a task type. The method includes determining a task type proficiency rule for the task type based on results of a plurality of players taking on a plurality of tasks having the task type. The method includes determining a player proficiency score for accomplishing the task based on the task type proficiency rule. The method includes determining a user predictive rate of success in accomplishing the task based on the player proficiency score, the task type proficiency rule, and the task. The method includes determining a recommendation for the user based on the user predictive rate of success.

公开(公告)号：US20190141260A1

公开(公告)日：2019-05-09

申请号：US16235582

申请日：2018-12-28

Applicant: Sony Interactive Entertainment Inc.

Inventor： Steven Osman

IPC: H04N5/272 ,  H04N21/44 ,  H04N5/232 ,  H04N5/262 ,  H04N21/414 ,  H04N21/2389 ,  H04N21/43 ,  H04N21/8358 ,  H04N21/4223

Abstract: Methods, systems, and devices are described for presenting non-video content through a mobile device that uses a video camera to track a video on another screen. In one embodiment, a system includes a video display, such as a TV, that displays video content. A mobile device with an integrated video camera captures video data from the TV and allows a user to select an area in the video in order to hear/feel/smell what is at that location in the video.

公开(公告)号：US20190080519A1

公开(公告)日：2019-03-14

申请号：US16190067

申请日：2018-11-13

Applicant: Sony Interactive Entertainment Inc.

Inventor： Steven Osman

IPC: G06T19/00 ,  A63F13/25 ,  G02B27/00 ,  G06T15/60 ,  G06T13/40 ,  G06F3/01 ,  G06T17/10 ,  A63F13/00 ,  H04N13/383 ,  H04N13/344 ,  A63F13/525 ,  G06K9/00 ,  G02B27/01 ,  A63F13/803

Abstract: A method for rendering a virtual reality (VR) scene viewable via a head mounted display (HMD) is provided. The method includes detecting eye gaze of a user using one or more eye gaze sensors disposed in a display housing of the HMD. And, capturing images of a mouth of the user using one or more cameras disposed on the HMD, wherein the images of the mouth include movements of the mouth. Then, the method includes generating a virtual face of the user. The virtual face includes virtual eye movement obtained from the eye gaze of the user and virtual mouth movement obtained from said captured images of the mouth. The method includes presenting an avatar of the user in the VR scene with the virtual face. The avatar of the user is viewable by another user having access to view the VR scene from a perspective that enables viewing of the avatar having the virtual face of the user. Facial expressions and movements of the mouth of the user wearing the HMD are viewable by said other user, and the virtual face of the user is presented without the HMD.

公开(公告)号：US20180311585A1

公开(公告)日：2018-11-01

申请号：US15963047

申请日：2018-04-25

Applicant: Sony Interactive Entertainment Inc.

Inventor： Steven Osman

IPC: A63F13/655 ,  H04L29/06 ,  A63F13/26 ,  A63F13/35 ,  A63F13/40 ,  A63F13/213 ,  A63F13/525

Abstract: Methods for observing a virtual reality environment of a virtual reality player is are provided. One method includes establishing, by a handheld device of an observer, a connection with a computer executing the virtual reality environment being presented to the virtual reality player using a head mounted display (HMD). Capturing, by a first camera of the handheld device, a current position of the virtual reality player in a real world space. Capturing, by a second camera of the handheld device, a face position of the observer. The face position being monitored to determine a viewing direction to a screen of the handheld device. The screen of the handheld device is configured to present the virtual reality environment and at least part of the virtual reality player interacting in the virtual reality environment. A current position of the handheld device and the face position of the observer are continually shared with the computer executing the virtual reality environment presented to the virtual reality player. The method then enables receiving, from the computer, a video stream of the virtual reality environment that includes at least part of the virtual reality player adjusted for the current position of the virtual reality player in the real world and the viewing direction of the observer to the screen of the handheld device.

公开(公告)号：US20180311583A1

公开(公告)日：2018-11-01

申请号：US16027199

申请日：2018-07-03

Applicant: Sony Interactive Entertainment Inc.

Inventor： Steven Osman ,  Xiaodong Mao

IPC: A63F13/537 ,  A63F13/212 ,  A63F13/26 ,  A63F13/211 ,  H04N13/344 ,  G06F3/01 ,  A63F13/5255 ,  A63F13/213 ,  A63F13/00

CPC classification number: A63F13/537 ,  A63F13/00 ,  A63F13/10 ,  A63F13/211 ,  A63F13/212 ,  A63F13/213 ,  A63F13/26 ,  A63F13/5255 ,  A63F2300/6653 ,  G06F3/011 ,  G06F3/013 ,  H04N13/344

Abstract: Systems and methods for executing content to be rendered on a screen of a head mounted display (HMD) are provided. One method includes executing the content to render interactive scenes on the screen and tracking an orientation direction of the HMD when worn on a head of a user and the interactive scenes are being rendered on the screen. The method includes changing view directions into the interactive scenes based on changes in the orientation direction of the HMD, such that movements of the head of the user causes the changes in the view directions into the interactive scenes. The method further includes receiving images of a real world space using a camera of the HMD. The camera of the HMD is configured to capture a location of real world objects in the real world space relative to the user of the HMD. The method includes detecting that at least one real world object is becoming proximate to the user of the HMD and generating a warning or message to be presented to the HMD, the warning or message indicating that the user is likely to bump or contact the at least one real world object. The method further includes transitioning at least a portion of the screen to a transparent mode. The transparent mode provides at least a partial view into the real world space using the camera of the HMD.

公开(公告)号：US10019057B2

公开(公告)日：2018-07-10

申请号：US14206219

申请日：2014-03-12

Applicant: Sony Interactive Entertainment Inc.

Inventor： Steven Osman ,  Xiadong Mao ,  Jeffrey Roger Stafford

IPC: G02B27/01 ,  G06F3/01 ,  G06T19/00 ,  G02B27/00

CPC classification number: G06F3/013 ,  G02B27/0093 ,  G02B27/017 ,  G02B27/0172 ,  G02B27/0179 ,  G02B2027/0187 ,  G06F2203/04804 ,  G06T19/006

Abstract: Methods, systems, and computer programs are presented for managing the display of images on a head mounted device (HMD). One method includes an operation for tracking the gaze of a user wearing the HMD, where the HMD is displaying a scene of a virtual world. In addition, the method includes an operation for detecting that the gaze of the user is fixed on a predetermined area for a predetermined amount of time. In response to the detecting, the method fades out a region of the display in the HMD, while maintaining the scene of the virtual world in an area of the display outside the region. Additionally, the method includes an operation for fading in a view of the real world in the region as if the HMD were transparent to the user while the user is looking through the region. The fading in of the view of the real world includes maintaining the scene of the virtual world outside the region.

公开(公告)号：US10010792B2

公开(公告)日：2018-07-03

申请号：US15087801

申请日：2016-03-31

Applicant: Sony Interactive Entertainment Inc.

Inventor： Steven Osman ,  Xiaodong Mao

IPC: A63F9/24 ,  A63F13/537 ,  A63F13/00 ,  G06F3/01 ,  H04N13/04 ,  A63F13/213 ,  A63F13/5255 ,  A63F13/212 ,  A63F13/40 ,  A63F13/211 ,  A63F13/26

CPC classification number: A63F13/537 ,  A63F13/00 ,  A63F13/10 ,  A63F13/211 ,  A63F13/212 ,  A63F13/213 ,  A63F13/26 ,  A63F13/5255 ,  A63F2300/6653 ,  G06F3/011 ,  G06F3/013 ,  H04N13/344

Abstract: Systems and methods for executing content to be rendered on a screen of a head mounted display (HMD) are provided. One method includes executing the content to render interactive scenes on the screen and tracking an orientation direction of the HMD when worn on a head of a user and the interactive scenes are being rendered on the screen. The method includes changing view directions into the interactive scenes based on changes in the orientation direction of the HMD, such that movements of the head of the user causes the changes in the view directions into the interactive scenes. The method further includes receiving images of a real world space using a camera of the HMD. The camera of the HMD is configured to capture a location of real world objects in the real world space relative to the user of the HMD. The method includes detecting that at least one real world object is becoming proximate to the user of the HMD and generating a warning or message to be presented to the HMD, the warning or message indicating that the user is likely to bump or contact the at least one real world object. The method further includes transitioning at least a portion of the screen to a transparent mode. The transparent mode provides at least a partial view into the real world space using the camera of the HMD.