公开(公告)号：US11528577B2

公开(公告)日：2022-12-13

申请号：US16875115

申请日：2020-05-15

Applicant: Sony Interactive Entertainment Inc.

Inventor： Fabio Cappello ,  Marina Villanueva-Barreiro ,  Oliver Hume

IPC: H04S7/00 ,  G06T17/00 ,  G06V40/16 ,  H04R1/40 ,  G01S5/16

Abstract: A method of obtaining a head-related transfer function for a user is provided. The method comprises generating an audio signal for output by a handheld device and outputting the generated audio signal at a plurality of locations by moving the handheld device to those locations. The audio output by the handheld device is detected at left-ear and right-ear microphones. A pose of the handheld device relative to the user's head is determined for at least some of the locations. One or more personalised HRTF features are then determined based on the detected audio and corresponding determined poses of the handheld device. The one or more personalised HRTF features are then mapped to a higher-quality HRTF for the user, wherein the higher-quality HRTF corresponds to an HRTF measured in an anechoic environment. This mapping may be learned using machine learning, for example. A corresponding system is also provided.

公开(公告)号：US11334978B2

公开(公告)日：2022-05-17

申请号：US16692752

申请日：2019-11-22

Applicant: Sony Interactive Entertainment Inc.

Inventor： Udupi Ramanath Bhat ,  Yasushi Okumura ,  Fabio Cappello

IPC: G06T7/00 ,  G06T7/73 ,  A63B24/00 ,  G06N3/02

Abstract: A platform to accurately detect user pose/verify against a reference ground truth and provide feedback using an accuracy score that represents the deviation of the user pose from the reference ground truth, typically established by an expert.

公开(公告)号：US20220148584A1

公开(公告)日：2022-05-12

申请号：US17452499

申请日：2021-10-27

Applicant: Sony Interactive Entertainment Inc.

Inventor： Fabio Cappello ,  Danjeli Schembri ,  Oliver Hume

IPC: G10L15/187 ,  G06F40/279 ,  G10L15/10 ,  G11B27/02

Abstract: A data processing apparatus includes storage circuitry to store audio data for a plurality of respective dialogue recordings for a content and to store text data indicative of a sequence of respective words within the audio data for each of the plurality of respective dialogue recordings, analysis circuitry to compare the text data for a current dialogue recording with predetermined text data for the content and to output comparison data for the current dialogue recording, the comparison data indicative of one or more differences between the text data for the current dialogue recording and the predetermined text data, selection circuitry to select one or more candidate dialogue recordings from the plurality of respective dialogue recordings for the content in dependence upon the comparison data, and recording circuitry to modify at least a portion of the audio data for the current dialogue recording in dependence upon the audio data for one or more of the candidate dialogue recordings to obtain modified audio data and to store the modified audio data for the current dialogue recording.

公开(公告)号：US20220111294A1

公开(公告)日：2022-04-14

申请号：US17498947

申请日：2021-10-12

Applicant: Sony Interactive Entertainment Inc.

Inventor： Marina Villanueva Barreiro ,  Michael Lee Jones ,  Oliver Hume ,  Fabio Cappello ,  Danjeli Schembri

IPC: A63F13/54 ,  G06F40/20 ,  G10L15/16 ,  G10L15/22 ,  G10L25/60 ,  G10L25/30

Abstract: A data processing apparatus includes input circuitry to receive audio data for a plurality of respective dialogue recordings for a video game, classification circuitry comprising one or more machine learning models to receive at least a portion of the audio data for each dialogue recording and trained to output classification data indicative of a quality classification of a dialogue recording in dependence upon one or more properties of the audio data for the dialogue recording, and storage circuitry to store identification data for one or more of the plurality of dialogue recordings in dependence upon the classification data.

公开(公告)号：US20220062770A1

公开(公告)日：2022-03-03

申请号：US17408684

申请日：2021-08-23

Applicant: Sony Interactive Entertainment Inc.

Inventor： Fabio Cappello ,  Maria Chiara Monti ,  Matthew Sanders ,  Timothy Bradley ,  Oliver Hume ,  Jason Craig Millson

IPC: A63F13/63 ,  A63F13/86 ,  A63F13/215 ,  G10L15/197

Abstract: A content generation system, the system comprising an input obtaining unit operable to obtain one or more samples of input text and/or audio relating to a first content, an input analysis unit operable to generate n-grams representing one or more elements of the obtained inputs, a representation generating unit operable to generate a visual representation of one or more of the generated n-grams, and a display generation unit operable to generate second content comprising one or more elements of the visual representation in association with the first content.

公开(公告)号：US11250618B2

公开(公告)日：2022-02-15

申请号：US17075243

申请日：2020-10-20

Applicant: Sony Interactive Entertainment Inc.

Inventor： Timothy Bradley ,  Fabio Cappello

IPC: G06T15/00 ,  G06T15/20 ,  G06T7/62 ,  G06T7/70 ,  G06T19/00 ,  G06T19/20

Abstract: A method of obtaining real world scale information for a scene includes obtaining at least one image of a plurality of objects in a scene; detecting at least some of the objects in the at least one image as corresponding to pre-determined objects; generating a 3D reconstruction of the scene based on the image content of the at least one image; determining a relative size of each object in the 3D reconstruction of the scene in at least one dimension, the relative size being defined in dimensions of the generated 3D reconstruction; where the relative size of each object is determined based on a distance between at least two points corresponding to that object as transformed into 3D space; obtaining a size probability distribution function for each object detected in the at least one image, each size probability distribution function defining a range of sizes in at least one dimension that a corresponding object is likely to possess in real world units; resealing the size probability distribution function for each detected object based on a corresponding relative size of that object in the 3D reconstruction; and estimating a geometry of the scene in real world units by combining the re-scaled probability distribution function for at least one detected object with the re-scaled probability distribution function for at least one other detected object.

公开(公告)号：US20210392318A1

公开(公告)日：2021-12-16

申请号：US17340282

申请日：2021-06-07

Applicant: Sony Interactive Entertainment Inc.

Inventor： Fabio Cappello ,  Maria Chiara Monti ,  Alexander Smith

IPC: H04N13/383 ,  H04N13/344 ,  G02B27/00 ,  G02B27/01

Abstract: A gaze tracking system comprising a first camera operable to capture images of a user within an environment, a second camera, having a smaller field of view than the first camera, operable to capture images of at least one of the user's eyes, an eye identification unit operable to identify a location of at least one of the user's eyes from images captured by the first camera, a camera control unit operable to modify the position and/or orientation of the second camera in dependence upon the detected location of the at least one of the user's eyes, so as to cause the second camera to be able to capture images of at least one of the user's eyes, and a gaze direction identification unit operable to identify a gaze direction of the user from images captured by the second camera.

公开(公告)号：US20210241495A1

公开(公告)日：2021-08-05

申请号：US17268856

申请日：2019-08-06

Applicant: Sony Interactive Entertainment Inc.

Inventor： Fabio Cappello ,  Nigel John Williams

IPC: G06T7/90 ,  G06T7/50 ,  G06T15/20

Abstract: A method of reconstructing colour and depth information of a scene includes receiving a colour image of a scene and obtaining depth information of the scene. The colour and depth images are used to generate a point cloud, which is then projected to an alternative viewpoint and converted to sparse colour and depth images. Colour information is then estimated for at least some parts of the sparse colour image, resulting in a reconstructed colour image. The reconstructed colour image is used with the existing depth information to estimate depth information for the sparse depth image. In this way, colour and depth information of the scene can be estimated, and used to generate colour and depth images of the scene from a desired viewpoint. A corresponding system for reconstructing colour and depth information is also provided.

公开(公告)号：US20210158501A1

公开(公告)日：2021-05-27

申请号：US16692752

申请日：2019-11-22

Applicant: Sony Interactive Entertainment Inc.

Inventor： Udupi Ramanath Bhat ,  Yasushi Okumura ,  Fabio Cappello

IPC: G06T7/00 ,  G06T7/73 ,  A63B24/00 ,  G06N3/02

Abstract: A platform to accurately detect user pose/verify against a reference ground truth and provide feedback using an accuracy score that represents the deviation of the user pose from the reference ground truth, typically established by an expert.

公开(公告)号：US20210124996A1

公开(公告)日：2021-04-29

申请号：US17074827

申请日：2020-10-20

Applicant: Sony Interactive Entertainment Inc.

Inventor： Mark Jacobus Breugelmans ,  Oliver Hume ,  Fabio Cappello ,  Nigel John Williams

IPC: G06K9/62 ,  H04N1/44 ,  G06F21/62

Abstract: An encoding apparatus is provided. The apparatus comprises an input unit operable to obtain a plurality of training images, said training images being for use in training a machine learning model. The apparatus also comprises a label unit operable to obtain a class label associated with the training images; and a key unit operable to obtain a secret key for use in encoding the training images. The apparatus further comprises an image noise generator operable to generate, based on the obtained secret key, noise for introducing into the training images. The image noise generator is configured to generate noise that correlates with the class label associated with the training images such that a machine learning model subsequently trained with the modified training images learns to associate the introduced noise with the class label for those images. A corresponding decoding apparatus is also provided.