公开(公告)号：US11482134B2

公开(公告)日：2022-10-25

申请号：US16536151

申请日：2019-08-08

Applicant: KOREA ELECTRONICS TECHNOLOGY INSTITUTE

Inventor： Hye Dong Jung ,  Sang Ki Ko ,  Han Mu Park ,  Chang Jo Kim

IPC: G09B21/00 ,  G06T11/00 ,  G10L15/22 ,  G10L15/24 ,  G10L25/63 ,  G10L25/90 ,  G06V40/20 ,  G06V40/16

Abstract: Disclosed is a method of providing a sign language video reflecting an appearance of a conversation partner. The method includes recognizing a speech language sentence from speech information, and recognizing an appearance image and a background image from video information. The method further comprises acquiring multiple pieces of word-joint information corresponding to the speech language sentence from joint information database, sequentially inputting the word-joint information to a deep learning neural network to generate sentence-joint information, generating a motion model on the basis of the sentence-joint information, and generating a sign language video in which the background image and the appearance image are synthesized with the motion model. The method provides a natural communication environment between a sign language user and a speech language user.

公开(公告)号：US12293488B2

公开(公告)日：2025-05-06

申请号：US18032825

申请日：2022-11-04

Applicant: KOREA ELECTRONICS TECHNOLOGY INSTITUTE

Inventor： Sa Im Shin ,  Bo Eun Kim ,  Han Mu Park ,  Chung Il Kim

IPC: G06T5/50 ,  G06T5/77 ,  G06T11/60

Abstract: The present invention relates to an image inpainting apparatus and an image inpainting method, the image inpainting apparatus including: a background inpainting part configured to generate a background-inpainted image by carrying out inpainting on a background with respect to an input image in which a region to be inpainted is set up; an object inpainting part configured to generate an object image by carrying out inpainting on an object; and an image overlapping part configured to generate an output image by causing the background-inpainted image and the object image, which are generated, to overlap each other.

公开(公告)号：US12248759B2

公开(公告)日：2025-03-11

申请号：US17563299

申请日：2021-12-28

Applicant: Korea Electronics Technology Institute

Inventor： Jin Yea Jang ,  Han Mu Park ,  Yoon Young Jeong ,  Sa Im Shin

IPC: G06F40/58 ,  G06F40/53 ,  G06N20/00

Abstract: There are provided a method and system for automatic augmentation of gloss-based sign language translation data. A system for automatic augmentation of sign language translation training data according to an embodiment includes: a database configured to store a sequence of sign language glosses and a sequence of spoken-language words in pairs; and an augmentation module configured to augment the pairs stored in the database. Accordingly, gloss-based training data of high quality may be acquired by performing automatic augmentation for gloss-based training data for sign language translation in an efficient method in terms of time and economic aspects, and eventually, accuracy of translation between sign language glosses and sentences may be enhanced.

公开(公告)号：US11798255B2

公开(公告)日：2023-10-24

申请号：US17563205

申请日：2021-12-28

Applicant: Korea Electronics Technology Institute

Inventor： Han Mu Park ,  Jin Yea Jang ,  Yoon Young Jeong ,  Sa Im Shin

IPC: G06N20/00 ,  G06V10/42 ,  G06V10/774 ,  G06V10/84 ,  G06V40/20 ,  G09B21/00

CPC classification number: G06V10/421 ,  G06V10/7747 ,  G06V10/84 ,  G09B21/009

Abstract: There are provided a method for segmenting a sign language video by gloss to recognize a sign language sentence, and a method for training. According to an embodiment, a sign language video segmentation method receives an input of a sign language sentence video, and segments the inputted sign language sentence video by gloss. Accordingly, there is suggested a method for segmenting a sign language sentence video by gloss, analyzing various gloss sequences from the linguistic perspective, understanding meanings robustly in spite of various changes in sentences, and translating sign language into appropriate Korean sentences.

公开(公告)号：US11741755B2

公开(公告)日：2023-08-29

申请号：US16942985

申请日：2020-07-30

Applicant: Korea Electronics Technology Institute

Inventor： Sang Ki Ko ,  Hye Dong Jung ,  Han Mu Park ,  Chang Jo Kim

IPC: G06V40/00 ,  G06V40/20 ,  G06F3/01 ,  G06N3/084 ,  G10L13/00 ,  G06N3/045 ,  G10L15/24 ,  G06V10/764 ,  G06V10/82 ,  G06V10/44

CPC classification number: G06V40/28 ,  G06F3/011 ,  G06F3/017 ,  G06N3/045 ,  G06N3/084 ,  G06V10/454 ,  G06V10/764 ,  G06V10/82 ,  G10L13/00 ,  G10L15/24

Abstract: A method and apparatus for recognizing a sign language or a gesture by using a three-dimensional (3D) Euclidean distance matrix (EDM) are disclosed. The method includes a two-dimensional (2D) EDM generation step for generating a 2D EDM including information about distances between feature points of a body recognized in image information by a 2D EDM generator, a 3D EDM generation step for receiving the 2D EDM and generating a 3D EDM by using a first deep learning neural network trained with training data in which input data is a 2D EDM and correct answer data is a 3D EDM by a 3D EDM generator, and a recognition step for recognizing a sign language or a gesture based on the 3D EDM.