公开(公告)号：US20210043110A1

公开(公告)日：2021-02-11

申请号：US16536151

申请日：2019-08-08

Applicant: KOREA ELECTRONICS TECHNOLOGY INSTITUTE

Inventor： Hye Dong JUNG ,  Sang Ki KO ,  Han Mu PARK ,  Chang Jo KIM

IPC: G09B21/00 ,  G10L25/90 ,  G10L25/63 ,  G06K9/00 ,  G10L15/22 ,  G10L15/24 ,  G06T11/00

Abstract: Disclosed is a method of providing a sign language video reflecting an appearance of a conversation partner. The method includes recognizing a speech language sentence from speech information, and recognizing an appearance image and a background image from video information. The method further comprises acquiring multiple pieces of word-joint information corresponding to the speech language sentence from joint information database, sequentially inputting the word-joint information to a deep learning neural network to generate sentence-joint information, generating a motion model on the basis of the sentence-joint information, and generating a sign language video in which the background image and the appearance image are synthesized with the motion model. The method provides a natural communication environment between a sign language user and a speech language user.

公开(公告)号：US20220415093A1

公开(公告)日：2022-12-29

申请号：US17851639

申请日：2022-06-28

Applicant: Korea Electronics Technology Institute

Inventor： Han Mu PARK ,  Jin Yea JANG ,  Sa Im SHIN

IPC: G06V40/20 ,  G06V10/774 ,  G06V20/40 ,  G06V40/10

Abstract: There are provided a method and a system for recognizing a finger language video in units of syllables based on AI. The finger language video recognition system includes: an extraction unit configured to extract posture information of a speaker from a finger language video; and a recognition unit configured to recognize a finger language of the speaker from the extracted posture information of the speaker in units of syllables, and to output a text. Accordingly, a language text in units of syllables may be generated from a finger language video, by using an AI-based syllable unit finger language recognition model.

公开(公告)号：US20240249558A1

公开(公告)日：2024-07-25

申请号：US18289958

申请日：2022-06-20

Applicant: Korea Electronics Technology Institute

Inventor： Han Mu PARK ,  Jin Yea JANG ,  Sa Im SHIN

IPC: G06V40/20 ,  G06V10/774 ,  G06V20/40 ,  G06V40/16

CPC classification number: G06V40/28 ,  G06V10/774 ,  G06V20/46 ,  G06V20/49 ,  G06V40/175

Abstract: Provided are a method for segmenting sign language into morphemes, a method for predicting morpheme positions, and a method for augmenting data. A system for analyzing sign language according to an embodiment of the present invention comprises: a recognition unit which recognizes key points of a speaker from a sign language video; and a prediction unit which inputs the recognized key points into an artificial intelligence model, segments the sign language into morphemes, and predicts position information of the segmented morphemes. Accordingly, by recognizing the morphemes of the sign language video frame by frame on the basis of a skeletal model and thereby segmenting the sign language into morphemes and predicting morpheme positions, it is possible to lay the foundations for accurate sign language translation.

公开(公告)号：US20220414350A1

公开(公告)日：2022-12-29

申请号：US17563299

申请日：2021-12-28

Applicant: Korea Electronics Technology Institute

Inventor： Jin Yea JANG ,  Han Mu PARK ,  Yoon Young JEONG ,  Sa Im SHIN

IPC: G06F40/58 ,  G06F40/53 ,  G06N20/00

Abstract: There are provided a method and system for automatic augmentation of gloss-based sign language translation data. A system for automatic augmentation of sign language translation training data according to an embodiment includes: a database configured to store a sequence of sign language glosses and a sequence of spoken-language words in pairs; and an augmentation module configured to augment the pairs stored in the database. Accordingly, gloss-based training data of high quality may be acquired by performing automatic augmentation for gloss-based training data for sign language translation in an efficient method in terms of time and economic aspects, and eventually, accuracy of translation between sign language glosses and sentences may be enhanced.

公开(公告)号：US20240378695A1

公开(公告)日：2024-11-14

申请号：US18032825

申请日：2022-11-04

Applicant: KOREA ELECTRONICS TECHNOLOGY INSTITUTE

Inventor： SA IM SHIN ,  Bo Eun KIM ,  Han Mu PARK ,  Chung Il KIM

IPC: G06T5/50 ,  G06T5/77 ,  G06T11/60

Abstract: The present invention relates to an image inpainting apparatus and an image inpainting method, the image inpainting apparatus including: a background inpainting part configured to generate a background-inpainted image by carrying out inpainting on a background with respect to an input image in which a region to be inpainted is set up; an object inpainting part configured to generate an object image by carrying out inpainting on an object; and an image overlapping part configured to generate an output image by causing the background-inpainted image and the object image, which are generated, to overlap each other.

公开(公告)号：US20220415009A1

公开(公告)日：2022-12-29

申请号：US17563205

申请日：2021-12-28

Applicant: Korea Electronics Technology Institute

Inventor： Han Mu PARK ,  Jin Yea JANG ,  Yoon Young JEONG ,  Sa Im SHIN

IPC: G06V10/42 ,  G09B21/00 ,  G06V10/774 ,  G06V10/84

Abstract: There are provided a method for segmenting a sign language video by gloss to recognize a sign language sentence, and a method for training. According to an embodiment, a sign language video segmentation method receives an input of a sign language sentence video, and segments the inputted sign language sentence video by gloss. Accordingly, there is suggested a method for segmenting a sign language sentence video by gloss, analyzing various gloss sequences from the linguistic perspective, understanding meanings robustly in spite of various changes in sentences, and translating sign language into appropriate Korean sentences.

公开(公告)号：US20210034846A1

公开(公告)日：2021-02-04

申请号：US16942985

申请日：2020-07-30

Applicant: Korea Electronics Technology Institute

Inventor： Sang Ki KO ,  Hye Dong JUNG ,  Han Mu PARK ,  Chang Jo KIM

IPC: G06K9/00 ,  G06F3/01 ,  G06N3/08 ,  G06N3/04 ,  G10L13/00

Abstract: A method and apparatus for recognizing a sign language or a gesture by using a three-dimensional (3D) Euclidean distance matrix (EDM) are disclosed. The method includes a two-dimensional (2D) EDM generation step for generating a 2D EDM including information about distances between feature points of a body recognized in image information by a 2D EDM generator, a 3D EDM generation step for receiving the 2D EDM and generating a 3D EDM by using a first deep learning neural network trained with training data in which input data is a 2D EDM and correct answer data is a 3D EDM by a 3D EDM generator, and a recognition step for recognizing a sign language or a gesture based on the 3D EDM.