公开(公告)号：US10691886B2

公开(公告)日：2020-06-23

申请号：US15888442

申请日：2018-02-05

Applicant: SAMSUNG ELECTRONICS CO., LTD.

Inventor： Seung-hak Yu ,  Nilesh Kulkarni ,  Hee-jun Song ,  Hae-jun Lee

IPC: G06F40/284 ,  G06F40/211 ,  G06F40/20 ,  G06N3/04 ,  G06N3/08 ,  G06F40/216 ,  G06N5/02

Abstract: An electronic apparatus for compressing a language model is provided, the electronic apparatus including a storage configured to store a language model which includes an embedding matrix and a softmax matrix generated by a recurrent neural network (RNN) training based on basic data including a plurality of sentences, and a processor configured to convert the embedding matrix into a product of a first projection matrix and a shared matrix, the product of the first projection matrix and the shared matrix having a same size as a size of the embedding matrix, and to convert a transposed matrix of the softmax matrix into a product of a second projection matrix and the shared matrix, the product of the second projection matrix and the shared matrix having a same size as a size of the transposed matrix of the softmax matrix, and to update elements of the first projection matrix, the second projection matrix and the shared matrix by performing the RNN training with respect to the first projection matrix, the second projection matrix and the shared matrix based on the basic data.

公开(公告)号：US11568240B2

公开(公告)日：2023-01-31

申请号：US16613317

申请日：2018-05-16

Applicant: Samsung Electronics Co., Ltd.

Inventor： Hee-jun Song ,  Nilesh Kulkarni

IPC: G06N3/08 ,  G06F40/30 ,  G06N3/04

Abstract: Provided are a method and apparatus for classifying a sentence into a class by using a deep neural network. The method includes respectively training first and second sentences by using first and second neural networks, obtaining a contrastive loss based on first and second feature vectors generated as output data of the training, and information about whether classes to which the first and second sentences belong are the same, and repeating the training in such a manner that the contrastive loss has a maximum value.

公开(公告)号：US20180260379A1

公开(公告)日：2018-09-13

申请号：US15888442

申请日：2018-02-05

Applicant: Samsung Electronics Co., Ltd.

Inventor： Seung-hak Yu ,  Nilesh Kulkarni ,  Hee-jun SONG ,  Hae-jun Lee

IPC: G06F17/27 ,  G06N3/08 ,  G06N3/04

Abstract: An electronic apparatus for compressing a language model is provided, the electronic apparatus including a storage configured to store a language model which includes an embedding matrix and a softmax matrix generated by a recurrent neural network (RNN) training based on basic data including a plurality of sentences, and a processor configured to convert the embedding matrix into a product of a first projection matrix and a shared matrix, the product of the first projection matrix and the shared matrix having a same size as a size of the embedding matrix, and to convert a transposed matrix of the softmax matrix into a product of a second projection matrix and the shared matrix, the product of the second projection matrix and the shared matrix having a same size as a size of the transposed matrix of the softmax matrix, and to update elements of the first projection matrix, the second projection matrix and the shared matrix by performing the RNN training with respect to the first projection matrix, the second projection matrix and the shared matrix based on the basic data.