公开(公告)号：US12008336B2

公开(公告)日：2024-06-11

申请号：US17479195

申请日：2021-09-20

Applicant: SAMSUNG ELECTRONICS CO., LTD.

Inventor： Mei Tu ,  Fan Zhang ,  Lijie Wang

IPC: G06F40/58 ,  G06F18/25 ,  G06F40/30 ,  G06F40/44 ,  G06N3/04 ,  G06V30/262

CPC classification number: G06F40/58 ,  G06F18/25 ,  G06F40/30 ,  G06F40/44 ,  G06N3/04 ,  G06V30/274

Abstract: A method for providing multimodal translation of a content in a source language is provided. The method includes receiving a user input with respect to a translation request of text included in the content, in response to receiving the user input, acquiring a multimodal input from the content, the multimodal input including location information related to the content other multimodal inputs, generating scene information representing the multimodal input related to the content by using a fusion layer based on the location information and the other multimodal inputs, identifying a candidate word set in a target language, determining at least one candidate word from the candidate word set based on the scene information, and translating the text included in the content into the target language using a translation model based on the determined at least one candidate word.