公开(公告)号：US20220093088A1

公开(公告)日：2022-03-24

申请号：US17031798

申请日：2020-09-24

Applicant: Apple Inc.

Inventor： Vivek Kumar RANGARAJAN SRIDHAR ,  Barada Prasanna Acharya ,  Douglas Ross Davidson ,  Xingwen XU

IPC: G10L15/197 ,  G10L15/18

Abstract: Methods and systems for embedding natural language sentences within a highly-dimensional vector space are provided. Additionally, various applications relating to natural language processing, are provided. Such applications include digital assistants and search engines, as well as systems for classifying, sorting, organizing, and/or pairing content that are associated with natural language objects. The sentence vector embeddings encode various semantic features of the sentence. Two separate language models, arranged in a serial architecture are employed to generate a sentence vector. The first language model generates token vectors for each of the tokens included in the sentence. The token vectors are employed as inputs to the second language model. The second language model generates the sentence vector for the sentence. A sentence vector embeds the semantic context of the corresponding natural language object within the vector space. The second language model may be trained via supervised learning on multiple semantic-related tasks.