公开(公告)号：US20180082171A1

公开(公告)日：2018-03-22

申请号：US15421016

申请日：2017-01-31

Applicant: salesforce.com, inc.

Inventor： Stephen Joseph MERITY ,  Caiming XIONG ,  James BRADBURY ,  Richard SOCHER

IPC: G06N3/04 ,  G06N3/08 ,  G06N7/00 ,  G06F17/27

CPC classification number: G06N3/0445 ,  G06F17/277 ,  G06N3/0454 ,  G06N3/0472 ,  G06N3/08 ,  G06N3/084 ,  G06N7/005

Abstract: The technology disclosed provides a so-called “pointer sentinel mixture architecture” for neural network sequence models that has the ability to either reproduce a token from a recent context or produce a token from a predefined vocabulary. In one implementation, a pointer sentinel-LSTM architecture achieves state of the art language modeling performance of 70.9 perplexity on the Penn Treebank dataset, while using far fewer parameters than a standard softmax LSTM.