-
公开(公告)号:US20210157845A1
公开(公告)日:2021-05-27
申请号:US16697948
申请日:2019-11-27
Applicant: Amazon Technologies, Inc.
Inventor: Jean-Pierre DODEL , Zhiheng HUANG , Xiaofei MA , Ramesh M. NALLAPATI , Krishnakumar RAJAGOPALAN , Milan SAINI , Sudipta SENGUPTA , Saurabh Kumar SINGH , Dimitrios SOULIOS , Ankit SULTANIA , Dong WANG , Zhiguo WANG , Bing XIANG , Peng XU , Yong YUAN
IPC: G06F16/901 , G06F16/903 , G06F16/2457 , G06N3/04
Abstract: Techniques for searching documents are described. An exemplary method includes receiving a document search query; querying at least one index based upon the document search query to identify matching data; fetching the identified matched data; determining one or more of a top ranked passage and top ranked documents from the set of documents based upon one or more invocations of one or more machine learning models based at least on the fetched identified matched data and the document search query; and returning one or more of the top ranked passage and the proper subset of documents.
-
公开(公告)号:US20200004597A1
公开(公告)日:2020-01-02
申请号:US16020810
申请日:2018-06-27
Applicant: Amazon Technologies, Inc.
Inventor: Sudipta SENGUPTA , Poorna Chand Srinivas PERUMALLA , Dominic Rajeev DIVAKARUNI , Nafea BSHARA , Leo Parker DIRAC , Bratin SAHA , Matthew James WOOD , Andrea OLGIATI , Swaminathan SIVASUBRAMANIAN
Abstract: Implementations detailed herein include description of a computer-implemented method. In an implementation, the method at least includes provisioning an application instance and portions of at least one accelerator attached to the application instance to execute a machine learning model of an application of the application instance; loading the machine learning model onto the portions of the at least one accelerator; receiving scoring data in the application; and utilizing each of the portions of the attached at least one accelerator to perform inference on the scoring data in parallel and only using one response from the portions of the accelerator
-
公开(公告)号:US20200004595A1
公开(公告)日:2020-01-02
申请号:US16020819
申请日:2018-06-27
Applicant: Amazon Technologies, Inc.
Inventor: Sudipta SENGUPTA , Poorna Chand Srinivas PERUMALLA , Dominic Rajeev DIVAKARUNI , Nafea BSHARA , Leo Parker DIRAC , Bratin SAHA , Matthew James WOOD , Andrea OLGIATI , Swaminathan SIVASUBRAMANIAN
Abstract: Implementations detailed herein include description of a computer-implemented method. In an implementation, the method at least includes attaching a first set of one or more accelerator slots of an accelerator appliance to an application instance of a multi-tenant provider network according to an application instance configuration, the application instance configuration to define per accelerator slot capabilities to be used by an application of the application instance, wherein the multi-tenant provider network comprises a plurality of computing devices configured to implement a plurality of virtual compute instances, and wherein the first set of one or more accelerator slots is implemented using physical accelerator resources accessible to the application instance; while performing inference using the loaded machine learning model of the application using the first set of one or more accelerator slots on the attached accelerator appliance, managing resources of the accelerator appliance using an accelerator appliance manager of the accelerator appliance.
-
公开(公告)号:US20200005124A1
公开(公告)日:2020-01-02
申请号:US16020788
申请日:2018-06-27
Applicant: Amazon Technologies, Inc.
Inventor: Sudipta SENGUPTA , Poorna Chand Srinivas PERUMALLA , Dominic Rajeev DIVAKARUNI , Nafea BSHARA , Leo Parker DIRAC , Bratin SAHA , Matthew James WOOD , Andrea OLGIATI , Swaminathan SIVASUBRAMANIAN
Abstract: Implementations detailed herein include description of a computer-implemented method. In an implementation, the method at least includes receiving an application instance configuration, an application of the application instance to utilize a portion of an attached accelerator during execution of a machine learning model and the application instance configuration including an arithmetic precision of the machine learning model to be used in determining the portion of the accelerator to provision; provisioning the application instance and the portion of the accelerator attached to the application instance, wherein the application instance is implemented using a physical compute instance in a first location, wherein the portion of the accelerator is implemented using a physical accelerator in the second location; loading the machine learning model onto the portion of the accelerator; and performing inference using the loaded machine learning model of the application using the portion of the accelerator on the attached accelerator.
-
公开(公告)号:US20210174238A1
公开(公告)日:2021-06-10
申请号:US16578060
申请日:2019-09-20
Applicant: Amazon Technologies, Inc.
Inventor: Sangil SONG , Yongsik YOON , Kamal Kant GUPTA , Saileshwar KRISHNAMURTHY , Stefano STEFANI , Sudipta SENGUPTA , Jaeyun NOH
IPC: G06N20/00 , G06F16/242 , G06F16/2453 , G06N5/04
Abstract: Techniques for making machine learning inference calls for database query processing are described. In some embodiments, a method of making machine learning inference calls for database query processing may include generating a first batch of machine learning requests based at least on a query to be performed on data stored in a database service, wherein the query identifies a machine learning service, sending the first batch of machine learning requests to an input buffer of an asynchronous request handler, the asynchronous request handler to generate a second batch of machine learning requests based on the first batch of machine learning requests, and obtaining a plurality of machine learning responses from an output buffer of the asynchronous request handler, the machine learning responses generated by the machine learning service using a machine learning model in response to receiving the second batch of machine learning requests.
-
公开(公告)号:US20210157857A1
公开(公告)日:2021-05-27
申请号:US16698080
申请日:2019-11-27
Applicant: Amazon Technologies, Inc.
Inventor: Cicero NOGUEIRA DOS SANTOS , Xiaofei MA , Peng XU , Ramesh M. NALLAPATI , Bing XIANG , Sudipta SENGUPTA , Zhiguo WANG , Patrick NG
IPC: G06F16/9032 , G06N20/00 , G06F16/9038 , G06K9/62
Abstract: Techniques for generation of synthetic queries from customer data for training of document querying machine learning (ML) models as a service are described. A service may receive one or more documents from a user, generate a set of question and answer pairs from the one or more documents from the user using a machine learning model trained to predict a question from an answer, and store the set of question and answer pairs generated from the one or more documents from the user. The question and answer pairs may be used to train another machine learning model, for example, a document ranking model, a passage ranking model, a question/answer model, or a frequently asked question (FAQ) model.
-
公开(公告)号:US20200004596A1
公开(公告)日:2020-01-02
申请号:US16020776
申请日:2018-06-27
Applicant: Amazon Technologies, Inc.
Inventor: Sudipta SENGUPTA , Poorna Chand Srinivas PERUMALLA , Dominic Rajeev DIVAKARUNI , Nafea BSHARA , Leo Parker DIRAC , Bratin SAHA , Matthew James WOOD , Andrea OLGIATI , Swaminathan SIVASUBRAMANIAN
Abstract: Implementations detailed herein include description of a computer-implemented method. In an implementation, the method at least includes receiving an application instance configuration, an application of the application instance to utilize a portion of an attached accelerator during execution of a machine learning model and the application instance configuration including: an indication of the central processing unit (CPU) capability to be used, an arithmetic precision of the machine learning model to be used, an indication of the accelerator capability to be used, a storage location of the application, and an indication of an amount of random access memory to use.
-
-
-
-
-
-