公开(公告)号：US11562288B2

公开(公告)日：2023-01-24

申请号：US16146295

申请日：2018-09-28

Applicant: Amazon Technologies, Inc.

Inventor： Enrico Sartorello ,  Stefano Stefani ,  Nikhil Kandoi ,  Rama Krishna Sandeep Pokkunuri ,  Kalpesh N. Sutaria ,  Navneet Sabbineni ,  Ganesh Kumar Gella ,  Cheng Ran Li

IPC: G06N20/00 ,  G06N5/04

Abstract: Techniques for hosting adding and warming a host are described. In some instances, a method of determining that at least one group of hosts is to be increased by adding an additional host to the group of hosts; sending a request to the group of hosts for a list of machine learning models loaded per host of the group of hosts; receiving, from each host, the list of loaded machine learning models; loading at least a proper subset of list of loaded machine learning models into random access memory of the at least one group; receiving a request to perform an inference; routing the request to the additional host of the group of hosts; performing an inference using the additional host of the group of hosts; and providing a result of the inference to an external entity is described.