公开(公告)号：US10257288B2

公开(公告)日：2019-04-09

申请号：US14570900

申请日：2014-12-15

Applicant: Amazon Technologies, Inc.

Inventor： Wei Xiao ,  David Alan Lutz ,  Timothy Andrew Rath ,  Maximiliano Maccanti ,  Miguel Mascarenhas Filipe ,  David Craig Yanacek

IPC: H04L29/08 ,  H04L29/06 ,  G06F11/34 ,  G06F9/50

Abstract: A system that provides services to clients may receive and service requests, various ones of which may require different amounts of work. The system may determine whether it is operating in an overloaded or underloaded state based on a current work throughput rate, a target work throughput rate, a maximum request rate, or an actual request rate, and may dynamically adjust the maximum request rate in response. For example, if the maximum request rate is being exceeded, the maximum request rate may be raised or lowered, dependent on the current work throughput rate. If the target or committed work throughput rate is being exceeded, but the maximum request rate is not being exceeded, a lower maximum request rate may be proposed. Adjustments to the maximum request rate may be made using multiple incremental adjustments. Service request tokens may be added to a leaky token bucket at the maximum request rate.

公开(公告)号：US10158709B1

公开(公告)日：2018-12-18

申请号：US14745278

申请日：2015-06-19

Applicant: Amazon Technologies, Inc.

Inventor： Kiran-Kumar Muniswamy-Reddy ,  Wei Xiao

IPC: G06F9/54 ,  H04L29/08 ,  G06F17/30

Abstract: A data storage system that implements identifying data store requests for asynchronous processing. A request may be received at a frontend task engine that processes requests for a network-based data store. The frontend task engine may evaluate the request to determine whether the request should be processed asynchronously. For a request identified for asynchronous processing, the task engine may initiate processing of the request at backend task engines for the network-based data store. Resources for processing the request at the frontend task engine may be reclaimed for processing other requests. A task sweeper that collects data for the network-based data store may detect that processing of the request has completed. The task sweeper may also provide an indication of the completion of the request.

公开(公告)号：US10102228B1

公开(公告)日：2018-10-16

申请号：US14182248

申请日：2014-02-17

Applicant: Amazon Technologies, Inc.

Inventor： Xianglong Huang ,  Yijun Lu ,  Wei Xiao ,  Jiandan Zheng

IPC: G06F17/30

Abstract: One or more table partitions may communicate with one or more index partitions. A communications channel may be dedicated to each combination of table partition and index partition. The communications channel may act as a queue of messages corresponding to updates to data maintained in a table partition that have yet to be processed by a corresponding index partition. Upon splitting a table partition, communications channels may be duplicated including unprocessed messages. Connections between new table partitions and an index may be finalized upon the index's receipt of a message indicative of the new table partition.

公开(公告)号：US10025673B1

公开(公告)日：2018-07-17

申请号：US14032870

申请日：2013-09-20

Applicant: Amazon Technologies, Inc.

Inventor： Maximiliano Maccanti ,  Timothy Andrew Rath ,  Rama Krishna Sandeep Pokkunuri ,  Akshat Vig ,  Clarence Wing Yin Ng ,  Srivaths Badrinath Copparam ,  Rajaprabhu Thiruchi Loganathan ,  Wei Xiao ,  William Alexander Stevenson

IPC: G06F17/30 ,  G06F11/14

Abstract: A system that implements a data storage service may store data for database tables in multiple replicated partitions on respective storage nodes. In response to a request to restore a given table that was backed up in a remote storage system (e.g., key-value durable storage system), the service may create a new table, and may import a copy of each of the partitions of the given table from the remote storage system into the new table. The request to restore the table may specify a modified value for a configuration parameter for the table or for one of its partitions. The service may apply the new configuration parameter value to the table or its partitions during the restore operation. The new configuration parameter value may indicate an increase or decrease in storage capacity or throughput capacity, and its application may automatically trigger a partition split or move operation.

公开(公告)号：US09996573B1

公开(公告)日：2018-06-12

申请号：US14222377

申请日：2014-03-21

Applicant: Amazon Technologies, Inc.

Inventor： Akshat Vig ,  Wei Xiao ,  Somasundaram Perianayagam ,  Timothy Andrew Rath ,  Barry Bailey Hunter, Jr. ,  Kiran-Kumar Muniswamy-Reddy ,  Yijun Lu ,  Qiang Liu ,  Ying Lin ,  Stuart Henry Seelye Marshall

IPC: G06F17/30

CPC classification number: G06F17/30584

Abstract: A hosted service may limit access to a table initially comprising one or more partitions. Access to the table may be limited to a provisioned capacity. A client of the service may request an increased capacity. A minimum number of partitions for providing the increased capacity may be determined. Proportions of the increased capacity may be allocated among members of successive generations of partitions to be provided by a member of a generation or its descendants. The proportions may be allocated to minimize the costs associated with splitting partitions based on the minimum number of partitions.

公开(公告)号：US20170228417A1

公开(公告)日：2017-08-10

申请号：US15495815

申请日：2017-04-24

Applicant: Amazon Technologies, Inc.

Inventor： Maximiliano Maccanti ,  Timothy Andrew Rath ,  Rama Krishna Sandeep Pokkunuri ,  Akshat Vig ,  Clarence Wing Yin NG ,  Srivaths Badrinath Copparam ,  Rajaprabhu Thiruchi Loganathan ,  Wei Xiao ,  William Alexander Stevenson

IPC: G06F17/30 ,  G06F11/14

CPC classification number: G06F16/2365 ,  G06F11/1451 ,  G06F11/1464 ,  G06F11/1469 ,  G06F11/2094 ,  G06F16/24561 ,  G06F16/278 ,  G06F2201/80 ,  G06F2201/84

Abstract: A system that implements a data storage service may store data for database tables in multiple replicated partitions on respective storage nodes. In response to a request to back up a table, the service may export individual partitions of the table from the database and package them to be independently uploaded (e.g., in parallel) to a remote storage system (e.g., a key-value durable storage system). Prior to uploading the exported and packaged partitions to the remote storage system, the service may verify that the exported and packaged partitions can be subsequently restored, which may include unpackaging and/or re-inflating the exported and packaged partitions to create additional unpackaged copies of the partitions, re-importing the additional unpackaged copies of the partitions into the database (e.g., as additional replicas), and/or comparing checksums generated for the exported partitions with checksums generated for the additional unpackaged copies of the partitions.

公开(公告)号：US09632878B1

公开(公告)日：2017-04-25

申请号：US14032894

申请日：2013-09-20

Applicant: Amazon Technologies, Inc.

Inventor： Maximiliano Maccanti ,  Timothy Andrew Rath ,  Rama Krishna Sandeep Pokkunuri ,  Akshat Vig ,  Clarence Wing Yin Ng ,  Srivaths Badrinath Copparam ,  Rajaprabhu Thiruchi Loganathan ,  Wei Xiao ,  William Alexander Stevenson

IPC: G06F7/00 ,  G06F17/00 ,  G06F11/14

CPC classification number: G06F11/1469 ,  G06F11/1464 ,  G06F11/2094 ,  G06F2201/80

Abstract: A system that implements a data storage service may store data for database tables in multiple replicated partitions on respective storage nodes. In response to a request to back up a table, the service may export individual partitions of the table from the database and package them to be independently uploaded (e.g., in parallel) to a remote storage system (e.g., a key-value durable storage system). Prior to uploading the exported and packaged partitions to the remote storage system, the service may verify that the exported and packaged partitions can be subsequently restored, which may include unpackaging and/or re-inflating the exported and packaged partitions to create additional unpackaged copies of the partitions, re-importing the additional unpackaged copies of the partitions into the database (e.g., as additional replicas), and/or comparing checksums generated for the exported partitions with checksums generated for the additional unpackaged copies of the partitions.

公开(公告)号：US09507818B1

公开(公告)日：2016-11-29

申请号：US14092779

申请日：2013-11-27

Applicant: Amazon Technologies, Inc.

Inventor： Stefano Stefani ,  Chiranjeeb Buragohain ,  Jai Vasanth ,  Wei Xiao

IPC: G06F17/30

CPC classification number: G06F17/30371 ,  G06F17/30002 ,  G06F17/30345 ,  G06F17/30365 ,  G06F17/30584

Abstract: A system that implements a scaleable data storage service may maintain tables in a non-relational data store on behalf of clients. Each table may include multiple items. Each item may include one or more attributes, each containing a name-value pair. Attribute values may be scalars or sets of numbers or strings. The system may provide an API usable to request that values of one or more of an item's attributes be updated. An update request may be conditional on expected values of one or more item attributes (e.g., the same or different item attributes). In response to a request to update the values of one or more item attributes, the previous values and/or updated values may be optionally returned for the updated item attributes or for all attributes of an item targeted by an update request. Items stored in tables may be indexed using a simple or composite primary key.

公开(公告)号：US09460185B2

公开(公告)日：2016-10-04

申请号：US14733887

申请日：2015-06-08

Applicant: Amazon Technologies, Inc.

Inventor： Bjorn Patrick Swift ,  Wei Xiao ,  Stuart Henry Seelye Marshall ,  Stefano Stefani ,  Timothy Andrew Rath ,  David Alan Lutz

IPC: G06F17/30

CPC classification number: G06F17/30584 ,  G06F17/30575

Abstract: A system that implements a data storage service may store data in multiple replicated partitions on respective storage nodes. The selection of the storage nodes (or storage devices thereof) on which to store the partition replicas may be performed by administrative components that are responsible for partition management and resource allocation for respective groups of storage nodes (e.g., based on a global view of resource capacity or usage), or the selection of particular storage devices of a storage node may be determined by the storage node itself (e.g., based on a local view of resource capacity or usage). Placement policies applied at the administrative layer or storage layer may be based on the percentage or amount of provisioned, reserved, or available storage or IOPS capacity on each storage device, and particular placements (or subsequent operations to move partition replicas) may result in an overall resource utilization that is well balanced.

公开(公告)号：US09330158B1

公开(公告)日：2016-05-03

申请号：US13898201

申请日：2013-05-20

Applicant: Amazon Technologies, Inc.

Inventor： Wei Xiao ,  Miguel Mascarenhas Filipe ,  Bjorn Patrick Swift ,  Kiran-Kumar Muniswamy-Reddy ,  Stuart Henry Seelye Marshall ,  Timothy Andrew Rath ,  Rande A. Blackman

IPC: G06F7/00 ,  G06F17/30

CPC classification number: G06F17/30584 ,  H04L67/1008 ,  H04L67/1095 ,  H04L67/1097

Abstract: Distributed database management systems may perform range queries over the leading portion of a primary key. Non-random distribution of data may improve performance related to the processing of range queries, but may tend to cause workload to be concentrated on particular partitions. Groups of partitions may be expanded and collapsed based on detection of disproportionate workload. Disproportionate write workload may be distributed among a group of partitions that can subsequently be queried using a federated approach. Disproportionate read workload may be distributed among a group of read-only replicated partitions.

Abstract translation: 分布式数据库管理系统可以在主键的前导部分上执行范围查询。 数据的非随机分布可以提高与范围查询的处理相关的性能，但可能倾向于使工作负载集中在特定分区上。 基于检测不成比例的工作负载，可能会将分区组扩展和折叠。 不成比例的写入工作负载可能会分布在一组分区中，随后可以使用联合方法进行查询。 不成比例的读取工作负载可能会分布在一组只读复制分区中。