公开(公告)号：US12299153B2

公开(公告)日：2025-05-13

申请号：US17547520

申请日：2021-12-10

Applicant: Amazon Technologies, Inc.

Inventor： Yan Valerie Leshinsky ,  Lon Lundgren ,  Stefano Stefani

IPC: H04L29/06 ,  G06F21/62 ,  H04L9/08 ,  H04L9/14

Abstract: A distributed data store may implement passive distribution encryption keys to enable access to encrypted data stored in the distributed data store. Keys to encrypt a data volume stored in the distributed data store may be encrypted according to a distribution key and provided to a client of the distributed data store. Storage nodes that maintain portions of the data volume may receive the encrypted key from a client to enable access to the data volume. The storage nodes may decrypt the key according to the distribution key and enable access to the data volume at the storage nodes. In some embodiments, a key hierarchy may be implemented to encrypt the keys that provide access to the encrypted data. The key hierarchy may include a user key.

公开(公告)号：US10534768B2

公开(公告)日：2020-01-14

申请号：US14981540

申请日：2015-12-28

Applicant: Amazon Technologies, Inc.

Inventor： Pradeep Jnana Madhavarapu ,  Raman Mittal ,  Benjamin Tobler ,  James McClellan Corey ,  Yan Valerie Leshinsky ,  Samuel James McKelvie ,  Anurag Windlass Gupta

IPC: G06F16/23 ,  G06F16/27

Abstract: A log-structured data store may implement optimized log storage for asynchronous log updates. In some embodiments, log records may be received indicating updates to data stored for a storage client and indicating positions in a log record sequence. The log records themselves may not be guaranteed to be received according to the log record sequence. Received log records may be stored in a hot log portion of a block-based storage device according to an order in which they are received. Log records in the hot log portion may then be identified to be moved to a cold log portion of the block-based storage device in order to complete a next portion of the log record sequence. Log records may be modified, such as compressed, or coalesced, before being stored together in a data block of the cold log portion according to the log record sequence.

公开(公告)号：US20190188406A1

公开(公告)日：2019-06-20

申请号：US16283517

申请日：2019-02-22

Applicant: Amazon Technologies, Inc.

Inventor： Samuel James McKelvie ,  Maximiliano Maccanti ,  Anurag Windlass Gupta ,  Pradeep Jnana Madhavarapu ,  Yan Valerie Leshinsky

IPC: G06F21/62

CPC classification number: G06F21/6218 ,  G06F21/6227 ,  G06F2221/2141

Abstract: A distributed database system may implement dynamic quorum group membership changes. In various embodiments, a quorum set may maintain a replica of a data object among group members according to a protection group policy for the data object. A group member may be identified as to be replaced. In response, a new quorum set may be created from the remaining group members and a new group member. The protection group policy may be updated to include the new group members such that subsequently received updates are maintained at both the previous to quorum set and the new quorum set. Previously received updates may be replicated on the new group member. Upon completion of replicating the previously received updates, the protection group policy for the data object may be revised such that subsequently received updates are maintained at the new quorum set.

公开(公告)号：US20220100883A1

公开(公告)日：2022-03-31

申请号：US17547520

申请日：2021-12-10

Applicant: Amazon Technologies, Inc.

Inventor： Yan Valerie Leshinsky ,  Lon Lundgren ,  Stefano Stefani

IPC: G06F21/62 ,  H04L9/08 ,  H04L9/14

Abstract: A distributed data store may implement passive distribution encryption keys to enable access to encrypted data stored in the distributed data store. Keys to encrypt a data volume stored in the distributed data store may be encrypted according to a distribution key and provided to a client of the distributed data store. Storage nodes that maintain portions of the data volume may receive the encrypted key from a client to enable access to the data volume. The storage nodes may decrypt the key according to the distribution key and enable access to the data volume at the storage nodes. In some embodiments, a key hierarchy may be implemented to encrypt the keys that provide access to the encrypted data. The key hierarchy may include a user key.

公开(公告)号：US11153380B2

公开(公告)日：2021-10-19

申请号：US16791154

申请日：2020-02-14

Applicant: Amazon Technologies, Inc.

Inventor： Yan Valerie Leshinsky ,  Lon Lundgren ,  Raman Mittal ,  Stefano Stefani

IPC: G06F15/16 ,  H04L29/08

Abstract: A distributed data store may provide continuous backup for data stored in the distributed data store. Updates to data may be replicated amongst storage nodes according to a peer-to-peer replication scheme. A backup node may participate in the peer-to-peer replication scheme to identify additional updates to be applied to a backup version of the data in a separate data store. The backup node may obtain the updates according to the peer-to-peer replication scheme and update the backup version of the data. In some embodiments, configuration changes to the data in the distributed data store may be detected via the peer-to-peer replication scheme such that a backup node can adapt performance of backup operations in conformity with the configuration change.

公开(公告)号：US10609123B2

公开(公告)日：2020-03-31

申请号：US15665080

申请日：2017-07-31

Applicant: Amazon Technologies, Inc.

Inventor： Samuel James McKelvie ,  Anurag Windlass Gupta ,  Pradeep Jnana Madhavarapu ,  James McClellan Corey ,  Yan Valerie Leshinsky

IPC: H04L29/08 ,  H04L29/06

Abstract: Hybrid quorum policies may be implemented for access requests to state maintained in a distributed system. An update to state maintained in a distributed system may be sent to multiple nodes in the distributed system maintain the state in order to satisfy a hybrid quorum policy for updating the state in the distributed system. The hybrid quorum policy may include individual quorum policies for multiple quorum sets including nodes of the distributed system. Acknowledgments received from nodes in the distributed system indicating that the update is performed may be evaluated according to the individual quorum policies to determine whether the hybrid quorum policy is satisfied. If the hybrid quorum policy is satisfied, then the update to the state may be considered durable.

公开(公告)号：US10229011B2

公开(公告)日：2019-03-12

申请号：US15413331

申请日：2017-01-23

Applicant: Amazon Technologies, Inc.

Inventor： Yan Valerie Leshinsky ,  James McClellan Corey ,  Pradeep Jnana Madhavarapu ,  Anurag Windlass Gupta ,  Benjamin Tobler ,  Samuel James McKelvie

IPC: G06F11/00 ,  G06F11/14 ,  G06F11/07 ,  G06F3/06

Abstract: A distributed database system may implement log-structured distributed storage using a single log sequence number space. A log for a data volume may be maintained in a log-structured distributed storage system. The log may be segmented across multiple protection groups according to a partitioning of user data for the data volume. Updates to the log may be assigned a log sequence number from a log sequence number space for the data volume. A protection group may be determined for an update according to which partition of user data space the update pertains. Metadata to be included with the log record may indicate a previous log sequence number of a log record maintained at the protection group. The log record may be sent to the protection group and identified as committed based on acknowledgments received from storage nodes implementing the protection group.

公开(公告)号：US10223184B1

公开(公告)日：2019-03-05

申请号：US14036792

申请日：2013-09-25

Applicant: Amazon Technologies, Inc.

Inventor： Samuel James McKelvie ,  Benjamin Tobler ,  James McClellan Corey ,  Pradeep Jnana Madhavarapu ,  Oscar Ricardo Moll Thomae ,  Christopher Richard Newcombe ,  Yan Valerie Leshinsky ,  Anurag Windlass Gupta

IPC: G06F11/00 ,  G06F11/07

Abstract: A log-structured distributed storage system may implement individual write quorums. Log records may be sent to different storage nodes of a quorum set storing data for a storage client sufficient to satisfy a write quorum requirement. For each log record, acknowledgments from storage nodes are received, and a determination is made whether the write quorum requirement is satisfied for the log record. Different log records may be maintained at different storage nodes, and still satisfy the write quorum requirement such that in some embodiments no one storage node may maintain all of the log records sent to storage nodes in the quorum set.

公开(公告)号：US20180067662A1

公开(公告)日：2018-03-08

申请号：US15811583

申请日：2017-11-13

Applicant: Amazon Technologies, Inc.

Inventor： Yan Valerie Leshinsky ,  James McCllelan Corey ,  Mustafa Abrar

IPC: G06F3/06 ,  G06F17/30

CPC classification number: G06F3/0617 ,  G06F3/064 ,  G06F3/065 ,  G06F3/0688 ,  G06F16/219 ,  G06F16/2358

Abstract: A data store maintaining data may implement memory-based on-demand data page generation. Log records may be maintained in system memory according to a version threshold for data pages of data stored in persistent storage. When a request for a version of a particular data page is received that is within the version threshold, the data page may be obtained from persistent storage and log records associated with the data page may be applied to the data page to generate the requested version of the data page. The version threshold for the data pages may change, and log records in system memory may be identified for coalesce operations. The coalesce operations may be efficiently performed (e.g., in the background), updating the data pages in persistent storage to include the changes of the identified log records and space in system memory for the identified log records may be reclaimed.

公开(公告)号：US09047189B1

公开(公告)日：2015-06-02

申请号：US13903674

申请日：2013-05-28

Applicant: Amazon Technologies, Inc.

Inventor： Anurag Windlass Gupta ,  James McClellan Corey ,  Yan Valerie Leshinsky ,  Pradeep Jnana Madhavarapu ,  Samuel James McKelvie

IPC: G06F11/00 ,  G06F11/07 ,  G06F11/10

CPC classification number: G06F11/0793 ,  G06F3/0619 ,  G06F3/064 ,  G06F3/067 ,  G06F3/0689 ,  G06F11/0709 ,  G06F11/0751 ,  G06F11/0787 ,  G06F11/079 ,  G06F11/10 ,  G06F11/1004 ,  G06F11/1076

Abstract: Self-describing data blocks of a minimum atomic write size may be stored for a data store. Data may be received for storage in a data block of a plurality of data blocks at a persistent storage device that are equivalent to a minimum atomic write size for the persistent storage device. Metadata may be generated for the data that includes an error detection code which is generated for the data and the metadata together. The data and the metadata are sent to the persistent storage device to store together in the data block. An individual atomic write operation may write together the data and the metadata in the data block. When accessed, the error detection code is applicable to detect errors. The metadata may also be applicable to determine whether the data is stored for a currently assigned purpose or a previously assigned purpose of the data block.

Abstract translation: 可以为数据存储器存储最小原子写入大小的自描述数据块。 可以接收数据以存储在永久存储设备处的多个数据块的数据块中，其等同于持久存储设备的最小原子写入大小。 可以为包括为数据和元数据生成的错误检测码的数据生成元数据。 将数据和元数据发送到持久存储设备，以一起存储在数据块中。 单独的原子写入操作可以将数据和元数据一起写入数据块中。 访问时，错误检测码适用于检测错误。 元数据还可以适用于确定数据是否存储在当前分配的目的或数据块的先前分配的目的上。