公开(公告)号：US20170031749A1

公开(公告)日：2017-02-02

申请号：US15289907

申请日：2016-10-10

Applicant: Amazon Technologies, Inc.

Inventor： ANURAG WINDLASS GUPTA ,  JAMES MCCLELLAN COREY ,  YAN VALERIE LESHINSKY ,  PRADEEP JNANA MADHAVARAPU ,  SAMUEL JAMES MCKELVIE

IPC: G06F11/07 ,  G06F3/06

CPC classification number: G06F11/0793 ,  G06F3/0619 ,  G06F3/064 ,  G06F3/067 ,  G06F3/0689 ,  G06F11/0709 ,  G06F11/0751 ,  G06F11/0787 ,  G06F11/079 ,  G06F11/10 ,  G06F11/1004 ,  G06F11/1076

Abstract: Self-describing data blocks of a minimum atomic write size may be stored for a data store. Data may be received for storage in a data block of a plurality of data blocks at a persistent storage device that are equivalent to a minimum atomic write size for the persistent storage device. Metadata may be generated for the data that includes an error detection code which is generated for the data and the metadata together. The data and the metadata are sent to the persistent storage to device to store together in the data block. An individual atomic write operation may write together the data and the metadata in the data block. When accessed, the error detection code is applicable to detect errors. The metadata may also be applicable to determine whether the data is stored for a currently assigned purpose or a previously assigned purpose of the data block.

Abstract translation: 可以为数据存储器存储最小原子写入大小的自描述数据块。 可以接收数据以存储在永久存储设备处的多个数据块的数据块中，其等同于持久存储设备的最小原子写入大小。 可以为包括为数据和元数据生成的错误检测码的数据生成元数据。 数据和元数据被发送到持久存储器到设备以一起存储在数据块中。 单独的原子写入操作可以将数据和元数据一起写入数据块中。 访问时，错误检测码适用于检测错误。 元数据还可以适用于确定数据是否存储在当前分配的目的或数据块的先前分配的目的上。

公开(公告)号：US20170024315A1

公开(公告)日：2017-01-26

申请号：US15284414

申请日：2016-10-03

Applicant: Amazon Technologies, Inc.

Inventor： YAN VALERIE LESHINSKY ,  JAMES MCCLELLAN COREY ,  SAMUEL JAMES MCKELVIE ,  OSCAR RICARDO MOLL THOMAE ,  PRADEEP JNANA MADHAVARAPU

IPC: G06F12/02 ,  G06F17/30

CPC classification number: G06F12/0261 ,  G06F12/0238 ,  G06F12/0253 ,  G06F16/1805 ,  G06F16/22 ,  G06F2212/1024 ,  G06F2212/1044 ,  G06F2212/163 ,  G06F2212/214 ,  G06F2212/7205

Abstract: A log-structured data store may implement efficient garbage collection. Log records may be maintained in data blocks according to a log record sequence. Based, at least in part, on a log reclamation point, the log records may be evaluated to identify data blocks to reclaim that have log records in the log sequence prior to the log reclamation point. New versions of data pages updated by log records in the identified data blocks may be generated and stored in base page storage for the log structured data store. The identified data blocks may then be reclaimed for storing new data.

Abstract translation: 日志结构的数据存储可以实现高效的垃圾收集。 日志记录可以根据日志记录顺序维护在数据块中。 至少部分地基于日志回收点，可以评估日志记录以识别在日志回收点之前的日志序列中具有日志记录的数据块。 可以生成在识别的数据块中的由日志记录更新的新版本的数据页，并存储在日志结构化数据存储的基页存储中。 然后可以回收所识别的数据块以存储新数据。

公开(公告)号：US20150261610A1

公开(公告)日：2015-09-17

申请号：US14727644

申请日：2015-06-01

Applicant: Amazon Technologies, Inc.

Inventor： ANURAG WINDLASS GUPTA ,  JAMES MCCLELLAN COREY ,  YAN VALERIE LESHINSKY ,  PRADEEP JNANA MADHAVARAPU ,  SAMUEL JAMES MCKELVIE

IPC: G06F11/10 ,  G06F3/06

CPC classification number: G06F11/0793 ,  G06F3/0619 ,  G06F3/064 ,  G06F3/067 ,  G06F3/0689 ,  G06F11/0709 ,  G06F11/0751 ,  G06F11/0787 ,  G06F11/079 ,  G06F11/10 ,  G06F11/1004 ,  G06F11/1076

Abstract: Self-describing data blocks of a minimum atomic write size may be stored for a data store. Data may be received for storage in a data block of a plurality of data blocks at a persistent storage device that are equivalent to a minimum atomic write size for the persistent storage device. Metadata may be generated for the data that includes an error detection code which is generated for the data and the metadata together. The data and the metadata are sent to the persistent storage device to store together in the data block. An individual atomic write operation may write together the data and the metadata in the data block. When accessed, the error detection code is applicable to detect errors. The metadata may also be applicable to determine whether the data is stored for a currently assigned purpose or a previously assigned purpose of the data block.

Abstract translation: 可以为数据存储器存储最小原子写入大小的自描述数据块。 可以接收数据以存储在永久存储设备处的多个数据块的数据块中，其等同于持久存储设备的最小原子写入大小。 可以为包括为数据和元数据生成的错误检测码的数据生成元数据。 将数据和元数据发送到持久存储设备，以一起存储在数据块中。 单独的原子写入操作可以将数据和元数据一起写入数据块中。 访问时，错误检测码适用于检测错误。 元数据还可以适用于确定数据是否存储在当前分配的目的或数据块的先前分配的目的上。

公开(公告)号：US20150268890A1

公开(公告)日：2015-09-24

申请号：US14733851

申请日：2015-06-08

Applicant: AMAZON TECHNOLOGIES, INC.

Inventor： STEFANO STEFANI ,  TIMOTHY ANDREW RATH ,  CHIRANJEEB BURAGAHAIN ,  YAN VALERIE LESHINSKY ,  DAVID ALAN LUTZ ,  JAKUB KULESZA ,  WEI XIAO ,  JAI VASANTH

IPC: G06F3/06 ,  G06F17/30

Abstract: A system that implements a scalable data storage service may maintain tables in a data store on behalf of storage service clients. The service may maintain table data in multiple replicas of partitions that are stored on respective computing nodes in the system. In response to detecting an anomaly in the system, detecting a change in data volume on a partition or service request traffic directed to a partition, or receiving a service request from a client to split a partition, the data storage service may create additional copies of a partition replica using a physical copy mechanism. The data storage service may issue a split command defined in an API for the data store to divide the original and additional replicas into multiple replica groups, and to configure each replica group to maintain a respective portion of the table data that was stored in the partition before the split.

Abstract translation: 实现可扩展数据存储服务的系统可以代表存储服务客户端来维护数据存储中的表。 该服务可以在存储在系统中的相应计算节点上的分区的多个副本中维护表数据。 响应于检测到系统中的异常，检测指向分区的分区或服务请求流量上的数据量的变化，或者从客户端接收分割分区的服务请求，数据存储服务可以创建附加副本 使用物理复制机制的分区副本。 数据存储服务可以发布在数据存储的API中定义的分割命令，以将原始和附加副本划分为多个副本组，并配置每个副本组以维护存储在分区中的表数据的相应部分 分裂之前。

公开(公告)号：US20180107704A1

公开(公告)日：2018-04-19

申请号：US15845841

申请日：2017-12-18

Applicant: Amazon Technologies, Inc.

Inventor： PRADEEP JNANA MADHAVARAPU ,  YAN VALERIE LESHINSKY

IPC: G06F17/30

CPC classification number: G06F16/2365 ,  G06F16/2379

Abstract: A data store maintaining data may implement reducing input/output (I/O) operations for on-demand data page generation. Log records may be maintained for data pages of data describing changes to the data pages. A coalesce operation may be performed when log records for a data page exceed a coalesce threshold for the data page, applying the log records for the data page to a version of the data page and creating a new version that includes the changes indicated by the log records. An indication may be received to increase the coalesce threshold for a particular data page, delaying to a coalesce operation for the data page according to the increased coalesce threshold. The indication may be received from a storage engine that identifies a delay for the particular data page.

公开(公告)号：US20160110408A1

公开(公告)日：2016-04-21

申请号：US14981540

申请日：2015-12-28

Applicant: Amazon Technologies, Inc.

Inventor： PRADEEP JNANA MADHAVARAPU ,  RAMAN MITTAL ,  BENJAMIN TOBLER ,  JAMES MCCLELLAN COREY ,  YAN VALERIE LESHINSKY ,  SAMUEL JAMES MCKELVIE ,  ANURAG WINDLASS GUPTA

IPC: G06F17/30

CPC classification number: G06F16/2365 ,  G06F16/2358 ,  G06F16/273

Abstract: A log-structured data store may implement optimized log storage for asynchronous log updates. In some embodiments, log records may be received indicating updates to data stored for a storage client and indicating positions in a log record sequence. The log records themselves may not be guaranteed to be received according to the log record sequence. Received log records may be stored in a hot log portion of a block-based storage device according to an order in which they are received. Log records in the hot log portion may then be identified to be moved to a cold log portion of the block-based storage device in order to complete a next portion of the log record sequence. Log records may be modified, such as compressed, or coalesced, before being stored together in a data block of the cold log portion according to the log record sequence.

Abstract translation: 日志结构化数据存储可以实现用于异步日志更新的优化日志存储。 在一些实施例中，可以接收指示对存储用于存储客户端的数据的更新并指示日志记录序列中的位置的日志记录。 可能无法保证根据日志记录顺序接收日志记录。 接收到的日志记录可以根据它们被接收的顺序存储在基于块的存储设备的热日志部分中。 然后可以将热日志部分中的日志记录识别为移动到基于块的存储设备的冷日志部分，以便完成日志记录序列的下一部分。 可以根据日志记录序列将日志记录修改，例如压缩或合并，然后一起存储在冷日志部分的数据块中。

公开(公告)号：US20160077930A1

公开(公告)日：2016-03-17

申请号：US14949644

申请日：2015-11-23

Applicant: Amazon Technologies, Inc.

Inventor： SAMUEL JAMES MCKELVIE ,  YAN VALERIE LESHINSKY ,  IAN P. NOWLAND ,  DARIN LEE FRINK ,  ANURAG WINDLASS GUPTA ,  ADAM DOUGLAS MORLEY ,  CHRISTOPHER NATHAN WATSON ,  MICHAEL DAVID MARR

IPC: G06F11/14 ,  G06F3/06

CPC classification number: G06F11/1469 ,  G06F3/0619 ,  G06F3/065 ,  G06F3/0688 ,  G06F11/14 ,  G06F11/1402 ,  G06F11/1441 ,  G06F11/1451 ,  G06F12/0246 ,  G06F2201/84

Abstract: Application program data stored in system memory may be selectively persisted. An indication may be provided to an application program that an application data object or a range of application data stored in system memory may be treated as persistent. Data backup may be enabled for the application data object or range of application data in the event of a system failure, copying the application data object or range of application data from system memory to non-volatile data storage. Upon recovery from a system failure, further data backup for the application data object or the range of application data may be disabled. In some embodiments, at least some of the application data object or range of application data may be recovered for the application program to access. Data backup for the application data object or the range of application data may also be re-enabled.

Abstract translation: 可以选择性地保持存储在系统存储器中的应用程序数据。 可以向应用程序提供可以将存储在系统存储器中的应用数据对象或应用数据的范围视为持久性的指示。 在系统故障的情况下，应用程序数据对象或应用程序数据范围可能启用数据备份，将应用程序数据对象或应用程序数据范围从系统内存复制到非易失性数据存储。 从系统故障恢复后，可能会禁用应用程序数据对象的进一步数据备份或应用程序数据范围。 在一些实施例中，可以为应用程序访问应用数据对象或应用数据范围中的至少一些。 还可以重新启用应用程序数据对象的数据备份或应用程序数据的范围。

公开(公告)号：US20140325116A1

公开(公告)日：2014-10-30

申请号：US13872530

申请日：2013-04-29

Applicant: Amazon Technologies, Inc.

Inventor： SAMUEL JAMES MCKELVIE ,  YAN VALERIE LESHINSKY ,  IAN P. NOWLAND ,  DARIN LEE FRINK ,  ANURAG WINDLASS GUPTA ,  ADAM DOUGLAS MORLEY ,  CHRISTOPHER NATHAN WATSON ,  MICHAEL DAVID MARR

IPC: G06F12/02

CPC classification number: G06F11/1469 ,  G06F3/0619 ,  G06F3/065 ,  G06F3/0688 ,  G06F11/14 ,  G06F11/1402 ,  G06F11/1441 ,  G06F11/1451 ,  G06F12/0246 ,  G06F2201/84

Abstract: Application program data stored in system memory may be selectively persisted. An indication may be provided to an application program that an application data object or a range of application data stored in system memory may be treated as persistent. Data backup may be enabled for the application data object or range of application data in the event of a system failure, copying the application data object or range of application data from system memory to non-volatile data storage. Upon recovery from a system failure, further data backup for the application data object or the range of application data may be disabled. In some embodiments, at least some of the application data object or range of application data may be recovered for the application program to access. Data backup for the application data object or the range of application data may also be re-enabled.

Abstract translation: 可以选择性地保持存储在系统存储器中的应用程序数据。 可以向应用程序提供可以将存储在系统存储器中的应用数据对象或应用数据的范围视为持久性的指示。 在系统故障的情况下，应用程序数据对象或应用程序数据范围可能启用数据备份，将应用程序数据对象或应用程序数据范围从系统内存复制到非易失性数据存储。 从系统故障恢复后，可能会禁用应用程序数据对象的进一步数据备份或应用程序数据范围。 在一些实施例中，可以为应用程序访问应用数据对象或应用数据范围中的至少一些。 还可以重新启用应用程序数据对象的数据备份或应用程序数据的范围。

公开(公告)号：US20140324785A1

公开(公告)日：2014-10-30

申请号：US13873467

申请日：2013-04-30

Applicant: Amazon Technologies, Inc.

Inventor： ANURAG WINDLASS GUPTA ,  PRADEEP JNANA MADHAVARAPU ,  SAMUEL JAMES MCKELVIE ,  YAN VALERIE LESHINSKY

IPC: G06F17/30

Abstract: A database system may receive a write request that specifies a modification to be made to a particular data record stored by the database system. A log record representing the modification to be made to the particular data record may be sent to a storage service of the database system. An indication (e.g., log record or other indication) that indicates a cached version of the particular data record stored in a read replica's cache is stale may be sent to a read replica. For a subsequent read of the particular data record received by the read replica, the read replica may request the particular data record from the storage service.

Abstract translation: 数据库系统可以接收指定对数据库系统存储的特定数据记录进行修改的写入请求。 表示对特定数据记录进行修改的日志记录可以被发送到数据库系统的存储服务。 指示存储在读取副本的高速缓存中的特定数据记录的缓存版本的指示（例如，日志记录或其他指示）可以被发送到读取副本。 为了对由读取的副本接收的特定数据记录的后续读取，读取的副本可以从存储服务请求特定的数据记录。