公开(公告)号：US20140325116A1

公开(公告)日：2014-10-30

申请号：US13872530

申请日：2013-04-29

Applicant: Amazon Technologies, Inc.

Inventor： SAMUEL JAMES MCKELVIE ,  YAN VALERIE LESHINSKY ,  IAN P. NOWLAND ,  DARIN LEE FRINK ,  ANURAG WINDLASS GUPTA ,  ADAM DOUGLAS MORLEY ,  CHRISTOPHER NATHAN WATSON ,  MICHAEL DAVID MARR

IPC: G06F12/02

CPC classification number: G06F11/1469 ,  G06F3/0619 ,  G06F3/065 ,  G06F3/0688 ,  G06F11/14 ,  G06F11/1402 ,  G06F11/1441 ,  G06F11/1451 ,  G06F12/0246 ,  G06F2201/84

Abstract: Application program data stored in system memory may be selectively persisted. An indication may be provided to an application program that an application data object or a range of application data stored in system memory may be treated as persistent. Data backup may be enabled for the application data object or range of application data in the event of a system failure, copying the application data object or range of application data from system memory to non-volatile data storage. Upon recovery from a system failure, further data backup for the application data object or the range of application data may be disabled. In some embodiments, at least some of the application data object or range of application data may be recovered for the application program to access. Data backup for the application data object or the range of application data may also be re-enabled.

Abstract translation: 可以选择性地保持存储在系统存储器中的应用程序数据。 可以向应用程序提供可以将存储在系统存储器中的应用数据对象或应用数据的范围视为持久性的指示。 在系统故障的情况下，应用程序数据对象或应用程序数据范围可能启用数据备份，将应用程序数据对象或应用程序数据范围从系统内存复制到非易失性数据存储。 从系统故障恢复后，可能会禁用应用程序数据对象的进一步数据备份或应用程序数据范围。 在一些实施例中，可以为应用程序访问应用数据对象或应用数据范围中的至少一些。 还可以重新启用应用程序数据对象的数据备份或应用程序数据的范围。

公开(公告)号：US20140324785A1

公开(公告)日：2014-10-30

申请号：US13873467

申请日：2013-04-30

Applicant: Amazon Technologies, Inc.

Inventor： ANURAG WINDLASS GUPTA ,  PRADEEP JNANA MADHAVARAPU ,  SAMUEL JAMES MCKELVIE ,  YAN VALERIE LESHINSKY

IPC: G06F17/30

Abstract: A database system may receive a write request that specifies a modification to be made to a particular data record stored by the database system. A log record representing the modification to be made to the particular data record may be sent to a storage service of the database system. An indication (e.g., log record or other indication) that indicates a cached version of the particular data record stored in a read replica's cache is stale may be sent to a read replica. For a subsequent read of the particular data record received by the read replica, the read replica may request the particular data record from the storage service.

Abstract translation: 数据库系统可以接收指定对数据库系统存储的特定数据记录进行修改的写入请求。 表示对特定数据记录进行修改的日志记录可以被发送到数据库系统的存储服务。 指示存储在读取副本的高速缓存中的特定数据记录的缓存版本的指示（例如，日志记录或其他指示）可以被发送到读取副本。 为了对由读取的副本接收的特定数据记录的后续读取，读取的副本可以从存储服务请求特定的数据记录。

公开(公告)号：US20140279929A1

公开(公告)日：2014-09-18

申请号：US14201493

申请日：2014-03-07

Applicant: Amazon Technologies, Inc.

Inventor： ANURAG WINDLASS GUPTA ,  NEAL FACHAN ,  SAMUEL JAMES MCKELVIE ,  LAURION DARRELL BURCHALL ,  CHRISTOPHER RICHARD NEWCOMBE ,  PRADEEP JNANA MADHAVARAPU ,  BENJAMIN TOBLER ,  JAMES MCCLELLAN COREY

IPC: G06F11/14

CPC classification number: G06F17/30371 ,  G06F11/1451 ,  G06F11/1471 ,  G06F11/2094 ,  G06F17/30345 ,  G06F2201/80

Abstract: A database system may include a database service and a separate distributed storage service. The database service (or a database engine head node thereof) may be responsible for query parsing, optimization, and execution, transactionality, and consistency, while the storage service may be responsible for generating data pages from redo log records and for durability of those data pages. For example, in response to a write request directed to a particular data page, the database engine head node may generate a redo log record and send it, but not the data page, to a storage service node. The storage service node may store the redo log record and return a write acknowledgement to the database service prior to applying the redo log record. The server node may apply the redo log record and other redo log records to a previously stored version of the data page to create a current version.

Abstract translation: 数据库系统可以包括数据库服务和单独的分布式存储服务。 数据库服务（或其数据库引擎头节点）可以负责查询解析，优化和执行，事务性和一致性，而存储服务可能负责从重做日志记录生成数据页面以及对这些数据的耐久性 页面。 例如，响应于针对特定数据页的写请求，数据库引擎头节点可以生成重做日志记录，并将其发送到存储服务节点，而不是数据页。 存储服务节点可以存储重做日志记录，并在应用重做日志记录之前向数据库服务返回写入确认。 服务器节点可以将重做日志记录和其他重做日志记录应用于先前存储的数据页面版本，以创建当前版本。

公开(公告)号：US20140149356A1

公开(公告)日：2014-05-29

申请号：US13792671

申请日：2013-03-11

Applicant: AMAZON TECHNOLOGIES, INC.

Inventor： DEEPAK AGARWAL ,  ANURAG WINDLASS GUPTA ,  JAKUB KULESZA

IPC: G06F17/30

CPC classification number: G06F17/30575 ,  G06F11/1446 ,  G06F11/1471 ,  G06F17/30008 ,  G06F17/30371 ,  G06F17/30424 ,  G06F2201/82

Abstract: A distributed data warehouse system maintains data blocks on behalf of clients, and stores primary and secondary copies of data blocks on different disks or nodes in a cluster. The data warehouse system may back up data blocks in a key-value backup storage system. In response to a query targeting a data block previously stored in the cluster, the data warehouse system may determine whether a consistent, uncorrupted copy of the data block is available in the cluster (e.g., by applying a consistency check). If not (e.g., if a disk or node failed), the data warehouse system may automatically initiate an operation to restore the data block from the backup storage system, using a unique identifier of the data block to access a backup copy. The target data may be returned in a query response prior to restoring primary and secondary copies of the data block in the cluster.

Abstract translation: 分布式数据仓库系统代表客户端维护数据块，并将数据块的主副本存储在集群中的不同磁盘或节点上。 数据仓库系统可以备份密钥值备份存储系统中的数据块。 响应于针对先前存储在集群中的数据块的查询，数据仓库系统可以确定数据块的一致的未破坏的副本在集群中是否可用（例如，通过应用一致性检查）。 如果没有（例如，如果磁盘或节点出现故障），则数据仓库系统可以使用数据块的唯一标识符来自备份存储系统自动启动恢复数据块的操作以访问备份副本。 在恢复群集中的数据块的主副本之前，可以在查询响应中返回目标数据。

公开(公告)号：US20180150529A1

公开(公告)日：2018-05-31

申请号：US15385787

申请日：2016-12-20

Applicant: Amazon Technologies, Inc.

Inventor： GEORGE STEVEN MCPHERSON ,  MEHUL A. SHAH ,  PRAJAKTA DATTA DAMLE ,  GOPINATH DUDDI ,  ANURAG WINDLASS GUPTA

IPC: G06F17/30 ,  G06F9/54

CPC classification number: G06F16/254 ,  G06F9/542

Abstract: Extract, Transform, Load (ETL) processing may be initiated by detected events. A trigger event may be associated with an ETL process apply one or more transformations to a source data object. The trigger event may be detected for the ETL process and evaluated with respect to one or more execution conditions for the ETL process. If the execution conditions for the ETL process are satisfied, then the ETL process may be executed. At least some of the source data object may be obtained, the one or more transformations of the ETL process may be applied, and one or more transformed data objects may be stored.

公开(公告)号：US20180025065A1

公开(公告)日：2018-01-25

申请号：US15706511

申请日：2017-09-15

Applicant: Amazon Technologies, Inc.

Inventor： ANURAG WINDLASS GUPTA

IPC: G06F17/30

CPC classification number: G06F16/254 ,  G06F16/221 ,  G06F16/2237 ,  G06F16/245 ,  G06F16/24554 ,  G06F16/278 ,  G06F16/283

Abstract: A probabilistic data structure is generated for efficient query processing using a histogram for unsorted data in a column of a columnar database. A bucket range size is determined for multiples buckets of a histogram of a column in a columnar database table. In at least some embodiments, the histogram may be a height-balanced histogram. A probabilistic data structure is generated to indicate for which particular buckets in the histogram there is a data value stored in the data block. When an indication of a query directed to the column for select data is received, the probabilistic data structure for each of the data blocks storing data for the column may be examined to determine particular ones of the data blocks which do not need to be read in order to service the query for the select data.

公开(公告)号：US20170083565A1

公开(公告)日：2017-03-23

申请号：US15369681

申请日：2016-12-05

Applicant: Amazon Technologies, Inc.

Inventor： ANURAG WINDLASS GUPTA ,  NEAL FACHAN ,  SAMUEL JAMES MCKELVIE ,  LAURION DARRELL BURCHALL ,  CHRISTOPHER RICHARD NEWCOMBE ,  PRADEEP JNANA MADHAVARAPU ,  BENJAMIN TOBLER ,  JAMES MCCLELLAN COREY

IPC: G06F17/30 ,  G06F11/14

Abstract: A database system may include a database service and a separate distributed storage service. The database service (or a database engine head node thereof) may be responsible for query parsing, optimization, and execution, transactionality, and consistency, while the storage service may be responsible for generating data pages from redo log records and for durability of those data pages. For example, in response to a write request directed to a particular data page, the database engine head node may generate a redo log record and send it, but not the data page, to a storage service node. The storage service node may store the redo log record and return a write acknowledgement to the database service prior to applying the redo log record. The server node may apply the redo log record and other redo log records to a previously stored version of the data page to create a current version.

公开(公告)号：US20170031749A1

公开(公告)日：2017-02-02

申请号：US15289907

申请日：2016-10-10

Applicant: Amazon Technologies, Inc.

Inventor： ANURAG WINDLASS GUPTA ,  JAMES MCCLELLAN COREY ,  YAN VALERIE LESHINSKY ,  PRADEEP JNANA MADHAVARAPU ,  SAMUEL JAMES MCKELVIE

IPC: G06F11/07 ,  G06F3/06

CPC classification number: G06F11/0793 ,  G06F3/0619 ,  G06F3/064 ,  G06F3/067 ,  G06F3/0689 ,  G06F11/0709 ,  G06F11/0751 ,  G06F11/0787 ,  G06F11/079 ,  G06F11/10 ,  G06F11/1004 ,  G06F11/1076

Abstract: Self-describing data blocks of a minimum atomic write size may be stored for a data store. Data may be received for storage in a data block of a plurality of data blocks at a persistent storage device that are equivalent to a minimum atomic write size for the persistent storage device. Metadata may be generated for the data that includes an error detection code which is generated for the data and the metadata together. The data and the metadata are sent to the persistent storage to device to store together in the data block. An individual atomic write operation may write together the data and the metadata in the data block. When accessed, the error detection code is applicable to detect errors. The metadata may also be applicable to determine whether the data is stored for a currently assigned purpose or a previously assigned purpose of the data block.

Abstract translation: 可以为数据存储器存储最小原子写入大小的自描述数据块。 可以接收数据以存储在永久存储设备处的多个数据块的数据块中，其等同于持久存储设备的最小原子写入大小。 可以为包括为数据和元数据生成的错误检测码的数据生成元数据。 数据和元数据被发送到持久存储器到设备以一起存储在数据块中。 单独的原子写入操作可以将数据和元数据一起写入数据块中。 访问时，错误检测码适用于检测错误。 元数据还可以适用于确定数据是否存储在当前分配的目的或数据块的先前分配的目的上。

公开(公告)号：US20140337393A1

公开(公告)日：2014-11-13

申请号：US13893004

申请日：2013-05-13

Applicant: Amazon Technologies, Inc.

Inventor： LAURION DARRELL BURCHALL ,  PRADEEP JNANA MADHAVARAPU ,  CHRISTOPHER RICHARD NEWCOMBE ,  ANURAG WINDLASS GUPTA

IPC: G06F17/30

CPC classification number: G06F17/30377 ,  G06F9/466 ,  G06F17/30351 ,  G06F17/30359 ,  G06F17/30371

Abstract: Nodes of a database service may receive a read request to perform a read of a record stored by the database service and a transaction request to perform a transaction to the record. First and second indications of time may be associated with the read and transaction, respectively. A potential read anomaly (e.g., fuzzy read, read skew, etc.) may be detected based, at least in part, on a determination that the first indication of time is within a threshold value of the second indication of time. In response to detecting the potential read anomaly, the read may be performed after the transaction specified by the transaction request, regardless of whether the first indication of time is indicative of an earlier point in time than the second indication of time.

Abstract translation: 数据库服务的节点可以接收读请求，以执行由数据库服务存储的记录的读取和用于对记录执行事务的事务请求。 时间的第一和第二指示可能分别与读取和交易相关联。 至少部分地可以基于时间的第一指示在第二时间指示的阈值内的确定来检测潜在的读取异常（例如，模糊读取，读取偏移等）。 响应于检测到潜在的读取异常，可以在由事务请求指定的事务之后执行读取，而不管时间的第一指示是否指示比第二时间指示更早的时间点。

公开(公告)号：US20140279900A1

公开(公告)日：2014-09-18

申请号：US14201512

申请日：2014-03-07

Applicant: Amazon Technologies, Inc.

Inventor： ANURAG WINDLASS GUPTA ,  PRADEEP JNANA MADHAVARAPU ,  SAMUEL JAMES MCKELVIE ,  NEAL FACHAN

IPC: G06F17/30

CPC classification number: G06F17/30309

Abstract: A database system may maintain a plurality of log records at a distributed storage system. Each of the plurality of log records may be associated with a respective change to a data page. A snapshot may be generated that is usable to read the data as of a state corresponding to the snapshot. Generating the snapshot may include generating metadata that is indicative of a particular log identifier of a particular one of the log records. Generating the snapshot may be performed without additional reading, copying, or writing of the data.

Abstract translation: 数据库系统可以在分布式存储系统上维护多个日志记录。 多个日志记录中的每一个可以与对数据页的相应改变相关联。 可以生成可用于读取与快照相对应的状态的数据的快照。 生成快照可以包括生成表示特定日志记录的特定日志标识符的元数据。 可以在不附加读取，复制或写入数据的情况下执行生成快照。