公开(公告)号：US20150149402A1

公开(公告)日：2015-05-28

申请号：US14611939

申请日：2015-02-02

Applicant: Amazon Technologies, Inc.

Inventor： ANURAG WINDLASS GUPTA

IPC: G06F17/30

CPC classification number: G06F17/30563 ,  G06F17/30315 ,  G06F17/30324 ,  G06F17/30424 ,  G06F17/30486 ,  G06F17/30584 ,  G06F17/30592

Abstract: A probabilistic data structure is generated for efficient query processing using a histogram for unsorted data in a column of a columnar database. A bucket range size is determined for multiples buckets of a histogram of a column in a columnar database table. In at least some embodiments, the histogram may be a height-balanced histogram. A probabilistic data structure is generated to indicate for which particular buckets in the histogram there is a data value stored in the data block. When an indication of a query directed to the column for select data is received, the probabilistic data structure for each of the data blocks storing data for the column may be examined to determine particular ones of the data blocks which do not need to be read in order to service the query for the select data.

Abstract translation: 生成概率数据结构，用于使用柱状数据库列中未排序数据的直方图进行有效的查询处理。 对列数据库表中的列的直方图的倍数桶确定桶范围大小。 在至少一些实施例中，直方图可以是高度平衡直方图。 生成概率数据结构以指示直方图中的哪个特定桶存在数据块中的数据值。 当接收到针对选择数据的列的查询的指示时，可以检查存储列的数据的每个数据块的概率数据结构，以确定不需要读取的特定数据块 命令为查询选择数据提供服务。

公开(公告)号：US20140279931A1

公开(公告)日：2014-09-18

申请号：US14201517

申请日：2014-03-07

Applicant: Amazon Technologies, Inc.

Inventor： ANURAG WINDLASS GUPTA ,  PRADEEP JNANA MADHAVARAPU ,  LAURION DARRELL BURCHALL ,  NEAL FACHAN

IPC: G06F11/14

CPC classification number: G06F17/30368 ,  G06F11/1474 ,  G06F17/30575 ,  G06F2201/80 ,  G06F2201/82

Abstract: A database system may maintain a plurality of log records at a distributed storage system. Each of the plurality of log records may be associated with a respective change to a data page. Upon detection of a coalesce event for a particular data page, log records linked to the particular data page may be applied to generate the particular data page in its current state. Detecting the coalesce event may be a determination that the number of log records linked to the particular data page exceeds a threshold.

Abstract translation: 数据库系统可以在分布式存储系统上维护多个日志记录。 多个日志记录中的每一个可以与对数据页的相应改变相关联。 在检测到特定数据页的聚结事件时，可以应用链接到特定数据页的日志记录来生成其当前状态的特定数据页。 检测聚结事件可以是确定链接到特定数据页的日志记录的数量超过阈值。

公开(公告)号：US20140149357A1

公开(公告)日：2014-05-29

申请号：US13792914

申请日：2013-03-11

Applicant: Amazon Technologies, Inc.

Inventor： ANURAG WINDLASS GUPTA

IPC: G06F17/30

CPC classification number: G06F17/30575 ,  G06F11/1446 ,  G06F11/1471 ,  G06F17/30008 ,  G06F17/30371 ,  G06F17/30424 ,  G06F2201/82

Abstract: A distributed data warehouse system may maintain data blocks on behalf of clients, and may store primary and secondary copies of each data block on different disks or nodes in a cluster. The warehouse system may back up data blocks in a remote key-value backup storage system. A restore operation may retrieve data blocks from backup storage using their unique identifiers as keys (while incoming queries are serviced) in response to a failure or a query targeting data that was lost or corrupted. The order in which data blocks are restored may be dependent on the relative likelihood that they will be accessed in the near future (e.g., based on how recently or frequently they were accessed, written, or backed up; the values of one or more access counters associated with each data block; or how recently a database table containing data in each data block was loaded).

Abstract translation: 分布式数据仓库系统可以代表客户端维护数据块，并且可以将每个数据块的主副本存储在集群中的不同磁盘或节点上。 仓库系统可以备份远程键值备份存储系统中的数据块。 恢复操作可以使用其唯一标识符作为密钥（在传入查询被服务时）从备份存储中检索数据块，以响应失败或针对丢失或损坏的数据的查询。 数据块恢复的顺序可能取决于它们在不久的将来将被访问的相对可能性（例如，基于最近或经常被访问，写入或备份）;一个或多个访问的值 与每个数据块关联的计数器;或者最近加载了包含每个数据块中的数据的数据库表）。

公开(公告)号：US20180150528A1

公开(公告)日：2018-05-31

申请号：US15385764

申请日：2016-12-20

Applicant: Amazon Technologies, Inc.

Inventor： MEHUL A. SHAH ,  GEORGE STEVEN MCPHERSON ,  PRAJAKTA DATTA DAMLE ,  GOPINATH DUDDI ,  ANURAG WINDLASS GUPTA ,  BENJAMIN ALBERT SOWELL ,  BOHOU LI

IPC: G06F17/30

CPC classification number: G06F16/254 ,  G06F16/282

Abstract: Data transformation workflows may be generated to transform data objects. A source data schema for a data object and a target data format or target data schema for a data object may be identified. A comparison of the source data schema and the target data format or schema may be made to determine what transformations can be performed to transform the data object into the target data format or schema. Code to execute the transformation operations may then be generated. The code may be stored for subsequent modification or execution.

公开(公告)号：US20170109245A1

公开(公告)日：2017-04-20

申请号：US15391729

申请日：2016-12-27

Applicant: Amazon Technologies, Inc.

Inventor： GRANT ALEXANDER MACDONALD MCALISTER ,  SAMUEL JAMES MCKELVIE ,  ANURAG WINDLASS GUPTA

IPC: G06F11/20 ,  H04L12/24 ,  G06Q40/04 ,  G06F11/14 ,  G06F17/30

CPC classification number: G06F11/203 ,  G06F11/1435 ,  G06F11/1471 ,  G06F11/2094 ,  G06F11/2097 ,  G06F16/27 ,  G06F2201/80 ,  G06F2201/805 ,  G06F2201/815 ,  G06Q40/04 ,  H04L41/0668 ,  H04L41/069 ,  H04L67/02

Abstract: A network-based services provider may reserve and provision primary resource instance capacity for a given service (e.g., enough compute instances, storage instances, or other virtual resource instances to implement the service) in one or more availability zones, and may designate contingency resource instance capacity for the service in another availability zone (without provisioning or reserving the contingency instances for the exclusive use of the service). For example, the service provider may provision resource instance(s) for a database engine head node in one availability zone and designate resource instance capacity for another database engine head node in another availability zone without instantiating the other database engine head node. While the service operates as expected using the primary resource instance capacity, the contingency resource capacity may be leased to other entities on a spot market. Leases for contingency instance capacity may be revoked when needed for the given service (e.g., during failover).

公开(公告)号：US20160314140A1

公开(公告)日：2016-10-27

申请号：US15199627

申请日：2016-06-30

Applicant: Amazon Technologies, Inc.

Inventor： ANURAG WINDLASS GUPTA

IPC: G06F17/30 ,  H03M7/30 ,  G06N99/00

CPC classification number: G06F17/30153 ,  G06N99/005 ,  H03M7/30 ,  H03M7/3079 ,  H03M7/6088

Abstract: Data may be efficiently analyzed and compressed as part of a data compression service. A data compression request may be received from a client indicating data to be compressed. An analysis of the data or metadata associated with the data may be performed. In at least some embodiments, this analysis may be a rules-based analysis. Some embodiments may employ one or more machine learning techniques to historical compression data to update the rules-based analysis. One or more compression techniques may be selected out of a plurality of compression techniques to be applied to the data. Data compression candidates may then be generated according to the selected compression techniques. In some embodiments, a compression service restriction may be enforced. One of the data compression candidates may be selected and sent in a response.

Abstract translation: 数据可以作为数据压缩服务的一部分被有效地分析和压缩。 可以从客户端接收指示要压缩的数据的数据压缩请求。 可以执行与数据相关联的数据或元数据的分析。 在至少一些实施例中，该分析可以是基于规则的分析。 一些实施例可以对历史压缩数据采用一种或多种机器学习技术来更新基于规则的分析。 可以从应用于数据的多种压缩技术中选择一种或多种压缩技术。 然后可以根据选择的压缩技术生成数据压缩候选。 在一些实施例中，可以强制执行压缩服务限制。 可以在响应中选择并发送其中一个数据压缩候选。

公开(公告)号：US20150032684A1

公开(公告)日：2015-01-29

申请号：US13953432

申请日：2013-07-29

Applicant: Amazon Technologies, Inc.

Inventor： ANURAG WINDLASS GUPTA

IPC: G06F17/30

CPC classification number: G06F16/283 ,  G06F16/22 ,  G06F16/221 ,  G06F16/2228 ,  G06F16/27

Abstract: A multi-column index is generated based on an interleaving of data bits for selectivity for efficient processing of data in a relational database system. Two or more columns may be identified for inclusion in the multi-column index for a relational database table. Based, at least in part, on the interleaving of data bits for selectivity from the identified columns, a multi-column index is generated for the relational database table that provides a respective index value for each entry in the relational database table. The entries of the relational database table may then be stored according to the index values of the multi-column index.

Abstract translation: 基于用于在关系数据库系统中的数据的有效处理的选择性的数据位的交织来生成多列索引。 可以识别两个或更多个列以包含在用于关系数据库表的多列索引中。 基于至少部分地基于从所识别的列的选择性的数据位的交织，为关系数据库表生成多列索引，该关系数据库表为关系数据库表中的每个条目提供相应的索引值。 然后可以根据多列索引的索引值来存储关系数据库表的条目。

公开(公告)号：US20160110408A1

公开(公告)日：2016-04-21

申请号：US14981540

申请日：2015-12-28

Applicant: Amazon Technologies, Inc.

Inventor： PRADEEP JNANA MADHAVARAPU ,  RAMAN MITTAL ,  BENJAMIN TOBLER ,  JAMES MCCLELLAN COREY ,  YAN VALERIE LESHINSKY ,  SAMUEL JAMES MCKELVIE ,  ANURAG WINDLASS GUPTA

IPC: G06F17/30

CPC classification number: G06F16/2365 ,  G06F16/2358 ,  G06F16/273

Abstract: A log-structured data store may implement optimized log storage for asynchronous log updates. In some embodiments, log records may be received indicating updates to data stored for a storage client and indicating positions in a log record sequence. The log records themselves may not be guaranteed to be received according to the log record sequence. Received log records may be stored in a hot log portion of a block-based storage device according to an order in which they are received. Log records in the hot log portion may then be identified to be moved to a cold log portion of the block-based storage device in order to complete a next portion of the log record sequence. Log records may be modified, such as compressed, or coalesced, before being stored together in a data block of the cold log portion according to the log record sequence.

Abstract translation: 日志结构化数据存储可以实现用于异步日志更新的优化日志存储。 在一些实施例中，可以接收指示对存储用于存储客户端的数据的更新并指示日志记录序列中的位置的日志记录。 可能无法保证根据日志记录顺序接收日志记录。 接收到的日志记录可以根据它们被接收的顺序存储在基于块的存储设备的热日志部分中。 然后可以将热日志部分中的日志记录识别为移动到基于块的存储设备的冷日志部分，以便完成日志记录序列的下一部分。 可以根据日志记录序列将日志记录修改，例如压缩或合并，然后一起存储在冷日志部分的数据块中。

公开(公告)号：US20160085643A1

公开(公告)日：2016-03-24

申请号：US14961775

申请日：2015-12-07

Applicant: Amazon Technologies, Inc.

Inventor： GRANT ALEXANDER MACDONALD MCALISTER ,  SAMUEL JAMES MCKELVIE ,  ANURAG WINDLASS GUPTA

IPC: G06F11/20 ,  H04L12/24

CPC classification number: G06F11/203 ,  G06F11/1435 ,  G06F11/1471 ,  G06F11/2094 ,  G06F11/2097 ,  G06F17/30283 ,  G06F2201/80 ,  G06F2201/805 ,  G06F2201/815 ,  G06Q40/04 ,  H04L41/0668 ,  H04L41/069 ,  H04L67/02

Abstract: A network-based services provider may reserve and provision primary resource instance capacity for a given service (e.g., enough compute instances, storage instances, or other virtual resource instances to implement the service) in one or more availability zones, and may designate contingency resource instance capacity for the service in another availability zone (without provisioning or reserving the contingency instances for the exclusive use of the service). For example, the service provider may provision resource instance(s) for a database engine head node in one availability zone and designate resource instance capacity for another database engine head node in another availability zone without instantiating the other database engine head node. While the service operates as expected using the primary resource instance capacity, the contingency resource capacity may be leased to other entities on a spot market. Leases for contingency instance capacity may be revoked when needed for the given service (e.g., during failover).

Abstract translation: 基于网络的服务提供商可以在一个或多个可用性区域中为给定服务（例如，足够的计算实例，存储实例或其他虚拟资源实例）预留和提供主资源实例容量，并且可以指定应急资源 另一个可用区域中的服务的实例容量（不提供或预留用于独占使用服务的应急实例）。 例如，服务提供商可以在一个可用性区域中为数据库引擎头节点提供资源实例，并且在另一个可用区域内为另一个数据库引擎头节点指定资源实例容量，而不实例化其他数据库引擎头节点。 虽然服务使用主要资源实例容量预期运行，但应急资源能力可以在现货市场上租赁给其他实体。 对于特定服务（例如，在故障转移期间）需要时，可能会撤销应急实例容量的租约。

公开(公告)号：US20160077930A1

公开(公告)日：2016-03-17

申请号：US14949644

申请日：2015-11-23

Applicant: Amazon Technologies, Inc.

Inventor： SAMUEL JAMES MCKELVIE ,  YAN VALERIE LESHINSKY ,  IAN P. NOWLAND ,  DARIN LEE FRINK ,  ANURAG WINDLASS GUPTA ,  ADAM DOUGLAS MORLEY ,  CHRISTOPHER NATHAN WATSON ,  MICHAEL DAVID MARR

IPC: G06F11/14 ,  G06F3/06

CPC classification number: G06F11/1469 ,  G06F3/0619 ,  G06F3/065 ,  G06F3/0688 ,  G06F11/14 ,  G06F11/1402 ,  G06F11/1441 ,  G06F11/1451 ,  G06F12/0246 ,  G06F2201/84

Abstract: Application program data stored in system memory may be selectively persisted. An indication may be provided to an application program that an application data object or a range of application data stored in system memory may be treated as persistent. Data backup may be enabled for the application data object or range of application data in the event of a system failure, copying the application data object or range of application data from system memory to non-volatile data storage. Upon recovery from a system failure, further data backup for the application data object or the range of application data may be disabled. In some embodiments, at least some of the application data object or range of application data may be recovered for the application program to access. Data backup for the application data object or the range of application data may also be re-enabled.

Abstract translation: 可以选择性地保持存储在系统存储器中的应用程序数据。 可以向应用程序提供可以将存储在系统存储器中的应用数据对象或应用数据的范围视为持久性的指示。 在系统故障的情况下，应用程序数据对象或应用程序数据范围可能启用数据备份，将应用程序数据对象或应用程序数据范围从系统内存复制到非易失性数据存储。 从系统故障恢复后，可能会禁用应用程序数据对象的进一步数据备份或应用程序数据范围。 在一些实施例中，可以为应用程序访问应用数据对象或应用数据范围中的至少一些。 还可以重新启用应用程序数据对象的数据备份或应用程序数据的范围。