-
公开(公告)号:US11475038B2
公开(公告)日:2022-10-18
申请号:US15893496
申请日:2018-02-09
Applicant: Amazon Technologies, Inc.
Inventor: Deepak Agarwal , Anurag Windlass Gupta , Jakub Kulesza
IPC: G06F16/00 , G06F16/27 , G06F16/245 , G06F16/23 , G06F11/14
Abstract: A distributed data warehouse system maintains data blocks on behalf of clients, and stores primary and secondary copies of data blocks on different disks or nodes in a cluster. The data warehouse system may back up data blocks in a key-value backup storage system. In response to a query targeting a data block previously stored in the cluster, the data warehouse system may determine whether a consistent, uncorrupted copy of the data block is available in the cluster (e.g., by applying a consistency check). If not (e.g., if a disk or node failed), the data warehouse system may automatically initiate an operation to restore the data block from the backup storage system, using a unique identifier of the data block to access a backup copy. The target data may be returned in a query response prior to restoring primary and secondary copies of the data block in the cluster.
-
公开(公告)号:US11422982B2
公开(公告)日:2022-08-23
申请号:US16283510
申请日:2019-02-22
Applicant: Amazon Technologies, Inc.
Inventor: Jakub Kulesza , Srividhya Srinivasan , Deepak Agarwal , Anurag Windlass Gupta
IPC: G06F16/21 , G06F16/182 , G06F9/50
Abstract: A stateful cluster may implement scaling of the stateful cluster while maintaining access to the state of the stateful cluster. A scaling event for a stateful cluster may be detected, and in response the stateful cluster may be adjusted to include a different number of nodes. The state of the cluster may then be logically distributed among the different number of nodes according to a monotone distribution scheme. The adjusted node may then service access requests according to the monotone distribution scheme. Prior to making the adjusted storage cluster available for servicing access requests, the nodes from the original cluster may still service access requests for state.
-
公开(公告)号:US10061834B1
公开(公告)日:2018-08-28
申请号:US14530495
申请日:2014-10-31
Applicant: Amazon Technologies, Inc.
Inventor: Jakub Kulesza , Bharath Kumar Chelepalli , Deepak Agarwal , Anurag Windlass Gupta
IPC: G06F17/30
CPC classification number: G06F16/283 , G06F16/27
Abstract: A data store may implement incremental out-of-place updates to a dataset. A dataset may maintain data across different storage locations linked together according to an ordering schema for servicing queries. As updates to the dataset are received, the updates may be persisted but not maintained in-place. In order to update the data store and maintain the ordering schema, incremental updates to the dataset may be performed without blocking queries directed toward the dataset. The dataset may be divided into multiple data chunks that correspond to different storage locations and an updated version of the data chunk may be generated in new storage locations. The new storage locations may then replace the storage locations of the prior version of the data chunk in order to link the new storage locations to the other linked storage locations in the dataset for servicing queries.
-
公开(公告)号:US10216770B1
公开(公告)日:2019-02-26
申请号:US14530480
申请日:2014-10-31
Applicant: Amazon Technologies, Inc.
Inventor: Jakub Kulesza , Srividhya Srinivasan , Deepak Agarwal , Anurag Windlass Gupta
IPC: G06F17/30
Abstract: A stateful cluster may implement scaling of the stateful cluster while maintaining access to the state of the stateful cluster. A scaling event for a stateful cluster may be detected, and in response the stateful cluster may be adjusted to include a different number of nodes. The state of the cluster may then be logically distributed among the different number of nodes according to a monotone distribution scheme. The adjusted node may then service access requests according to the monotone distribution scheme. Prior to making the adjusted storage cluster available for servicing access requests, the nodes from the original cluster may still service access requests for state.
-
公开(公告)号:US09892182B2
公开(公告)日:2018-02-13
申请号:US15269511
申请日:2016-09-19
Applicant: Amazon Technologies, Inc.
Inventor: Deepak Agarwal , Anurag Windlass Gupta , Jakub Kulesza
CPC classification number: G06F17/30575 , G06F11/1446 , G06F11/1471 , G06F17/30008 , G06F17/30371 , G06F17/30424 , G06F2201/82
Abstract: A distributed data warehouse system maintains data blocks on behalf of clients, and stores primary and secondary copies of data blocks on different disks or nodes in a cluster. The data warehouse system may back up data blocks in a key-value backup storage system. In response to a query targeting a data block previously stored in the cluster, the data warehouse system may determine whether a consistent, uncorrupted copy of the data block is available in the cluster (e.g., by applying a consistency check). If not (e.g., if a disk or node failed), the data warehouse system may automatically initiate an operation to restore the data block from the backup storage system, using a unique identifier of the data block to access a backup copy. The target data may be returned in a query response prior to restoring primary and secondary copies of the data block in the cluster.
-
公开(公告)号:US09600500B1
公开(公告)日:2017-03-21
申请号:US13923688
申请日:2013-06-21
Applicant: Amazon Technologies, Inc.
Inventor: Anurag Windlass Gupta , Jakub Kulesza , Don Johnson , Deepak Agarwal , Tushar Jain
CPC classification number: G06F17/30283 , G06F11/1464 , G06F11/1469 , G06F11/1471 , G06F17/30289 , G06F17/30348 , G06F17/30356 , G06F17/30368 , G06F17/30377 , G06F17/30581 , G06F2201/80 , G06F2201/84
Abstract: A distributed database system may perform a single phase commit for transactions involving updates to multiple databases of the distributed database system. A client request may be received that involves updates to multiple database of the distributed database system. The updates may be performed at a front-end database and a back-end database. Log records indicating the updates to the front-end database may be sent to the back-end database. The log records and the updates performed at the back-end database may be committed together as a single phase commit at the back-end database. In the event of a system failure of the front-end database, log records may be requested and received from the back-end database. A restoration of the front-end database may be performed based, at least in part, on the received log records.
-
公开(公告)号:US11068501B2
公开(公告)日:2021-07-20
申请号:US15464272
申请日:2017-03-20
Applicant: Amazon Technologies, Inc.
Inventor: Anurag Windlass Gupta , Jakub Kulesza , Don Johnson , Deepak Agarwal , Tushar Jain
Abstract: A distributed database system may perform a single phase commit for transactions involving updates to multiple databases of the distributed database system. A client request may be received that involves updates to multiple database of the distributed database system. The updates may be performed at a front-end database and a back-end database. Log records indicating the updates to the front-end database may be sent to the back-end database. The log records and the updates performed at the back-end database may be committed together as a single phase commit at the back-end database. In the event of a system failure of the front-end database, log records may be requested and received from the back-end database. A restoration of the front-end database may be performed based, at least in part, on the received log records.
-
公开(公告)号:US20180165340A1
公开(公告)日:2018-06-14
申请号:US15893496
申请日:2018-02-09
Applicant: Amazon Technologies, Inc.
Inventor: Deepak Agarwal , Anurag Windlass Gupta , Jakub Kulesza
Abstract: A distributed data warehouse system maintains data blocks on behalf of clients, and stores primary and secondary copies of data blocks on different disks or nodes in a cluster. The data warehouse system may back up data blocks in a key-value backup storage system. In response to a query targeting a data block previously stored in the cluster, the data warehouse system may determine whether a consistent, uncorrupted copy of the data block is available in the cluster (e.g., by applying a consistency check). If not (e.g., if a disk or node failed), the data warehouse system may automatically initiate an operation to restore the data block from the backup storage system, using a unique identifier of the data block to access a backup copy. The target data may be returned in a query response prior to restoring primary and secondary copies of the data block in the cluster.
-
公开(公告)号:US20170193012A1
公开(公告)日:2017-07-06
申请号:US15464272
申请日:2017-03-20
Applicant: Amazon Technologies, Inc.
Inventor: Anurag Windlass Gupta , Jakub Kulesza , Don Johnson , Deepak Agarwal , Tushar Jain
CPC classification number: G06F16/27 , G06F11/1464 , G06F11/1469 , G06F11/1471 , G06F16/21 , G06F16/2308 , G06F16/2329 , G06F16/2358 , G06F16/2379 , G06F16/275 , G06F2201/80 , G06F2201/84
Abstract: A distributed database system may perform a single phase commit for transactions involving updates to multiple databases of the distributed database system. A client request may be received that involves updates to multiple database of the distributed database system. The updates may be performed at a front-end database and a back-end database. Log records indicating the updates to the front-end database may be sent to the back-end database. The log records and the updates performed at the back-end database may be committed together as a single phase commit at the back-end database. In the event of a system failure of the front-end database, log records may be requested and received from the back-end database. A restoration of the front-end database may be performed based, at least in part, on the received log records.
-
公开(公告)号:US20190188190A1
公开(公告)日:2019-06-20
申请号:US16283510
申请日:2019-02-22
Applicant: Amazon Technologies, Inc.
Inventor: Jakub Kulesza , Srividhya Srinivasan , Deepak Agarwal , Anurag Windlass Gupta
IPC: G06F16/21 , G06F16/182
CPC classification number: G06F16/21 , G06F16/182
Abstract: A stateful cluster may implement scaling of the stateful cluster while maintaining access to the state of the stateful cluster. A scaling event for a stateful cluster may be detected, and in response the stateful cluster may be adjusted to include a different number of nodes. The state of the cluster may then be logically distributed among the different number of nodes according to a monotone distribution scheme. The adjusted node may then service access requests according to the monotone distribution scheme. Prior to making the adjusted storage cluster available for servicing access requests, the nodes from the original cluster may still service access requests for state.
-
-
-
-
-
-
-
-
-