-
公开(公告)号:US10235402B1
公开(公告)日:2019-03-19
申请号:US14973712
申请日:2015-12-17
Applicant: Amazon Technologies, Inc.
IPC: G06F17/30
Abstract: One or more grids of redundancy coded shards, such as those stored or otherwise represented on grid encoded storage systems, are combinable or extensible. For example, a generator matrix of a redundancy code may be configured so as to have a sufficient number of fields to generate a grid. The generator matrix may initially be used to generate smaller grids, which can be combined into the target grid without re-encoding most or all of the data represented thereon. In some cases, vertically derived shards of the input grids may be combined using, e.g., matrix addition, which may then be directly allocated to the target grid, while data shards and horizontally derived shards may be allocated to the target grid with no further transformation.
-
公开(公告)号:US10678664B1
公开(公告)日:2020-06-09
申请号:US15083145
申请日:2016-03-28
Applicant: Amazon Technologies, Inc.
Inventor: Colin Laird Lazier , James Caleb Kirschner , Adam Frederick Brock
Abstract: A cluster of data transfer devices is used to augment the capabilities of a data storage system. For example, the cluster of data transfer devices may be configured to store a portion of a bundle of redundancy coded shards in a similar fashion as a data storage system. As another example, the cluster may be configured to provide other capabilities incident to the devices used, such as computational capabilities. Data stored on the cluster may be read from and written directly to the cluster without transfer of data to the data storage system. In some embodiments, a connecting entity (such as a customer entity) may interchangeably interface with the data storage system and the cluster, and the requested capabilities may be directed to either in a fashion that is transparent to the requestor.
-
公开(公告)号:US10061668B1
公开(公告)日:2018-08-28
申请号:US15083115
申请日:2016-03-28
Applicant: Amazon Technologies, Inc.
Inventor: Colin Laird Lazier , Adam Frederick Brock
CPC classification number: G06F11/2094 , G06F11/1076
Abstract: A data transfer device is used to augment the capabilities of a data storage system. The data transfer device may be capable of persistently storing data for an indeterminate amount of time, and may be configured to store a portion of a bundle of redundancy coded shards that span between the data transfer device and a data storage system configured to store the remainder of the bundle. Data stored on the data transfer device may be read from and written directly to the data transfer device without transfer of data to the data storage system. If the data transfer device is not available, the remaining shards of the bundle may provide a regenerated, original form of the data.
-
公开(公告)号:US11137980B1
公开(公告)日:2021-10-05
申请号:US15277922
申请日:2016-09-27
Applicant: Amazon Technologies, Inc.
Inventor: Rishabh Animesh , Adam Frederick Brock , Umar Farooq , James Caleb Kirschner
Abstract: A data storage system implements techniques for efficient retrieval of data stored thereon, using time of upload or another monotonically increasing variable as a key or identifier for the data to be stored and/or retrieved. Data is sorted according to, e.g., upload time, and the data is addressed with respect to time of upload and byte offset within the archive.
-
公开(公告)号:US09853662B1
公开(公告)日:2017-12-26
申请号:US14742684
申请日:2015-06-17
Applicant: Amazon Technologies, Inc.
Inventor: Colin Laird Lazier , Adam Frederick Brock , Paul David Franklin
CPC classification number: H03M13/2942 , G06F11/1076 , H04L63/0428 , H04L67/1097
Abstract: Techniques described and suggested herein include systems and methods for optimizing random access characteristics for data archives stored on data storage systems using redundancy coding techniques. For example, redundancy coded shards, which may include identity shards that contain unencoded original data of archives, may be configured such that a variable number of the shards can be leveraged to meet random access requirements for retrieval requests associated with the archives stored and/or encoded therein. Implementing systems may monitor random access rates, capabilities, and burdens, so as to adaptively account for changes to some or all of the monitored parameters.
-
公开(公告)号:US11113161B2
公开(公告)日:2021-09-07
申请号:US16114033
申请日:2018-08-27
Applicant: Amazon Technologies, Inc.
Inventor: Colin Laird Lazier , Adam Frederick Brock
Abstract: A data transfer device is used to augment the capabilities of a data storage system. The data transfer device may be capable of persistently storing data for an indeterminate amount of time, and may be configured to store a portion of a bundle of redundancy coded shards that span between the data transfer device and a data storage system configured to store the remainder of the bundle. Data stored on the data transfer device may be read from and written directly to the data transfer device without transfer of data to the data storage system. If the data transfer device is not available, the remaining shards of the bundle may provide a regenerated, original form of the data.
-
公开(公告)号:US10366062B1
公开(公告)日:2019-07-30
申请号:US15083125
申请日:2016-03-28
Applicant: Amazon Technologies, Inc.
Inventor: Colin Laird Lazier , Adam Frederick Brock , James Caleb Kirschner
Abstract: A cluster of data transfer devices is used to augment the capabilities of a data storage system. For example, the cluster of data transfer devices may be configured to store a portion of a bundle of redundancy coded shards in a similar fashion as a data storage system. As another example, the cluster may be configured to provide other capabilities incident to the devices used, such as computational capabilities. Data stored on the cluster may be read from and written directly to the cluster without transfer of data to the data storage system. In some embodiments, a connecting entity (such as a customer entity) may interchangeably interface with the data storage system and the cluster, and the requested capabilities may be directed to either in a fashion that is transparent to the requestor.
-
公开(公告)号:US20180365119A1
公开(公告)日:2018-12-20
申请号:US16114033
申请日:2018-08-27
Applicant: Amazon Technologies, Inc.
Inventor: Colin Laird Lazier , Adam Frederick Brock
Abstract: A data transfer device is used to augment the capabilities of a data storage system. The data transfer device may be capable of persistently storing data for an indeterminate amount of time, and may be configured to store a portion of a bundle of redundancy coded shards that span between the data transfer device and a data storage system configured to store the remainder of the bundle. Data stored on the data transfer device may be read from and written directly to the data transfer device without transfer of data to the data storage system. If the data transfer device is not available, the remaining shards of the bundle may provide a regenerated, original form of the data.
-
公开(公告)号:US09998539B1
公开(公告)日:2018-06-12
申请号:US14789815
申请日:2015-07-01
Applicant: Amazon Technologies, Inc.
Inventor: Adam Frederick Brock , Bryan James Donlan , James Caleb Kirschner
IPC: G06F15/167 , H04L29/08
CPC classification number: H04L67/1097 , G06F11/08
Abstract: Techniques for encoding data storage systems using grid-encoded data storage systems with non-parity linear redundancy encoding schemes are described herein. A grid of shards with derived shards and data shards is generated that is indexed by a first index and a second index and is configured so that each shard is reproducible from other shards with the same first index and is also reproducible from other shards with the same second index. The grid of shards is further configured so that each data row of the grid of shards has at least two derived shards and at least twice as many data shards as derived shards.
-
-
-
-
-
-
-
-