Consolidating data in storage host groupings

    公开(公告)号:US09858325B1

    公开(公告)日:2018-01-02

    申请号:US14500782

    申请日:2014-09-29

    CPC classification number: G06F11/14 G06F17/30584

    Abstract: A data storage service distributes a plurality of data fragments corresponding to a data object among one or more data storage host groupings in a manner that avoids a possibility of correlated loss of multiple data fragments by consolidation of data of a data storage host grouping onto a single data storage host. The data storage service selects a data storage host grouping and determines an amount of used capacity for the selected data storage host grouping. If the selected grouping satisfies an emptiness threshold, the data storage service selects a data storage host from the grouping and consolidates one or more data sets of the grouping onto the selected data storage host. Subsequently, the data storage service updates metadata for each data storage host of the selected data storage host grouping to specify a location of data stored therein.

    Audit correctness of asynchronous batch transformation systems

    公开(公告)号:US10915417B1

    公开(公告)日:2021-02-09

    申请号:US16358273

    申请日:2019-03-19

    Abstract: First audit information corresponding to a first set of log entries associated with a transformation performed on first data is obtained, where the first audit information includes a first commutative result produced by applying a commutative function to object identifiers associated with the first set of log entries. Second audit information corresponding to a second set of log entries associated with the transformation performed on second data is obtained, with the second data being a different representation of the first data and where the second audit information includes a second commutative result produced by applying the commutative function to object identifiers associated with the second set of log entries. The first commutative result is compared with the second commutative result to validate whether the second data matches the first data. One or more actions are performed depending on whether the second data is the successful transformation of the first data.

    Cycled clustering for redundancy coded data storage systems

    公开(公告)号:US10366062B1

    公开(公告)日:2019-07-30

    申请号:US15083125

    申请日:2016-03-28

    Abstract: A cluster of data transfer devices is used to augment the capabilities of a data storage system. For example, the cluster of data transfer devices may be configured to store a portion of a bundle of redundancy coded shards in a similar fashion as a data storage system. As another example, the cluster may be configured to provide other capabilities incident to the devices used, such as computational capabilities. Data stored on the cluster may be read from and written directly to the cluster without transfer of data to the data storage system. In some embodiments, a connecting entity (such as a customer entity) may interchangeably interface with the data storage system and the cluster, and the requested capabilities may be directed to either in a fashion that is transparent to the requestor.

    Non-parity in grid encoded data storage systems

    公开(公告)号:US09998539B1

    公开(公告)日:2018-06-12

    申请号:US14789815

    申请日:2015-07-01

    CPC classification number: H04L67/1097 G06F11/08

    Abstract: Techniques for encoding data storage systems using grid-encoded data storage systems with non-parity linear redundancy encoding schemes are described herein. A grid of shards with derived shards and data shards is generated that is indexed by a first index and a second index and is configured so that each shard is reproducible from other shards with the same first index and is also reproducible from other shards with the same second index. The grid of shards is further configured so that each data row of the grid of shards has at least two derived shards and at least twice as many data shards as derived shards.

Patent Agency Ranking