Local storage clustering for redundancy coded data storage system

    公开(公告)号:US11113161B2

    公开(公告)日:2021-09-07

    申请号:US16114033

    申请日:2018-08-27

    Abstract: A data transfer device is used to augment the capabilities of a data storage system. The data transfer device may be capable of persistently storing data for an indeterminate amount of time, and may be configured to store a portion of a bundle of redundancy coded shards that span between the data transfer device and a data storage system configured to store the remainder of the bundle. Data stored on the data transfer device may be read from and written directly to the data transfer device without transfer of data to the data storage system. If the data transfer device is not available, the remaining shards of the bundle may provide a regenerated, original form of the data.

    Object loss reporting in a data storage system

    公开(公告)号:US10970276B2

    公开(公告)日:2021-04-06

    申请号:US16105258

    申请日:2018-08-20

    Abstract: In response to receiving a request from a client to store an object, a key-durable storage system may assign the object to a volume in its data store, generate a key for the object (e.g., an opaque identifier that encodes information for locating the object in the data store), store the object on one disk in the assigned volume, store the key redundantly in the assigned volume (e.g., using a replication or erasure coding technique), and may return the key to the client. To retrieve the object, the client may send a request including the key, and the system may return the object to the client. If a disk fails, the system may determine which objects were lost, and may return the corresponding keys to the appropriate clients in a notification. The system may be used to back up a more expensive object-redundant storage system.

    MIGRATION OF INFORMATION VIA STORAGE DEVICES
    43.
    发明申请

    公开(公告)号:US20200379651A1

    公开(公告)日:2020-12-03

    申请号:US16998709

    申请日:2020-08-20

    Abstract: A device such as a network-attachable data transfer device may be configured to operate in a cluster to coordinate the storage of data. A first manifest may be generated inventorying a first set of data successfully transferred to the data transfer device from a data source. A second manifest may be generated inventorying a second set of data successfully transferred from the data transfer device to a data destination. The first manifest may be compared with the second manifest to determine a transfer status of one or more data objects. The transfer status may indicate one or more data objects successfully transferred to the data destination from the data source. The one or more objects may be processed according to the transfer status.

    Using feedback for adaptive data compression

    公开(公告)号:US10732902B1

    公开(公告)日:2020-08-04

    申请号:US16143258

    申请日:2018-09-26

    Abstract: A hub device or edge device implements adaptive data compression. The model training service analyzes the received data. A machine learning model of the hub device receives time-series data from one or more data sources and classifies respective portions of the time-series data as respective patterns. A data compressor at the hub device generates compressed data by applying different compression techniques to the respective portions of the time-series data according to a mapping of the compression techniques to the respective patterns. The hub device then transmits the compressed data to an endpoint for processing (e.g., another device that uses the compressed data). The hub device receives feedback for the compressed data. In response to the feedback, the hub device changes one or more of the compression techniques that are mapped to the respective patterns.

    Systems and techniques for data recovery in a keymapless data storage system

    公开(公告)号:US10521356B1

    公开(公告)日:2019-12-31

    申请号:US13770892

    申请日:2013-02-19

    Abstract: Components of a data object are distributed throughout a data storage system. Manifests are used to store the locations of the components of data objects in a data storage system to allow for subsequent reconstruction of the data objects. The manifests may be stored in another data storage system when cost projections indicate it being economical to do so. If a manifest for a data object becomes lost or otherwise inaccessible, clues are used to regenerate the manifest, thereby providing a continued ability to access the components of the data object to reconstruct the data object.

    SHIPPABLE NETWORK-ATTACHED DATA STORAGE DEVICE WITH UPDATEABLE ELECTRONIC DISPLAY

    公开(公告)号:US20190340571A1

    公开(公告)日:2019-11-07

    申请号:US16517311

    申请日:2019-07-19

    Abstract: A network-attachable data transfer device housed within a shippable enclosure that incorporates an updateable electronic display for displaying shipping destination information is disclosed. The device may be initialized (e.g., prepared to receive data, and the updateable electronic shipping display set to the shipping destination) by a service provider and shipped, in accordance with the displayed destination address, as a self-contained shipping unit. The device may be installed onto a network at the destination and loaded with data. The display may also be updated with the next destination address such that the device is shipped to the updated destination address (e.g., back to the service provider, or onto other destinations before being send back to the service provider). When the device is received back at the service provider, the data is transferred from the device to a service provider storage facility, wiped of data, and prepared to be sent out again.

    Cycled clustering for redundancy coded data storage systems

    公开(公告)号:US10366062B1

    公开(公告)日:2019-07-30

    申请号:US15083125

    申请日:2016-03-28

    Abstract: A cluster of data transfer devices is used to augment the capabilities of a data storage system. For example, the cluster of data transfer devices may be configured to store a portion of a bundle of redundancy coded shards in a similar fashion as a data storage system. As another example, the cluster may be configured to provide other capabilities incident to the devices used, such as computational capabilities. Data stored on the cluster may be read from and written directly to the cluster without transfer of data to the data storage system. In some embodiments, a connecting entity (such as a customer entity) may interchangeably interface with the data storage system and the cluster, and the requested capabilities may be directed to either in a fashion that is transparent to the requestor.

    Failure mode-sensitive layered redundancy coding techniques

    公开(公告)号:US10270476B1

    公开(公告)日:2019-04-23

    申请号:US14741407

    申请日:2015-06-16

    Abstract: Techniques and methods for generating and implementing multiple layers of redundancy coded data are disclosed. For example, a redundancy coding scheme may include data elements that include data that is unencoded relative to the input, yet may still fully participate in providing redundancy to any data element in a given set. In a layered scheme, the input may include a bundle or group of encoded (or unencoded) data elements, thereby nesting two or more layers of redundancy coding. The specific amount of redundancy generated by such a scheme may be adjusted and adapted to failure characteristics of the entity on which the data elements are stored.

    Incremental backups for removable media

    公开(公告)号:US10162709B1

    公开(公告)日:2018-12-25

    申请号:US15360864

    申请日:2016-11-23

    Abstract: Techniques for storing incremental backups in long-term storage are described herein. A backup data set is generated from a set of data to backup and a deletion time for that backup data set is determined. A scheduled time period to perform the backup is selected based at least in part on a requested time period to perform the backup. The requested time period is altered if the performing the backup at the requested time period would produce a period of high resource contention at the destination for the backup. The backup is then stored at the destination at the scheduled time.

Patent Agency Ranking