Abstract:
Systems and methods providing for storing fragments for one or more source objects at storage nodes of multiple cluster instances comprising a cluster set are disclosed. Repair of the stored data is operated within cluster instances autonomously or semi-autonomously of other cluster instances of the cluster set according to embodiments. Embodiments may provide a storage process operable to generate fragments for a first source object to be stored in a storage system using a first encoding, wherein a different plurality of fragments of the generated fragments are stored at different cluster instances of a cluster set. A repair process may be enacted at each cluster instance of the cluster set, wherein a repair process enacted at a cluster instance is used to maintain the recoverability of the fragments of the first source object stored at that cluster instance using a second encoding.
Abstract:
Systems and methods utilizing available storage space within a storage system (e.g., as “supplemental” storage) and/or implement less physical storage space in the storage system (e.g., reduced storage overhead) through operation of fragment pre-storage techniques are disclosed. Such fragment pre-storage utilization of the aforementioned available storage space may provide operation emulating larger storage overhead than is actually provided in the storage system, facilitate improved repair rates, and/or facilitate reduced repair bandwidth in the storage system according to embodiments. A fragment pre-storage repair policy may implement source object repair whereby additional fragments for the source object are pre-generated and pre-stored in the storage system as transient fragments, whereby the transient fragments are moved to corresponding storage nodes when those storage nodes become physically present in the storage system.
Abstract:
Systems and methods which implement one or more data organization techniques that facilitate efficient access to source data stored by a storage system are disclosed. Data organization techniques implemented according to embodiments are adapted to optimize (e.g., maximize) input/output efficiency and/or (e.g., minimize) storage overhead, while maintaining mean time to data loss, repair efficiency, and/or traffic efficiency. Data organization techniques as may be implemented by embodiments include blob based organization techniques, grouped symbols organization techniques, data ordering organization techniques, and combinations thereof.
Abstract:
Systems and methods providing for storing fragments for one or more source objects at storage nodes of multiple cluster instances comprising a cluster set are disclosed. Repair of the stored data is operated within cluster instances autonomously or semi-autonomously of other cluster instances of the cluster set according to embodiments. Embodiments may provide a storage process operable to generate fragments for a first source object to be stored in a storage system using a first encoding, wherein a different plurality of fragments of the generated fragments are stored at different cluster instances of a cluster set. A repair process may be enacted at each cluster instance of the cluster set, wherein a repair process enacted at a cluster instance is used to maintain the recoverability of the fragments of the first source object stored at that cluster instance using a second encoding.
Abstract:
Transport accelerator (TA) systems and methods for accelerating delivery of content to a user agent (UA) of a client device are provided according to embodiments of the present disclosure. Embodiments comprise a TA architecture implementing a connection manager (CM) and a request manager (RM). A CM of embodiments requests chunks of content from a content server, receives data in response to requesting the chunks of content, wherein the received data is missing data from a requested chunk of content, and provides a receipt acknowledgement (ACK) for the missing data. The received data, which is missing data from a requested chunk of the chunks of content, may be passed through a communication protocol stack to an application for assembly into a one or more content objects.
Abstract:
Transport accelerator (TA) systems and methods for delivery of content to a user agent (UA) of the client device from a content server are provided according to embodiments of the present disclosure. Embodiments of a TA operate to subdivide, by a request manager (RM) of the TA, fragment requests provided by the UA each into a plurality of chunk requests for requesting chunks of the content and to provide, by the RM to a connection manager (CM) of the TA, chunk requests of the plurality of chunk requests for requesting chunks of the content. Requests may thus be made, by the CM, for the chunks of the content from the content server via a plurality of connections established between the CM and the content server.
Abstract:
Systems and methods which implement storage system data repair control techniques, such as may provide a feedback control structure for regulating source object redundancy and/or repair bandwidth in the storage system. Embodiments control a source object redundancy level to be used in a storage system by analyzing source objects represented in a repair queue to determine repair rate metrics for the source objects and determining a source object redundancy level based on the repair rate metrics. For example, embodiments may cause more redundant fragments for each source object to be generated and stored during repair where the repair rate metrics indicate an increase in storage node failure rate. Additionally, embodiments may determine a per storage object repair rate (e.g., a repair rate preference for each of a plurality of source objects) and select a particular repair rate (e.g., a maximum repair rate) for use by a repair policy.
Abstract:
Systems and methods which implement forward checking of data integrity are disclosed. A storage system of embodiments may, for example, comprise data integrity forward checking logic which is operable to perform forward checking of data integrity in real-time or near real-time to check that a number of node failures can be tolerated without loss of data. Embodiments may be utilized to provide assurance that a number of fragments needed for source data recovery will be available for the source objects most susceptible to failure when a certain number of additional fragments are lost, such as due to storage node failures.
Abstract:
A receiver receives media for playing out using a presentation element of the receiver can make requests and wait for responses, but can also cancel requests, possibly reissuing new requests, to improve a user experience taking into account network and other conditions. The receiver can select a playback rate and make requests at that playback rate, monitor a presentation buffer that stores media data to be consumed by a presentation element, store an indication of a buffer level corresponding to how much of the presentation buffer is occupied by the media data that is received and not yet consumed by the presentation element, maintain a state of an issued request for downloading a selected first chunk of media data, and when an issued request is outstanding, determine, based on network conditions and the state of the issued request, whether to continue the request or cancel the request.
Abstract:
A client device includes one or more processors configured to receive, from a server device, forward-error corrected data via a plurality of parallel network paths, determine losses of the data over each of the network paths, and send data representing the losses of the data over each of the network paths to the server device. Additionally or alternatively, a client device includes one or more processors configured to receive a first set of encoding units for a first block, wherein the first set of encoding units includes fewer than a minimum number of encoding units needed to recover the first block, after receiving the first set of encoding units, receive a second set of encoding units for a second block, and after receiving the second set of encoding units, receive a third set of encoding units including one or more encoding units for the first block.