Acceleration Of Data Transmission Between Producers And Consumers Of A Distributed-Clustered Application

    公开(公告)号:US20240362182A1

    公开(公告)日:2024-10-31

    申请号:US18308876

    申请日:2023-04-28

    申请人: NetApp, Inc.

    IPC分类号: G06F16/11 G06F16/182

    CPC分类号: G06F16/122 G06F16/183

    摘要: The technology disclosed herein enables accelerated data transmission between producers and consumers. In a particular example, a method includes receiving a first request from a producer-connector component of a producer component to store a payload to a storage repository. In response to the first request, the method includes providing a unique identifier to the connector component. The connector component provides the unique identifier to the distributed-clustered application. The method further includes storing the payload in association with the unique identifier to the storage repository. The method also includes retrieving the payload from the storage repository using the unique identifier to identify the payload in the storage repository. The method includes receiving a second request from a consumer-connector component of the consumer component to retrieve the payload. In response to the second request, the method includes supplying the payload to the consumer component.

    MIRRORING OBJECTS BETWEEN DIFFERENT CLOUD PROVIDERS WITH DIFFERENT DATA LAYOUT REQUIREMENTS

    公开(公告)号:US20240362124A1

    公开(公告)日:2024-10-31

    申请号:US18308337

    申请日:2023-04-27

    申请人: NetApp Inc.

    摘要: Techniques are provided for mirroring objects between object stores hosted by cloud providers that have different data layout requirements. An object may be stored within a first object store that supports a fix offset format where uncompressed data is stored according to fixed offsets and boundaries within fixed size objects. A mirroring operation may be used to mirror the object to a second object store that supports a unified object format where compressed data can be stored at non-fixed offsets and boundaries within variable sized objects. The mirroring operation selects a compression algorithm and compresses the object on the fly to create a mirrored object having the unified object format. The mirrored object, populated with the compressed data and slot header metadata comprising compression information for how to locate and decompress the data in the mirrored object, is stored into the second object store.

    READ ACCESS DURING CLONE VOLUME SPLIT OPERATION

    公开(公告)号:US20240361944A1

    公开(公告)日:2024-10-31

    申请号:US18308704

    申请日:2023-04-28

    申请人: NetApp Inc.

    IPC分类号: G06F3/06

    摘要: Techniques are provided for processing read operations while splitting a clone volume from a parent volume whose data is stored within objects of an object store. A transfer map is created to track mappings of child object identifiers to parent object identifiers of the parent objects to copy as child objects having the child object identifiers. The transfer map for the object store is traversed to copy the parent objects as the child objects for the split clone operation. The child objects are verified as being successfully created with valid data. In response to determining that the parent object has been copied as the child object, a context check is performed using the reverse map to verify a block within the child object. In response a successful context check, the read operation is processed using the block of the child object.

    DYNAMIC QUALITY OF SERVICE IMPLEMENTATION BASED UPON RESOURCE SATURATION

    公开(公告)号:US20240361912A1

    公开(公告)日:2024-10-31

    申请号:US18307097

    申请日:2023-04-26

    申请人: NetApp Inc.

    IPC分类号: G06F3/06

    摘要: Techniques are provided for dynamically implementing quality of service policies for a distributed storage system based upon resources saturation. A quality of service policy is defined for throttling I/O operations received by a node of the distributed storage system based upon whether resources of the node have become saturated. The quality of service policy is dynamically implemented based upon ever changing resource utilization and saturation. Dynamically implementing the quality of service policy improves the ability to efficiently utilize resources of the node compared to conventional static polices that cannot adequately react to such changing considerations and resource utilization/saturation. With conventional static policies, an administrator manually defines a minimum amount of guaranteed resources and/or a maximum resource usage cap that could be set to values that result in inefficient operation and resource starvation. Dynamically implementing the quality of service policy results in more efficient operation and mitigates resource starvation.

    PRESERVING STORAGE EFFICIENCY DURING RESTORATION OF DATA FROM THE CLOUD TO A DATA STORAGE SYSTEM

    公开(公告)号:US20240354198A1

    公开(公告)日:2024-10-24

    申请号:US18304583

    申请日:2023-04-21

    申请人: NetApp, Inc.

    IPC分类号: G06F11/14

    CPC分类号: G06F11/1451

    摘要: Systems and methods for preserving storage efficiency during restoration of data from the cloud are provided. In one embodiment, a CBMAP is maintained that maps cloud block numbers (CBNs) to respective corresponding block numbers of a volume of a data storage system in which previously restored data has been stored by a previously restored file. By making use of the CBMAP during the restoration process, storage of duplicate file data blocks on the volume may be avoided by sharing with a current file being restored a reference to the corresponding file data block previously stored on the volume and associated with the previously restored file. In addition to preserving storage efficiency, use of the CBMAP facilitates avoidance of repeated GET operations for data associated with CBNs previously retrieved from the cloud and stored to the volume, thereby reducing data access costs as well as latency of the restore operation.

    Configurable quality of service provider pipeline

    公开(公告)号:US12126502B1

    公开(公告)日:2024-10-22

    申请号:US18307103

    申请日:2023-04-26

    申请人: NetApp Inc.

    摘要: Techniques are provided for dynamically implementing quality of service policies using a configurable quality of service provider pipeline. A quality of service policy is defined for throttling I/O operations received by a node based upon whether resources of the node have become over utilized. The quality of service policy is used to dynamically construct a quality of service provider pipeline with select quality of service providers that improve the ability to efficiently utilize resources compared to conventional static polices that cannot adequately react to changing considerations and resource utilization/saturation. With conventional static policies, an administrator manually defines a minimum amount of guaranteed resources and/or a maximum resource usage cap that could be set to values that result in inefficient operation and resource starvation. Dynamically constructing and utilizing the quality of service provider pipeline results in more efficient operation and mitigates resource starvation.

    BACKUP AND RESTORE OF RESOURCES LOCATED WITHIN A REMOTE OBJECT STORE

    公开(公告)号:US20240320103A1

    公开(公告)日:2024-09-26

    申请号:US18731624

    申请日:2024-06-03

    申请人: NetApp, Inc.

    IPC分类号: G06F11/14 G06F9/54

    摘要: Techniques are provided for backing up and restoring a file system or storage virtual machine located within a remote object store. A specification is parsed to identify resources associated with and including a primary resource hosted within a remote object store and to identify REST API endpoints of the resources. GET operations targeting the REST API endpoints of the resources are performed to retrieve the resources and properties of the resources. A link relationship specification is parsed to identify links corresponding to dependencies amongst the resources. A backup of the primary resource is generated to include the resources, the properties of the resources, and dependency information derived from the links. The backup can be used to restore the primary resource to the remote object store in manner that preserves the dependencies amongst the resources.