PARTITIONING MECHANISM FOR PARALLEL PROCESSING IN DELTA GENERATION

    公开(公告)号:US20230409597A1

    公开(公告)日:2023-12-21

    申请号:US18181414

    申请日:2023-03-09

    IPC分类号: G06F16/27

    CPC分类号: G06F16/27

    摘要: Techniques are described for partitioning B-tree keys of file systems into key ranges for parallel processing in delta generation during file storage replications between file systems in different cloud infrastructure regions. In certain embodiments, a delta generation processing for cross-region replication may utilize a key-range splitting mechanism involving a recursive algorithm that partitions B-tree keys of a source file system into roughly equal-size key ranges. All the partitioned key ranges may be processed in parallel and concurrently by different processing threads, one thread per key range, to improve the performance of the delta generation and achieve scalability.

    CONCURRENT AND NON-BLOCKING OBJECT DELETION FOR CROSS-REGION REPLICATIONS

    公开(公告)号:US20240094937A1

    公开(公告)日:2024-03-21

    申请号:US18521176

    申请日:2023-11-28

    IPC分类号: G06F3/06

    摘要: Techniques are described for enabling concurrent and non-blocking replication object deletion during cross-region replications. In some embodiments, in a target file system, a target replication pipeline as part of a cross-region replication, and a deletion pipeline operate in parallel. The deletion pipeline deletes processed objects reaching the last pipeline stage of the target replication pipeline after each checkpoint in the target replication pipeline. In some embodiments, after a non-recoverable failure during the cross-region replication, the cross-region replication can be restarted from the beginning (i.e., fresh restart) without waiting for its unused objects in the Object Store to be deleted by utilizing a generation number associated with each object to delete the unused objects in a background process while allowing deleting processed objects as normal for the freshly restarted cross-region replication.