Scalable segment cleaning for a log-structured file system

    公开(公告)号:US11494110B2

    公开(公告)日:2022-11-08

    申请号:US16999897

    申请日:2020-08-21

    Applicant: VMware, Inc.

    Abstract: Scalable segment cleaning for log-structured file systems (LFSs) includes determining counts of segment cleaners and virtual nodes, with each virtual node being associated with a plurality of objects. Each virtual node is assigned to a selected segment cleaner. Based at least on the assignments, performing, for each virtual node, segment cleaning of the objects by the assigned segment cleaner. A portion, less than all, of the virtual nodes are reassigned to a newly selected segment cleaner based on a change of the count of the segment cleaners and/or a change of the count of the virtual nodes. Based at least on the reassignments, segment cleaning of the objects is performed, for each reassigned virtual node, by the reassigned segment cleaner. In some examples, the objects comprise virtual machine disks (VMDKs) and the segment cleaning uses a segment usage table (SUT) to track segment usage and identify segment cleaning candidates.

    Shrinking segment cleaning algorithm in an object storage

    公开(公告)号:US11435935B2

    公开(公告)日:2022-09-06

    申请号:US17100663

    申请日:2020-11-20

    Applicant: VMware, Inc.

    Abstract: A method for cleaning an object storage having a plurality of segments is provided. Each segment includes an identifier through which the segment is accessed. The method identifies a first segment in the plurality of segments. The first segment includes a first identifier and a first size. The method determines that a utilization ratio for the first segment is below a threshold. As a result, the method generates a second segment from the first segment, such that the second segment includes a second identifier that is the same as the first identifier and a second size that is smaller than the first size. The method then writes the second segment to the object storage.

    Issuing efficient writes to erasure coded objects in a distributed storage system with two tiers of storage

    公开(公告)号:US11334277B2

    公开(公告)日:2022-05-17

    申请号:US16842649

    申请日:2020-04-07

    Applicant: VMware, Inc.

    Abstract: Techniques for issuing efficient writes to an erasure coded storage object in a distributed storage system are provided. In one set of embodiments, a node of the system can receive a write request for updating a logical data block of the storage object, write data/metadata for the block to a record in a data log of a metadata object of the storage object (where the metadata object is stored on a performance storage tier), place the block data in a free slot of an in-memory bank, and determine whether the in-memory bank has become full. If the in-memory bank is full, the node can further allocate a segment in a capacity object of the storage object for holding contents of the in-memory bank (where the capacity object is stored on a capacity storage tier), and write the in-memory bank contents via a full stripe write to the allocated segment.

    SCALABLE SEGMENT CLEANING FOR A LOG-STRUCTURED FILE SYSTEM

    公开(公告)号:US20220057955A1

    公开(公告)日:2022-02-24

    申请号:US16999897

    申请日:2020-08-21

    Applicant: VMware, Inc.

    Abstract: Scalable segment cleaning for log-structured file systems (LFSs) includes determining counts of segment cleaners and virtual nodes, with each virtual node being associated with a plurality of objects. Each virtual node is assigned to a selected segment cleaner. Based at least on the assignments, performing, for each virtual node, segment cleaning of the objects by the assigned segment cleaner. A portion, less than all, of the virtual nodes are reassigned to a newly selected segment cleaner based on a change of the count of the segment cleaners and/or a change of the count of the virtual nodes. Based at least on the reassignments, segment cleaning of the objects is performed, for each reassigned virtual node, by the reassigned segment cleaner. In some examples, the objects comprise virtual machine disks (VMDKs) and the segment cleaning uses a segment usage table (SUT) to track segment usage and identify segment cleaning candidates.

    REDUCED WRITE AMPLIFICATION USING SMALL-BLOCK LOG

    公开(公告)号:US20220012132A1

    公开(公告)日:2022-01-13

    申请号:US16925168

    申请日:2020-07-09

    Applicant: VMware, Inc.

    Abstract: Processing write requests from clients includes logging the associated data into logs corresponding to the data objects targeted by the write requests. The logs are persisted by combining log entries from each of the logs into one ore more fixed-size data blocks. The fixed-size data blocks are inserted into a data tree stored on a block-based storage device.

    Enhanced data encryption in distributed datastores using a cluster-wide fixed random tweak

    公开(公告)号:US11061594B1

    公开(公告)日:2021-07-13

    申请号:US16827674

    申请日:2020-03-23

    Applicant: VMware, Inc.

    Abstract: A method for encrypting data in one or more data blocks is provided. The method generates a fixed random tweak. The method receives first and second data blocks to write on at least one physical disk of a set of physical disks associated with a set of host machines. The method applies a fixed random tweak to data indicative of the first data block and data indicative of the second data block to generate, respectively, first and second encrypted data blocks. The method writes first and second entries to a data log in a cache, the first entry comprising a first header and the first encrypted data block and the second entry comprising a second header and the second encrypted data block. The method then writes the first and second encrypted data blocks to the at least one physical disk.

    Log-structured formats for managing archived storage of objects

    公开(公告)号:US11436102B2

    公开(公告)日:2022-09-06

    申请号:US16998060

    申请日:2020-08-20

    Applicant: VMware, Inc.

    Abstract: Solutions for managing archived storage include receiving, at a first node, a snapshot comprising object data (e.g., a virtual machine disk snapshot) from a second node (e.g., a software defined data center), and storing the snapshot in a tiered structure that includes a data tier and a metadata tier. Snapshots may be used for fail-over operations and/or backups, to support disaster recovery. The data tier comprises a log-structured file system (LFS), and the metadata tier comprises a content addressable storage (CAS) identifying addresses within the LFS. The metadata tier also comprises a logical layer indicating content in the CAS. Segment cleaning of the data tier is performed using a segment usage table (SUT). Some examples include performing a fail-over operation from the second node to a third node using at least the stored snapshot for workload recovery. In some examples, the CAS comprises a log-structured merge-tree (LSM-tree).

Patent Agency Ranking