Fast algorithm to find file system difference for deduplication

    公开(公告)号:US11775484B2

    公开(公告)日:2023-10-03

    申请号:US16552965

    申请日:2019-08-27

    申请人: VMware, Inc.

    摘要: The disclosure provides techniques for deduplicating files. The techniques include, upon creating or modifying a file, placing a logical timestamp of the current logical time, within a queue associated with the directory of the file. The techniques further include placing the logical timestamp within a queue of each parent directory of the directory of the file. To determine a set of files for deduplication, the techniques disclosed herein identify files that have been modified within a logical time range. The set of files modified within a logical time is identified by traversing directories of a storage system, the directories being organized within a tree structure. If a directory's queue does not contain a timestamp that is within the logical time range, then all child directories can be skipped over for further processing, such that no files within the child directories end up being within the set of files for deduplication.

    Interrupt coalescing for outstanding input/output completions
    3.
    发明授权
    Interrupt coalescing for outstanding input/output completions 有权
    中断合并为突出的输入/输出完成

    公开(公告)号:US09336165B2

    公开(公告)日:2016-05-10

    申请号:US13933921

    申请日:2013-07-02

    申请人: VMware, Inc.

    IPC分类号: G06F13/24 G06F3/06 H04L12/801

    摘要: In a computer system, a method of controls interrupts which correspond to input/output (I/O) processing. For each delivery of an I/O completion interrupt, the method provides a recordation of a delivery time; identifies I/O completions for which deliveries of corresponding I/O completion interrupts involve deliveries of inter-processor interrupts; and for each of the identified I/O completions, accesses the recordation of the most recent delivery time to determine whether a selected period of time has elapsed since a last delivery of an inter-processor interrupt. As a response to a determination that the selected period has elapsed, an inter-processor interrupt is delivers. As a response to a determination that less than the duration of the selected period has elapsed, the method refrains from delivering an inter-processor interrupt.

    摘要翻译: 在计算机系统中,控制对应于输入/输出(I / O)处理的中断的方法。 对于每次交付I / O完成中断,该方法提供了交货时间的记录; 识别相应I / O完成中断的交付涉及交付处理器间中断的I / O完成; 并且对于每个所识别的I / O完成,访问最近传送时间的记录,以确定自上一次交付处理器中断之后所选择的时间段是否已经过去。 作为对所选择的时间段已经过去的确定的响应,提供了处理器间中断。 作为对小于所选择的时间段的持续时间的确定的响应,该方法不会传递处理器间中断。

    Efficient virtualization of input/output completions for a virtual device
    4.
    发明授权
    Efficient virtualization of input/output completions for a virtual device 有权
    虚拟设备的输入/输出完成的高效虚拟化

    公开(公告)号:US08875162B2

    公开(公告)日:2014-10-28

    申请号:US13963442

    申请日:2013-08-09

    申请人: VMware, Inc.

    摘要: Completion interrupts corresponding to I/O requests issued by a virtual machine guest, which runs on a host platform, are virtualized in such a way that I/O completion interrupts to the requesting guest are delivered no faster than it can stably handle them, but, when possible, faster than the nominal speed of a virtual device to which a virtual machine addresses the I/O request. In general, completion events received from the host platform in response to guest I/O requests are examined with respect to time. If enough time has passed that the virtual device would normally have completed the I/O request, then the completion interrupt is delivered to the guest. If the nominal time has not elapsed, however, the invention enqueues and time-stamps the event and delivers it at the earliest of a) the normal maturity time, or b) at a safepoint.

    摘要翻译: 在主机平台上运行的虚拟机访客发出的I / O请求相应的完成中断被虚拟化,使得对请求的客户端的I / O完成中断的传送速度不会比可以稳定地处理它们的速度更快,而是 在可能的情况下,虚拟机对于I / O请求所对应的虚拟设备的标称速度更快。 一般来说,根据客户端I / O请求从主机平台接收到的完成事件将相对于时间进行检查。 如果虚拟设备通常已经完成了I / O请求的足够的时间,则完成中断将传递给客户端。 然而,如果没有经过标称时间,本发明会在a)正常到期时间的最早时间排队和时间戳,并且b)在一个安全点。

    Loose synchronization of virtual disks
    5.
    发明授权
    Loose synchronization of virtual disks 有权
    虚拟磁盘的松动同步

    公开(公告)号:US08694746B2

    公开(公告)日:2014-04-08

    申请号:US13653048

    申请日:2012-10-16

    申请人: VMware, Inc.

    IPC分类号: G06F12/00

    CPC分类号: G06F11/1662 G06F11/2094

    摘要: In order to synchronize copies of a virtual disk, a virtualization layer maintains a first record of file system blocks of a first copy of the virtual disk that are modified during an access session by a virtual machine using the first copy of the virtual disk. The file system blocks correspond to a file system of the virtual disk. During an attempt to synchronize the first copy with a second copy of the virtual disk, (i) a second record of file system blocks that are currently used by the file system is obtained from the guest operating system, and (ii) file system blocks in the first copy of the virtual disk that are present in both the first record and the second record are copied into the second copy of the virtual disk.

    摘要翻译: 为了同步虚拟磁盘的副本,虚拟化层维护虚拟磁盘的第一副本的文件系统块的第一记录,该虚拟磁盘的第一副本在虚拟机的访问会话期间使用虚拟磁盘的第一副本进行修改。 文件系统块对应于虚拟磁盘的文件系统。 在尝试将第一副本与虚拟磁盘的第二副本同步时,(i)从客户操作系统获得文件系统当前使用的文件系统块的第二记录,以及(ii)文件系统块 在第一个记录和第二个记录中存在的虚拟磁盘的第一个副本被复制到虚拟磁盘的第二个副本中。

    Writing data to an LSM tree file structure using consistent cache staging

    公开(公告)号:US11620261B2

    公开(公告)日:2023-04-04

    申请号:US16213815

    申请日:2018-12-07

    申请人: VMware, Inc.

    摘要: The disclosure herein describes writing data to a log-structured merge (LSM) tree file system on an object storage platform. Write data instructions indicating data for writing to the LSM tree file system are received. Based on the received instructions, the data is written to the first data cache. Based on an instruction to transfer data in the live data cache to the LSM tree file system, the first data cache is converted to a stable cache. A second data cache configured as a live data cache is then generated based on cloning the first data cache. The data in the first data cache is then written to the LSM tree file system. Use of a stable cache and a cloned live data cache enables parallel writing data to the file system by the stable cache and handling write data instructions by the live data cache.

    Efficient garbage collection of variable size chunking deduplication

    公开(公告)号:US11461229B2

    公开(公告)日:2022-10-04

    申请号:US16552954

    申请日:2019-08-27

    申请人: VMware, Inc.

    IPC分类号: G06F3/06 G06F12/02

    摘要: The present disclosure provides techniques for deallocating previously allocated storage blocks. The techniques include obtaining a list of chunk IDs to analyze, choosing a chunk ID, and determining the storage blocks spanned by the chunk corresponding to the chosen chunk ID. The technique further includes determining whether any file references any storage blocks spanned by the chunk. The determining may be performed by comparing an internal reference count to a total reference count, where the internal reference count is the number of reference to the storage block by a chunk ID data structure. If no files reference any of the storage blocks spanned by the chunk, then all the storage blocks of the chunk can be deallocated.

    LOW-LATENCY SHARED MEMORY CHANNEL ACROSS ADDRESS SPACES IN A COMPUTING SYSTEM

    公开(公告)号:US20220019456A1

    公开(公告)日:2022-01-20

    申请号:US17013727

    申请日:2020-09-07

    申请人: VMware, Inc.

    摘要: Examples provide a method of communication between a client driver and a filesystem server. The client driver executes in a virtual machine (VM) and the filesystem server executes in a hypervisor. The method includes: allocating, by the client driver, shared memory in an address space of the VM for the communication; sending identification information for the shared memory from the client driver to the filesystem server through an inter-process communication channel between the client driver and the filesystem server; identifying, by the filesystem server in cooperation with a kernel of the hypervisor, the shared memory within an address space of the hypervisor, based on the identification information, to create a shared memory channel; sending commands from the client driver to the filesystem server through the shared memory channel; and receiving completion messages for the commands from the filesystem server to the client driver through the shared memory channel.