摘要:
The disclosure provides techniques for deduplicating files. The techniques include, upon creating or modifying a file, placing a logical timestamp of the current logical time, within a queue associated with the directory of the file. The techniques further include placing the logical timestamp within a queue of each parent directory of the directory of the file. To determine a set of files for deduplication, the techniques disclosed herein identify files that have been modified within a logical time range. The set of files modified within a logical time is identified by traversing directories of a storage system, the directories being organized within a tree structure. If a directory's queue does not contain a timestamp that is within the logical time range, then all child directories can be skipped over for further processing, such that no files within the child directories end up being within the set of files for deduplication.
摘要:
Examples disclosed herein relate to propagating changes made on a file system volume of a primary cluster of nodes to the same file system volume also being managed by a secondary cluster of nodes. An application is executed on both clusters, and data changes on the primary cluster are mirrored to the secondary cluster using an exo-clone file. The exo-clone file includes the differences between two or more snapshots of the volume on the primary cluster, along with identifiers of the change blocks and (optionally) state information thereof. Just these changes, identifiers, and state information are packaged in the exo-clone file and then exported to the secondary cluster, which in turn makes the changes to its version of the volume. Exporting just the changes to the data blocks and the corresponding block identifiers drastically reduces the information needed to be exchanged and processed to keep the two volumes consistent.
摘要:
In a computer system, a method of controls interrupts which correspond to input/output (I/O) processing. For each delivery of an I/O completion interrupt, the method provides a recordation of a delivery time; identifies I/O completions for which deliveries of corresponding I/O completion interrupts involve deliveries of inter-processor interrupts; and for each of the identified I/O completions, accesses the recordation of the most recent delivery time to determine whether a selected period of time has elapsed since a last delivery of an inter-processor interrupt. As a response to a determination that the selected period has elapsed, an inter-processor interrupt is delivers. As a response to a determination that less than the duration of the selected period has elapsed, the method refrains from delivering an inter-processor interrupt.
摘要:
Completion interrupts corresponding to I/O requests issued by a virtual machine guest, which runs on a host platform, are virtualized in such a way that I/O completion interrupts to the requesting guest are delivered no faster than it can stably handle them, but, when possible, faster than the nominal speed of a virtual device to which a virtual machine addresses the I/O request. In general, completion events received from the host platform in response to guest I/O requests are examined with respect to time. If enough time has passed that the virtual device would normally have completed the I/O request, then the completion interrupt is delivered to the guest. If the nominal time has not elapsed, however, the invention enqueues and time-stamps the event and delivers it at the earliest of a) the normal maturity time, or b) at a safepoint.
摘要:
In order to synchronize copies of a virtual disk, a virtualization layer maintains a first record of file system blocks of a first copy of the virtual disk that are modified during an access session by a virtual machine using the first copy of the virtual disk. The file system blocks correspond to a file system of the virtual disk. During an attempt to synchronize the first copy with a second copy of the virtual disk, (i) a second record of file system blocks that are currently used by the file system is obtained from the guest operating system, and (ii) file system blocks in the first copy of the virtual disk that are present in both the first record and the second record are copied into the second copy of the virtual disk.
摘要:
In one set of embodiments, a computer system can create a snapshot of a data set, where the snapshot includes a plurality of data blocks of the data set. The computer system can upload the snapshot to a cloud object storage platform of a cloud infrastructure, where the snapshot is uploaded as a plurality of log segments conforming to an object format of the cloud object storage platform, and where each log segment includes one or more data blocks in the plurality of data blocks, and a set of metadata comprising, for each of the one or more data blocks, an identifier of the data set, an identifier of the snapshot, and a logical block address (LBA) of the data block. The computer system can then communicate the set of metadata to a server component running in a cloud compute and block storage platform of the cloud infrastructure.
摘要:
The disclosure herein describes writing data to a log-structured merge (LSM) tree file system on an object storage platform. Write data instructions indicating data for writing to the LSM tree file system are received. Based on the received instructions, the data is written to the first data cache. Based on an instruction to transfer data in the live data cache to the LSM tree file system, the first data cache is converted to a stable cache. A second data cache configured as a live data cache is then generated based on cloning the first data cache. The data in the first data cache is then written to the LSM tree file system. Use of a stable cache and a cloned live data cache enables parallel writing data to the file system by the stable cache and handling write data instructions by the live data cache.
摘要:
The present disclosure provides techniques for deallocating previously allocated storage blocks. The techniques include obtaining a list of chunk IDs to analyze, choosing a chunk ID, and determining the storage blocks spanned by the chunk corresponding to the chosen chunk ID. The technique further includes determining whether any file references any storage blocks spanned by the chunk. The determining may be performed by comparing an internal reference count to a total reference count, where the internal reference count is the number of reference to the storage block by a chunk ID data structure. If no files reference any of the storage blocks spanned by the chunk, then all the storage blocks of the chunk can be deallocated.
摘要:
The present disclosure provides techniques for deduplicating files. The techniques include creating a data structure that organizes metadata about chunks of files, the organization of the metadata preserving order and locality of the chunks within files. The organization of the metadata within storage blocks of storage devices matches the order of chunks within files. Upon a read or write operation to a metadata, the preservation of locality of metadata results in the likely fetching, from storage into a memory cache, metadata of subsequent and contiguous chunks. The preserved locality results in faster subsequent read and write operations of metadata, because the read and write operations are likely to be executed from memory rather than from storage.
摘要:
Examples provide a method of communication between a client driver and a filesystem server. The client driver executes in a virtual machine (VM) and the filesystem server executes in a hypervisor. The method includes: allocating, by the client driver, shared memory in an address space of the VM for the communication; sending identification information for the shared memory from the client driver to the filesystem server through an inter-process communication channel between the client driver and the filesystem server; identifying, by the filesystem server in cooperation with a kernel of the hypervisor, the shared memory within an address space of the hypervisor, based on the identification information, to create a shared memory channel; sending commands from the client driver to the filesystem server through the shared memory channel; and receiving completion messages for the commands from the filesystem server to the client driver through the shared memory channel.