公开(公告)号：US20240152290A1

公开(公告)日：2024-05-09

申请号：US18415352

申请日：2024-01-17

Applicant: HUAWEI TECHNOLOGIES CO., LTD. ,  TSINGHUA UNIVERSITY

Inventor： Jiwu Shu ,  Youyou Lu ,  Jian Gao ,  Xiaodong Tan ,  Wenlin Cui

IPC: G06F3/06

CPC classification number: G06F3/064 ,  G06F3/0619 ,  G06F3/067

Abstract: This application discloses a data writing method. A network controller performs erasure code encoding on original data, and writes a third quantity of target data blocks of a plurality of obtained target data blocks into a storage node. The network controller reads a first quantity of target data blocks of the third quantity of the target data blocks from the storage node, and decodes the read target data blocks. The plurality of target data blocks include a first quantity of original data blocks and a second quantity of check data blocks, two ends of a target data block include same version information, and the third quantity is greater than the first quantity.

公开(公告)号：US10191851B2

公开(公告)日：2019-01-29

申请号：US14980346

申请日：2015-12-28

Applicant: TSINGHUA UNIVERSITY

Inventor： Jiwu Shu ,  Youyou Lu ,  Fei Li

IPC: G06F12/00 ,  G06F12/1009 ,  G06F3/06

Abstract: The present invention discloses a method for distributed transaction processing in a flash memory, including the following steps: S1. performing two-phase commit on a transaction, removing a state log record of a two-phase commit protocol, and internalizing as an operation on flash memory metadata in a transaction interface; S2. storing a temporary data object as a shadow version while performing the transaction, using a shadow mapping table to store the address of the shadow version or a state of a page, using page metadata to record transaction information, using a transaction metadata page to record a transaction state, and using a transaction state table to record an address of the transaction metadata page; and S3. when the coordinator or a participant fails, recovering the FTL mapping table, shadow mapping table, and transaction state table with the help of stored data and flash memory metadata. The present invention has the following advantages: the number of logs is reduced, and performance of transaction processing in a distributed flash memory environment is improved.

公开(公告)号：US10048999B2

公开(公告)日：2018-08-14

申请号：US14980571

申请日：2015-12-28

Applicant: TSINGHUA UNIVERSITY

Inventor： Jiwu Shu ,  Zhirong Shen

IPC: G06F11/00 ,  G06F11/07 ,  H03M13/15

Abstract: The present invention discloses a method for optimizing recovery of a single-disk failure, including the following steps: obtaining, according to current load information, an amount of data expected to be read and an allowed number of iterations; obtaining a recovery optimization policy for failed data in each single stripe, and combining an initial recovery policy for multiple stripes; and further optimizing the initial recovery policy by using a greedy algorithm based on tabu search, subject to the amount of data expected to be read and the allowed number of iterations, to finally obtain an optimal recovery policy with a smallest quantity of seeks. The optimization method of the present invention reduces the amount of data to be read and the quantity of seek operations, and improves the efficiency of recovering a single-disk failure. The present invention further discloses an apparatus for optimizing recovery of a single-disk failure.

公开(公告)号：US11687392B2

公开(公告)日：2023-06-27

申请号：US17534121

申请日：2021-11-23

Applicant: Tsinghua University

Inventor： Jiwu Shu ,  Qing Wang ,  Youyou Lu ,  Zhuo Cheng

IPC: G06F16/23 ,  G06F16/901 ,  G06F11/07 ,  G06F9/30 ,  G06F11/14 ,  G06F12/02

CPC classification number: G06F11/073 ,  G06F9/3009 ,  G06F11/0772 ,  G06F11/1438 ,  G06F12/0238 ,  G06F16/2365 ,  G06F2212/2542 ,  Y10S707/99953

Abstract: A method for constructing a persistent memory index in a non-uniform memory access architecture includes: maintaining partial persistent views in a persistent memory and maintaining a global volatile view in a DRAM; an underlying persistent memory index processing a request in a foreground thread when cold data is accessed; when hot data is accessed, reading a key-value pair for a piece of hot data in the global volatile view in response to a query operation carried in the request, and in response to an insert/update/delete operation carried in the request, updating a local partial persistent view and the global volatile view; and in response to a hotspot migration, a background thread generating new partial persistent views and a new global volatile view, and recycling the partial persistent views and the global volatile view for old hot data into the underlying persistent memory index.

公开(公告)号：US09959053B2

公开(公告)日：2018-05-01

申请号：US14980462

申请日：2015-12-28

Applicant: TSINGHUA UNIVERSITY

Inventor： Jiwu Shu ,  Jiaxin Ou ,  Youyou Lu

IPC: G06F12/00 ,  G06F3/06 ,  G06F17/30

CPC classification number: G06F3/061 ,  G06F3/0659 ,  G06F3/0679 ,  G06F17/3007 ,  G06F17/30168 ,  G06F17/30218

Abstract: The present invention provides a method for constructing an NVRAM-based efficient file system, including the following steps: S1. determining a file operation type of the file system, where the file operation type includes a file read operation, a non-persistent file write operation, and a persistent file write operation; and S2. if the file operation type is a non-persistent file write operation, writing, by the file system, content of the non-persistent file write operation to a dynamic random access memory DRAM, updating a corresponding DRAM cache block index, and flushing, at a preset time point, the content of the non-persistent file write operation back to a non-volatile random access memory NVRAM asynchronously, or otherwise, copying, by the file system, related data directly between the NVRAM/DRAM and the user buffer. The method of the present invention can not only avoid additional copies and overheads of conventional software layers, but also improve write performance of the NVRAM file system by fully utilizing a characteristic of high write performance of the DRAM, thereby improving overall performance of the system.

公开(公告)号：US20160350216A1

公开(公告)日：2016-12-01

申请号：US14980248

申请日：2015-12-28

Applicant: TSINGHUA UNIVERSITY

Inventor： Jiwu Shu ,  Youyou Lu

IPC: G06F12/08 ,  G06F11/07 ,  G06F3/06

CPC classification number: G06F11/14 ,  G06F12/0238 ,  G06F2212/1032 ,  G06F2212/22 ,  G06F2212/60 ,  G06F2212/601 ,  G06F2212/604

Abstract: The present invention provides a method and an apparatus for cache management of transaction processing in persistent memory. The method includes: when a transaction starts, reading old version data from non-volatile memory to a processor cache, and executing the transaction; during the execution of the transaction, allocating a space to generated new version data, and using a steal write-back technology to allow persistence of uncommitted data; when the transaction is committed or aborted, implementing forced persistence of transaction data to the non-volatile memory; after persistence of the transaction data or state is implemented to the non-volatile memory, writing the transaction data back to original data addresses, and using a no-force write-back technology to relax persistence of committed data to the non-volatile memory, where the transaction processing module periodically implements persistence of cached data to the non-volatile memory using bulk persistence; and when a system failure occurs, performing failure recovery processing on the transaction data. The method can reduce frequencies of data replication and data persistence in persistent memory.

Abstract translation: 本发明提供了一种用于持久存储器中的事务处理的高速缓存管理的方法和装置。 该方法包括：当事务开始时，将旧版本数据从非易失性存储器读取到处理器高速缓存，并执行事务; 在执行交易期间，分配空间以生成新版本数据，并使用窃取回写技术来允许持续的未提交数据; 当事务被提交或中止时，将事务数据强制持久化到非易失性存储器; 在将事务数据或状态持续存在于非易失性存储器之后，将事务数据写回到原始数据地址，并且使用无力写回技术来将已提交数据的持续性放在非易失性存储器中， 其中事务处理模块使用批量持久性周期性地将高速缓存的数据持久化到非易失性存储器; 并且当发生系统故障时，对事务数据执行故障恢复处理。 该方法可以减少持久存储器中的数据复制频率和数据持久性。

公开(公告)号：US11899939B2

公开(公告)日：2024-02-13

申请号：US17372250

申请日：2021-07-09

Applicant: Huawei Technologies Co., Ltd. ,  TSINGHUA UNIVERSITY

Inventor： Jiwu Shu ,  Youmin Chen ,  Youyou Lu ,  Wenlin Cui

IPC: G06F3/06

CPC classification number: G06F3/0613 ,  G06F3/0604 ,  G06F3/067 ,  G06F3/0659

Abstract: A read/write request processing method and server are provided. In this method, each terminal is grouped, and different service durations are assigned for all terminal groups, so that a server can process, within any service duration, only a read/write request sent by a terminal in a terminal group corresponding to the service duration. According to the application, a cache area of a network interface card of the server is enabled to store only limited quantities of queue pairs (QPs) and work queue elements (WQEs), thereby preventing uneven resource distribution in the cache area of the network interface card.

公开(公告)号：US11086560B2

公开(公告)日：2021-08-10

申请号：US16553276

申请日：2019-08-28

Applicant: Tsinghua University

Inventor： Jiwu Shu ,  Youmin Chen ,  Bohong Zhu ,  Youyou Lu

IPC: G06F9/46 ,  G06F3/06

Abstract: The invention discloses a data storage access method, device and apparatus for persistent memory. The method includes: enabling a file system in device to receive, in a kernel space, an access request of a user library, where the user library operates in user mode, and the access request is initiated by third-party application through user library and carries operation type; if the operation type is read operation, enabling the file system to allow third-party application to directly access persistent memory space of device through user library; and if the operation type is not read operation, enabling the file system to allow third-party application to access persistent memory space of device through user library and kernel thread, where kernel thread operates in kernel mode. In this way, the present invention can achieve both high performance and data security of persistent memory, and provide technical support for scalability in multi-core scenario.

公开(公告)号：US20170091018A1

公开(公告)日：2017-03-30

申请号：US14980571

申请日：2015-12-28

Applicant: TSINGHUA UNIVERSITY

Inventor： Jiwu Shu ,  Zhirong Shen

IPC: G06F11/07 ,  H03M13/15

CPC classification number: G06F11/0793 ,  G06F11/0727 ,  G06F11/0751 ,  G06F11/079 ,  G06F11/1092 ,  H03M13/154

Abstract: The present invention discloses a method for optimizing recovery of a single-disk failure, including the following steps: obtaining, according to current load information, an amount of data expected to be read and an allowed number of iterations; obtaining a recovery optimization policy for failed data in each single stripe, and combining an initial recovery policy for multiple stripes; and further optimizing the initial recovery policy by using a greedy algorithm based on tabu search, subject to the amount of data expected to be read and the allowed number of iterations, to finally obtain an optimal recovery policy with a smallest quantity of seeks. The optimization method of the present invention reduces the amount of data to be read and the quantity of seek operations, and improves the efficiency of recovering a single-disk failure. The present invention further discloses an apparatus for optimizing recovery of a single-disk failure.

公开(公告)号：US11269772B2

公开(公告)日：2022-03-08

申请号：US16553253

申请日：2019-08-28

Applicant: Tsinghua University

Inventor： Jiwu Shu ,  Youmin Chen ,  Bohong Zhu ,  Youyou Lu

IPC: G06F12/06 ,  G06F16/21 ,  G06F16/22

Abstract: The present invention provides persistent memory storage engine device based on log structure and a control method, including persistent memory allocators, persistent operation logs, and a volatile index structure. The control method of log structure-based storage engine includes: allocating by persistent memory allocators, new spaces to each processor for storing updated key value pairs; organizing acquired operation information into compact log entries, and adding compact log entries into persistent operation logs according to first preset rule, where first preset rule is performing batch persistency on compact log entries from the plurality of processor cores; and updating index entries in volatile index structure to point to new key value pairs. This application fully exploits opportunity to reduce persistence overhead by redesigning log structure storage format and batch persistence mode. An efficient persistent memory key-value storage engine based on log structure is designed which reduces latency while maintaining high system throughput.