公开(公告)号：US10311026B2

公开(公告)日：2019-06-04

申请号：US15167277

申请日：2016-05-27

Applicant: International Business Machines Corporation

Inventor： M. Corneliu Constantinescu ,  Leo Shyh-Wei Luan ,  Wayne A. Sawdon ,  Frank B. Schmuck

IPC: G06F16/00 ,  G06F16/174 ,  G06F16/17

Abstract: The embodiments described herein relate to managing compressed data to optimize file compression for efficient random access to the data. A first partition of a first data block of a compression group is compressed. The first compressed partition is stored in a first compression entity. An in-memory table is maintained, which includes updating the in-memory table with data associated with an address of the stored compressed first partition. At such time as it is determined that the first compression entity is full, the in-memory table is compressed and written to the first compression entity. Accordingly, the in-memory table, which stores partition compression data, is store with the compression entity.

公开(公告)号：US20160246799A1

公开(公告)日：2016-08-25

申请号：US14627385

申请日：2015-02-20

Applicant: International Business Machines Corporation

Inventor： M. Corneliu Constantinescu ,  Marc A. Kaplan ,  Leo Luan ,  Sven Oehme ,  Wayne A. Sawdon ,  Frank B. Schmuck

IPC: G06F17/30

CPC classification number: G06F17/30156

Abstract: Embodiments relate to policy-based, multi-scheme data reduction for a computer memory. An aspect includes receiving a plurality of policy rules by a policy engine of the computer memory, wherein a first policy rule specifies applying a first data reduction scheme to data in the computer memory based on the data matching first characteristics, wherein a second policy rule specifies applying a second data reduction scheme to data in the computer memory based on the data matching second characteristics, wherein the first data reduction scheme is different from the second data reduction scheme. Another aspect includes determining, by the policy engine, that first data in the computer memory matches the first characteristics, and that second data in the computer memory matches the second characteristics. Yet another aspect includes applying the first data reduction scheme to the first data, and applying the second data reduction scheme to the second data.

Abstract translation: 实施例涉及用于计算机存储器的基于策略的多方案数据简化。 一方面包括由计算机存储器的策略引擎接收多个策略规则，其中第一策略规则指定基于匹配第一特征的数据将第一数据缩减方案应用于计算机存储器中的数据，其中第二策略规则指定 基于所述数据匹配第二特性对所述计算机存储器中的数据应用第二数据简化方案，其中所述第一数据简化方案与所述第二数据简化方案不同。 另一方面包括由策略引擎确定计算机存储器中的第一数据与第一特性匹配，并且计算机存储器中的第二数据与第二特性匹配。 另一方面包括将第一数据简化方案应用于第一数据，以及将第二数据简化方案应用于第二数据。

公开(公告)号：US20180336219A1

公开(公告)日：2018-11-22

申请号：US16047322

申请日：2018-07-27

Applicant: INTERNATIONAL BUSINESS MACHINES CORPORATION

Inventor： M. Corneliu Constantinescu ,  Marc A. Kaplan ,  Leo Luan ,  Sven Oehme ,  Wayne A. Sawdon ,  Frank B. Schmuck

IPC: G06F17/30

CPC classification number: G06F17/30156

Abstract: Embodiments relate to policy-based, multi-scheme data reduction for a computer memory. An aspect includes receiving a plurality of policy rules by a policy engine of the computer memory, wherein a first policy rule specifies applying a first data reduction scheme to data in the computer memory based on the data matching first characteristics, wherein a second policy rule specifies applying a second data reduction scheme to data in the computer memory based on the data matching second characteristics, wherein the first data reduction scheme is different from the second data reduction scheme. Another aspect includes determining, by the policy engine, that first data in the computer memory matches the first characteristics, and that second data in the computer memory matches the second characteristics. Yet another aspect includes applying the first data reduction scheme to the first data, and applying the second data reduction scheme to the second data.

公开(公告)号：US20170344586A1

公开(公告)日：2017-11-30

申请号：US15167056

申请日：2016-05-27

Applicant: International Business Machines Corporation

Inventor： M. Corneliu Constantinescu ,  Ramani R. Routray ,  Kensworth C. Subratie

IPC: G06F17/30 ,  G06N7/00

CPC classification number: G06F16/215 ,  G06F16/1727 ,  G06F16/1748 ,  G06F16/9024 ,  G06N5/003

Abstract: Embodiments are provided for enhancing storage efficiency in a de-duplication enabled storage system. Metadata of a shared-nothing clustered file system is scanned, and a first state of the storage system is determined. One or more cores are located from the metadata. Each core includes a grouping of objects having a minimum coreness. An arrangement of the located cores is optimized to improve global de-duplication efficiency by evaluating the objects of each core, identifying respective nodes in the storage system to maintain each core for de-duplication efficiency based on the evaluation, and re-arranging one or more of the evaluated objects in the storage system.

公开(公告)号：US20170344578A1

公开(公告)日：2017-11-30

申请号：US15167277

申请日：2016-05-27

Applicant: International Business Machines Corporation

Inventor： M. Corneliu Constantinescu ,  Leo Shyh-Wei Luan ,  Wayne A. Sawdon ,  Frank B. Schmuck

IPC: G06F17/30

CPC classification number: G06F17/30153 ,  G06F17/30138

Abstract: The embodiments described herein relate to managing compressed data to optimize file compression for efficient random access to the data. A first partition of a first data block of a compression group is compressed. The first compressed partition is stored in a first compression entity. An in-memory table is maintained, which includes updating the in-memory table with data associated with an address of the stored compressed first partition. At such time as it is determined that the first compression entity is full, the in-memory table is compressed and written to the first compression entity. Accordingly, the in-memory table, which stores partition compression data, is store with the compression entity.

公开(公告)号：US20170177603A1

公开(公告)日：2017-06-22

申请号：US14970592

申请日：2015-12-16

Applicant: International Business Machines Corporation

Inventor： M. Corneliu Constantinescu ,  Leo Shyh-Wei Luan ,  Wayne A. Sawdon ,  Frank B. Schmuck

IPC: G06F17/30

CPC classification number: G06F16/1744

Abstract: The embodiments described herein relate to managing compressed data to optimize file compression. A first compression is performed on a first set of data to create first compressed data. The first compressed data is stored in one or more blocks of a first compression group. A size of free space of a last block of the first compression group is discovered and calculated. A second compression is performed on a second set of data to create second compressed data. At least a portion of the second compressed data is supplied to the first compression group for padding into the last block in response to determining that the size of the free space is sufficient. An unpadded portion of the second compressed data is stored in one or more blocks of a second compression group.

公开(公告)号：US10804930B2

公开(公告)日：2020-10-13

申请号：US14970636

申请日：2015-12-16

Applicant: International Business Machines Corporation

Inventor： M. Corneliu Constantinescu ,  Leo Shyh-Wei Luan ,  Wayne A. Sawdon ,  Frank B. Schmuck

IPC: H03M7/30

Abstract: The embodiments described herein relate to managing compressed data to optimize file compression. A compression is performed on a first set of data to create a first set of compressed data partitions in a compression group. A partition table is constructed, and partition entries for the first data block are added to the table in conjunction with the first set. A current size of the compression group is assessed. In response to a compression ratio being greater than a target compression ratio and internal fragmentation of the compression group being smaller than a threshold, the compression group is dynamically completed. The dynamic completion decides a size for the compression group. The partition table is added to the compression group by assigning space within the first compression group for the table. The compression group is written to persistent storage.

公开(公告)号：US20170177602A1

公开(公告)日：2017-06-22

申请号：US14970636

申请日：2015-12-16

Applicant: International Business Machines Corporation

Inventor： M. Corneliu Constantinescu ,  Leo Shyh-Wei Luan ,  Wayne A. Sawdon ,  Frank B. Schmuck

IPC: G06F17/30 ,  H03M7/30

CPC classification number: H03M7/3077

Abstract: The embodiments described herein relate to managing compressed data to optimize file compression. A compression is performed on a first set of data to create a first set of compressed data partitions in a compression group. A partition table is constructed, and partition entries for the first data block are added to the table in conjunction with the first set. A current size of the compression group is assessed. In response to a compression ratio being greater than a target compression ratio and internal fragmentation of the compression group being smaller than a threshold, the compression group is dynamically completed. The dynamic completion decides a size for the compression group. The partition table is added to the compression group by assigning space within the first compression group for the table. The compression group is written to persistent storage.

公开(公告)号：US20160034201A1

公开(公告)日：2016-02-04

申请号：US14450889

申请日：2014-08-04

Applicant: International Business Machines Corporation

Inventor： David D. Chambliss ,  M. Corneliu Constantinescu ,  Joseph S. Glider ,  Danny Harnik ,  Maohua Lu ,  David P. Woodruff

IPC: G06F3/06 ,  G06F12/10

CPC classification number: G06F3/0608 ,  G06F3/0641 ,  G06F3/0671 ,  G06F12/1018 ,  G06F16/1748 ,  G06F2212/1044 ,  G06F2212/65

Abstract: A protocol is employed to estimate duplication of data in a storage system. This estimate is employed as a factor of enabling de-duplication, and if de-duplication is enabled, the data sets which will be subject to the de-duplication. The protocol includes a measurement procedure and an execution procedure. The measurement procedure characterizes data duplication in part of the data on the storage system, and the execution procedure use the characterization to adjust selection of which data sets are subject to de-duplication.

Abstract translation: 采用协议来估计存储系统中数据的重复。 这种估算被用作启用重复数据删除的一个因素，如果重复数据删除功能被启用，将会被重复数据删除的数据集。 该协议包括测量过程和执行过程。 测量过程表征存储系统上部分数据的数据复制，执行过程使用表征来调整选择哪些数据集进行重复数据删除。

公开(公告)号：US20160004715A1

公开(公告)日：2016-01-07

申请号：US14321981

申请日：2014-07-02

Applicant: International Business Machines Corporation

Inventor： Jonathan Amit ,  David D. Chambliss ,  M. Corneliu Constantinescu

IPC: G06F17/30

CPC classification number: G06F3/0638 ,  G06F3/0608 ,  G06F3/0676

Abstract: Embodiments of the invention relate to compressed storage systems, and reducing metadata representing compressed data. Compressed data is stored in units referred to as partitions, with each partition having a header that contains a virtual address of data stored in the partition. A linear function is providing to represent a mapping between a virtual address segment and a compressed data extent, with a slope of the function representing an associated compression ratio. A read operation is supported by consulting the mapping and using the mapping to locate the corresponding compressed extent. Similarly, a write operation is supported by writing a new segment, compressing content in the segment, and computing a new mapping of the compressed segment metadata in memory. The new mapping is represented in the linear function.

Abstract translation: 本发明的实施例涉及压缩存储系统，并减少表示压缩数据的元数据。 压缩数据以被称为分区的单元存储，每个分区具有包含存储在分区中的数据的虚拟地址的报头。 提供线性函数来表示虚拟地址段和压缩数据范围之间的映射，其中该函数的斜率表示相关联的压缩比。 通过查看映射并使用映射来定位相应的压缩范围，支持读操作。 类似地，通过写入新段，压缩段中的内容以及计算存储器中的压缩段元数据的新映射来支持写入操作。 新的映射在线性函数中表示。