公开(公告)号：US10210082B2

公开(公告)日：2019-02-19

申请号：US15581800

申请日：2017-04-28

申请人： NetApp, Inc.

发明人： Dhaval Patel ,  Manish Swaminathan ,  Edward D. McClanahan ,  John Muth

IPC分类号： G06F12/02 ,  G06F3/06

摘要： A rate matching technique may be configured to adjust a rate of cleaning of one or more selected segments of the storage array to accommodate a variable rate of incoming workload processed by a storage input/output (I/O) stack executing on one or more nodes of a cluster. An extent store layer of the storage I/O stack may clean a segment in accordance with segment cleaning which, illustratively, may be embodied as a segment cleaning process. The rate matching technique may be implemented as a feedback control mechanism configured to adjust the segment cleaning process based on the incoming workload. Components of the feedback control mechanism may include one or more weight schedulers and various accounting data structures, e.g., counters, configured to track the progress of segment cleaning and free space usage. The counters may also be used to balance the rates of segment cleaning and incoming I/O workload, which may change depending upon an incoming I/O rate. When the incoming I/O rate changes, the rate of segment cleaning may be adjusted accordingly to ensure that rates are substantially balanced.

公开(公告)号：US10049118B2

公开(公告)日：2018-08-14

申请号：US14727005

申请日：2015-06-01

申请人： NetApp, Inc.

发明人： Dhaval Patel ,  Chaitanya Patel ,  John Muth ,  Srinath Krishnamachari

IPC分类号： G06F17/30

摘要： A cluster-wide consistency checker ensures that two file systems of a storage input/output (I/O) stack executing on each node of a cluster are self-consistent as well as consistent with respect to each other. The file systems include a deduplication file system and a host-facing file system that cooperate to provide a layered file system of the storage I/O stack. The deduplication file system is a log-structured file system managed by an extent store layer of the storage I/O stack, whereas the host-facing file system is managed by a volume layer of the stack. Illustratively, each log-structured file system implements a key-value store and cooperates with other nodes of the cluster to provide a cluster-wide (global) key-value store. The consistency checker verifies and/or fixes on-disk structures of the layered file system to ensure its consistency. To that end, the consistency checker may determine whether there are inconsistencies in the key-value store and, if so, reconciles those inconsistencies from a client (volume layer) perspective.

公开(公告)号：US09671960B2

公开(公告)日：2017-06-06

申请号：US14484565

申请日：2014-09-12

申请人： NetApp, Inc.

发明人： Dhaval Patel ,  Manish Swaminathan ,  Edward D. McClanahan ,  John Muth

IPC分类号： G06F3/06 ,  G06F12/02

CPC分类号： G06F12/0253 ,  G06F3/0608 ,  G06F3/061 ,  G06F3/0619 ,  G06F3/0641 ,  G06F3/0656 ,  G06F3/0688 ,  G06F12/0246 ,  G06F2212/1016 ,  G06F2212/1024 ,  G06F2212/1044 ,  G06F2212/702 ,  G06F2212/7201 ,  G06F2212/7205

摘要： A rate matching technique may be configured to adjust a rate of cleaning of one or more selected segments of the storage array to accommodate a variable rate of incoming workload processed by a storage input/output (I/O) stack executing on one or more nodes of a cluster. An extent store layer of the storage I/O stack may clean a segment in accordance with segment cleaning which, illustratively, may be embodied as a segment cleaning process. The rate matching technique may be implemented as a feedback control mechanism configured to adjust the segment cleaning process based on the incoming workload. Components of the feedback control mechanism may include one or more weight schedulers and various accounting data structures, e.g., counters, configured to track the progress of segment cleaning and free space usage. The counters may also be used to balance the rates of segment cleaning and incoming I/O workload, which may change depending upon an incoming I/O rate. When the incoming I/O rate changes, the rate of segment cleaning may be adjusted accordingly to ensure that rates are substantially balanced.

公开(公告)号：US20180074924A1

公开(公告)日：2018-03-15

申请号：US15813941

申请日：2017-11-15

申请人： NetApp, Inc.

发明人： Bob Schatz ,  John Muth ,  Stephen H. Strange

IPC分类号： G06F11/20

CPC分类号： G06F11/2033 ,  G06F11/1425 ,  G06F11/2046 ,  G06F2201/805

摘要： A third vote consensus technique enables a first node, i.e., a surviving node, of a two-node cluster to establish a quorum and continue to operate in response to failure of a second node of the cluster. Each node maintains configuration information organized as a cluster database (CDB) which may be changed according to a consensus-based protocol. Changes to the CDB are logged on a third copy file system (TCFS) stored on a local copy of TCFS (L-TCFS). A shared copy of the TCFS (i.e., S-TCFS) may be stored on shared storage devices of one or more storage arrays coupled to the nodes. The local copy of the TCFS (i.e., L-TCFS) represents a quorum vote for each node of the cluster, while the S-TCFS represents an additional “tie-breaker” vote of a consensus-based protocol. The additional vote may be obtained from the shared storage devices by the surviving node as a third vote to establish the quorum and enable the surviving node to cast two of three votes (i.e., a majority of votes) needed to continue operation of the cluster. That is, the majority of votes allows the surviving node to update the CDB with the configuration information changes so as to continue proper operation of the cluster.

公开(公告)号：US09836366B2

公开(公告)日：2017-12-05

申请号：US14924318

申请日：2015-10-27

申请人： NetApp, Inc.

发明人： Bob Schatz ,  John Muth ,  Stephen H. Strange

IPC分类号： G06F11/00 ,  G06F11/20

CPC分类号： G06F11/2033 ,  G06F11/1425 ,  G06F11/2046 ,  G06F2201/805

摘要： A third vote consensus technique enables a first node, i.e., a surviving node, of a two-node cluster to establish a quorum and continue to operate in response to failure of a second node of the cluster. Each node maintains configuration information organized as a cluster database (CDB) which may be changed according to a consensus-based protocol. Changes to the CDB are logged on a third copy file system (TCFS) stored on a local copy of TCFS (L-TCFS). A shared copy of the TCFS (i.e., S-TCFS) may be stored on shared storage devices of one or more storage arrays coupled to the nodes. The local copy of the TCFS (i.e., L-TCFS) represents a quorum vote for each node of the cluster, while the S-TCFS represents an additional “tie-breaker” vote of a consensus-based protocol. The additional vote may be obtained from the shared storage devices by the surviving node as a third vote to establish the quorum and enable the surviving node to cast two of three votes (i.e., a majority of votes) needed to continue operation of the cluster. That is, the majority of votes allows the surviving node to update the CDB with the configuration information changes so as to continue proper operation of the cluster.

公开(公告)号：US20170116095A1

公开(公告)日：2017-04-27

申请号：US14924318

申请日：2015-10-27

申请人： NetApp, Inc.

发明人： Bob Schatz ,  John Muth ,  Stephen H. Strange

IPC分类号： G06F11/20

CPC分类号： G06F11/2033 ,  G06F11/1425 ,  G06F11/2046 ,  G06F2201/805

摘要： A third vote consensus technique enables a first node, i.e., a surviving node, of a two-node cluster to establish a quorum and continue to operate in response to failure of a second node of the cluster. Each node maintains configuration information organized as a cluster database (CDB) which may be changed according to a consensus-based protocol. Changes to the CDB are logged on a third copy file system (TCFS) stored on a local copy of TCFS (L-TCFS). A shared copy of the TCFS (i.e., S-TCFS) may be stored on shared storage devices of one or more storage arrays coupled to the nodes. The local copy of the TCFS (i.e., L-TCFS) represents a quorum vote for each node of the cluster, while the S-TCFS represents an additional “tie-breaker” vote of a consensus-based protocol. The additional vote may be obtained from the shared storage devices by the surviving node as a third vote to establish the quorum and enable the surviving node to cast two of three votes (i.e., a majority of votes) needed to continue operation of the cluster. That is, the majority of votes allows the surviving node to update the CDB with the configuration information changes so as to continue proper operation of the cluster.

公开(公告)号：US20160077745A1

公开(公告)日：2016-03-17

申请号：US14484565

申请日：2014-09-12

申请人： NetApp, Inc.

发明人： Dhaval Patel ,  Manish Swaminathan ,  Edward D. McClanahan ,  John Muth

IPC分类号： G06F3/06 ,  G06F12/02

CPC分类号： G06F12/0253 ,  G06F3/0608 ,  G06F3/061 ,  G06F3/0619 ,  G06F3/0641 ,  G06F3/0656 ,  G06F3/0688 ,  G06F12/0246 ,  G06F2212/1016 ,  G06F2212/1024 ,  G06F2212/1044 ,  G06F2212/702 ,  G06F2212/7201 ,  G06F2212/7205

摘要： A rate matching technique may be configured to adjust a rate of cleaning of one or more selected segments of the storage array to accommodate a variable rate of incoming workload processed by a storage input/output (I/O) stack executing on one or more nodes of a cluster. An extent store layer of the storage I/O stack may clean a segment in accordance with segment cleaning which, illustratively, may be embodied as a segment cleaning process. The rate matching technique may be implemented as a feedback control mechanism configured to adjust the segment cleaning process based on the incoming workload. Components of the feedback control mechanism may include one or more weight schedulers and various accounting data structures, e.g., counters, configured to track the progress of segment cleaning and free space usage. The counters may also be used to balance the rates of segment cleaning and incoming I/O workload, which may change depending upon an incoming I/O rate. When the incoming I/O rate changes, the rate of segment cleaning may be adjusted accordingly to ensure that rates are substantially balanced.

摘要翻译： 速率匹配技术可以被配置为调整存储阵列的一个或多个所选段的清理速率，以适应由在一个或多个节点上执行的存储输入/输出（I / O）栈处理的进入工作负载的可变速率 的集群。 存储I / O堆栈的盘区存储层可以根据段清洁来清洁段，其示例性地可以被实现为段清理过程。 速率匹配技术可以被实现为反馈控制机制，其被配置为基于输入的工作负载来调整段清除过程。 反馈控制机构的组件可以包括一个或多个权重调度器和各种会计数据结构，例如计数器，其被配置为跟踪段清洁和可用空间使用的进度。 这些计数器也可用于平衡段清除和输入I / O工作负载的速率，这可能会根据传入的I / O速率而改变。 当进入的I / O速率变化时，可以相应地调节段清洁的速率，以确保速率基本平衡。

公开(公告)号：US10133511B2

公开(公告)日：2018-11-20

申请号：US14484820

申请日：2014-09-12

申请人： NetApp, Inc.

发明人： John Muth ,  Edward D. McClanahan ,  Dhaval Patel ,  Manish Swaminathan

IPC分类号： G06F3/06

摘要： An optimized segment cleaning technique is configured to efficiently clean one or more selected portions or segments of a storage array coupled to one or more nodes of a cluster. A bottom-up approach of the segment cleaning technique is configured to read all blocks of a segment to be cleaned (i.e., an “old” segment) to locate extents stored on the SSDs of the old segment and examine extent metadata to determine whether the extents are valid and, if so, relocate the valid extents to a segment being written (i.e., a “new” segment). A top-down approach of the segment cleaning technique obviates reading of the blocks of the old segment to locate the extents and, instead, examines the extent metadata to determine the valid extents of the old segment. A hybrid approach may extend the top-down approach to include only full stripe read operations needed for relocation and reconstruction of blocks as well as retrieval of valid extents from the stripes, while also avoiding any unnecessary read operations of the bottom-down approach.

公开(公告)号：US20160350358A1

公开(公告)日：2016-12-01

申请号：US14727005

申请日：2015-06-01

申请人： NetApp, Inc.

发明人： Dhaval Patel ,  Chaitanya Patel ,  John Muth ,  Srinath Krishnamachari

IPC分类号： G06F17/30

CPC分类号： G06F17/30156

摘要： A cluster-wide consistency checker ensures that two file systems of a storage input/output (I/O) stack executing on each node of a cluster are self-consistent as well as consistent with respect to each other. The file systems include a deduplication file system and a host-facing file system that cooperate to provide a layered file system of the storage I/O stack. The deduplication file system is a log-structured file system managed by an extent store layer of the storage I/O stack, whereas the host-facing file system is managed by a volume layer of the stack. Illustratively, each log-structured file system implements a key-value store and cooperates with other nodes of the cluster to provide a cluster-wide (global) key-value store. The consistency checker verifies and/or fixes on-disk structures of the layered file system to ensure its consistency. To that end, the consistency checker may determine whether there are inconsistencies in the key-value store and, if so, reconciles those inconsistencies from a client (volume layer) perspective.

摘要翻译： 集群范围的一致性检查器确保在集群的每个节点上执行的存储输入/输出（I / O）堆栈的两个文件系统是自相矛盾的，并且彼此一致。 文件系统包括重复数据删除文件系统和协作提供存储I / O堆栈的分层文件系统的面向主机的文件系统。 重复数据删除文件系统是由存储I / O堆栈的扩展区存储层管理的日志结构文件系统，而面向主机的文件系统由堆栈的卷层管理。 说明性地，每个日志结构的文件系统实现一个键值存储，并与集群的其他节点协作，以提供一个集群范围的（全局）键值存储。 一致性检查器验证和/或修复分层文件系统的磁盘结构，以确保其一致性。 为此，一致性检查器可以确定密钥值存储中是否存在不一致，如果是，则将这些不一致性与客户端（卷层）的角度进行对照。

公开(公告)号：US20160077746A1

公开(公告)日：2016-03-17

申请号：US14484820

申请日：2014-09-12

申请人： NetApp, Inc.

发明人： John Muth ,  Edward D. McClanahan ,  Dhaval Patel ,  Manish Swaminathan

IPC分类号： G06F3/06

CPC分类号： G06F3/0652 ,  G06F3/0608 ,  G06F3/061 ,  G06F3/0644 ,  G06F3/0673 ,  G06F3/0688 ,  G06F2212/7205

摘要： An optimized segment cleaning technique is configured to efficiently clean one or more selected portions or segments of a storage array coupled to one or more nodes of a cluster. A bottom-up approach of the segment cleaning technique is configured to read all blocks of a segment to be cleaned (i.e., an “old” segment) to locate extents stored on the SSDs of the old segment and examine extent metadata to determine whether the extents are valid and, if so, relocate the valid extents to a segment being written (i.e., a “new” segment). A top-down approach of the segment cleaning technique obviates reading of the blocks of the old segment to locate the extents and, instead, examines the extent metadata to determine the valid extents of the old segment. A hybrid approach may extend the top-down approach to include only full stripe read operations needed for relocation and reconstruction of blocks as well as retrieval of valid extents from the stripes, while also avoiding any unnecessary read operations of the bottom-down approach.

摘要翻译： 经优化的段清洁技术被配置为有效地清洁耦合到群集的一个或多个节点的存储阵列的一个或多个选定部分或段。 分段清洁技术的自下而上的方法被配置为读取待清理的段的所有块（即，“旧”段））以定位存储在旧段的SSD上的区段，并检查扩展元数据以确定是否 范围是有效的，如果是，将有效范围重定位到正在写入的段（即“新”段）。 段清除技术的自上而下的方法避免了读取旧段的块以定位范围，而是检查元数据的大小以确定旧段的有效范围。 混合方法可以扩展自上而下的方法，以仅包括块的重定位和重建所需的全条带读取操作以及从条带检索有效范围，同时还避免了自下而上的方法的任何不必要的读操作。