-
公开(公告)号:US10949397B1
公开(公告)日:2021-03-16
申请号:US14568002
申请日:2014-12-11
Applicant: Amazon Technologies, Inc.
Inventor: Jacob A. Strauss , Matteo Frigo
IPC: G06F16/176
Abstract: Techniques described and suggested include systems and processes for handling data and client lock management in multi-tenant storage systems, such as distributed storage systems. In some embodiments, data lock and client lock records are stored in connection with a plurality of data stores on which client data resides. In some embodiments, a storage hub operably connected to the data stores determines and retain client lease information. In some embodiments, the client lease information is used in conjunction with the data lock and client lock records to effect and determine the validity of locks on various data elements.
-
公开(公告)号:US09519510B2
公开(公告)日:2016-12-13
申请号:US14231077
申请日:2014-03-31
Applicant: Amazon Technologies, Inc.
Inventor: Jacob A. Strauss , Matteo Frigo , Alex Haugland , Matti Juhani Oikarinen , Pradeep Vincent , Joshua Samuel Hendrickson
Abstract: A node of a storage service is selected as a coordinator of a distributed transaction involving multiple page-level modifications. The coordinator identifies other nodes as members of a node chain collectively storing physical data pages at which proposed modifications are to be performed, including a decider node responsible for a decision to commit the transaction. The coordinator generates a transaction preparation message comprising a representation of an order of respective commit decisions associated with the proposed modifications, and transmits the message to a selected node of the chain for a sequential propagation along the chain. Each chain node performs a local commit analysis for its changes and stores a record of its intent to commit. If a decision to commit is reached at the decider, the proposed modifications are completed.
Abstract translation: 选择存储服务的节点作为涉及多个页面级修改的分布式事务的协调器。 协调器将其他节点标识为节点链的成员,共同存储将要执行所提出的修改的物理数据页,包括负责决定提交事务的决定节点。 协调器生成交易准备消息,其包括与所提出的修改相关联的各自提交决定的顺序的表示,并且将消息发送到链的所选节点,以沿着链的顺序传播。 每个链节点对其更改执行本地提交分析,并存储其提交意图的记录。 如果决定者达成了作出决定,则建议的修改完成。
-
3.
公开(公告)号:US09294558B1
公开(公告)日:2016-03-22
申请号:US14231047
申请日:2014-03-31
Applicant: Amazon Technologies, Inc.
Inventor: Pradeep Vincent , Matti Juhani Oikarinen , Douglas Stewart Laurence , Matteo Frigo
CPC classification number: H04L67/1006 , H04L67/1004 , H04L67/1097
Abstract: At a particular node of a storage service to which connections have been established on behalf of one or more clients, respective workload indicators are collected from a set of peer nodes of the storage service. A determination is made at the particular node that (a) a local workload metric exceeds a connection rebalancing threshold, and (b) a peer capacity availability criterion has been met. The peer capacity availability criterion may be determined from the respective workload indicators. In response to the determination, a particular client connection is closed.
Abstract translation: 在代表一个或多个客户端建立连接的存储服务的特定节点处,从存储服务的一组对等节点收集相应的工作负载指示符。 在特定节点处确定(a)本地工作负载度量超过连接重新平衡阈值,并且(b)已经满足了对等容量可用性标准。 可以从相应的工作负载指示符确定对等容量可用性标准。 响应于该确定,特定的客户端连接被关闭。
-
公开(公告)号:US09274710B1
公开(公告)日:2016-03-01
申请号:US14231063
申请日:2014-03-31
Applicant: Amazon Technologies, Inc.
Inventor: Matti Juhani Oikarinen , Matteo Frigo , Michael Anthony Nuss , Wayne William Duso
CPC classification number: G06F13/18 , G06F3/061 , G06F3/0659 , G06F3/067 , G06F13/1642 , G06F2206/1012
Abstract: An I/O request directed to a portion of a storage object managed at a distributed storage service is received. A congestion control parameter value to be used to schedule a storage operation corresponding to the I/O request is determined. The congestion control parameter is based at least in part on an offset within the storage object to which the I/O request is directed. The storage operation is scheduled in accordance with the congestion control parameter at a selected physical storage device to which the portion of the storage object is mapped.
Abstract translation: 接收针对分布式存储服务管理的存储对象的一部分的I / O请求。 确定用于调度与I / O请求对应的存储操作的拥塞控制参数值。 拥塞控制参数至少部分地基于I / O请求所针对的存储对象内的偏移量。 存储操作根据存储对象的部分映射到的所选物理存储设备的拥塞控制参数进行调度。
-
公开(公告)号:US10592344B1
公开(公告)日:2020-03-17
申请号:US15654065
申请日:2017-07-19
Applicant: Amazon Technologies, Inc.
Inventor: Bryan James Donlan , Matteo Frigo , Colin Laird Lazier
IPC: G06F17/30 , G06F11/14 , G06F11/10 , G06F16/182 , G06F16/10 , G06F16/188 , G06F16/2455
Abstract: Erasure encoded fragments are generated by an erasure encoding scheme, represented by an erasure encoding matrix, operating on a data file. A new erasure encoded fragment may be generated from previously-generated erasure encoded fragments without reconstructing the original data file. Available and valid erasure encoded fragments are identified and a set of those fragments is selected. A composite encoding matrix is generated based upon the selected fragments and the fragment specified to be generated. The composite matrix is applied to the selected fragments to produce a plurality of partial sums. The partial sums are then combined to generate the specified fragment. The partial sums may be produced by different devices so as to distribute the computational workload and/or to reduce network traffic. The integrity of a generated fragment may be verified by generating the specified fragment twice, using two different sets of fragments, and then comparing the two results.
-
公开(公告)号:US20180041572A1
公开(公告)日:2018-02-08
申请号:US15783857
申请日:2017-10-13
Applicant: Amazon Technologies, Inc.
Inventor: Michael Robert Frasca , Matteo Frigo , Jacob A. Strauss
CPC classification number: H04L67/1008 , H04L43/0876 , H04L67/1095 , H04L67/1097
Abstract: A computing resource service provider may operate a data storage service configured to provide data storage for one or more customers of the computing resource service provider. The data storage service may store customer data in one or more replicated state machines, where the replicated state machines comprise a plurality of replicated state machine-shards. The replicated state machine-shards may cause the computer system hosting the replicated state machine-shard to transmit a consensus message to other computer system. The consensus message may include utilization information corresponding to the other computer system. The utilization information may be used to calculate a utilization rate for the replicated state machine usable in block allocation operations.
-
公开(公告)号:US09753807B1
公开(公告)日:2017-09-05
申请号:US14307313
申请日:2014-06-17
Applicant: Amazon Technologies, Inc.
Inventor: Bryan James Donlan , Matteo Frigo , Colin Laird Lazier
CPC classification number: G06F11/1435 , G06F11/1076 , G06F17/30067 , G06F17/30206 , G06F17/30233 , G06F17/30516
Abstract: Erasure encoded fragments are generated by an erasure encoding scheme, represented by an erasure encoding matrix, operating on a data file. A new erasure encoded fragment may be generated from previously-generated erasure encoded fragments without reconstructing the original data file. Available and valid erasure encoded fragments are identified and a set of those fragments is selected. A composite encoding matrix is generated based upon the selected fragments and the fragment specified to be generated. The composite matrix is applied to the selected fragments to produce a plurality of partial sums. The partial sums are then combined to generate the specified fragment. The partial sums may be produced by different devices so as to distribute the computational workload and/or to reduce network traffic. The integrity of a generated fragment may be verified by generating the specified fragment twice, using two different sets of fragments, and then comparing the two results.
-
公开(公告)号:US09710407B2
公开(公告)日:2017-07-18
申请号:US15056894
申请日:2016-02-29
Applicant: Amazon Technologies, Inc.
Inventor: Matti Juhani Oikarinen , Matteo Frigo , Michael Anthony Nuss , Wayne William Duso
CPC classification number: G06F13/18 , G06F3/061 , G06F3/0659 , G06F3/067 , G06F13/1642 , G06F2206/1012
Abstract: An I/O request directed to a portion of a storage object managed at a distributed storage service is received. A congestion control parameter value to be used to schedule a storage operation corresponding to the I/O request is determined. The congestion control parameter is based at least in part on an offset within the storage object to which the I/O request is directed. The storage operation is scheduled in accordance with the congestion control parameter at a selected physical storage device to which the portion of the storage object is mapped.
-
公开(公告)号:US09794331B1
公开(公告)日:2017-10-17
申请号:US14500435
申请日:2014-09-29
Applicant: Amazon Technologies, Inc.
Inventor: Michael Robert Frasca , Matteo Frigo , Jacob A. Strauss
IPC: G06F15/173 , H04L29/08 , H04L12/26
CPC classification number: H04L67/1008 , H04L43/0876 , H04L67/1095 , H04L67/1097
Abstract: A computing resource service provider may operate a data storage service configured to provide data storage for one or more customers of the computing resource service provider. The data storage service may store customer data in one or more replicated state machines, where the replicated state machines comprise a plurality of replicated state machines-shards. The replicated state machines-shards may cause the computer system hosting the replicated state machine-shard to transmit a consensus message to other computer system. The consensus message may include utilization information corresponding to the other computer system. The utilization information may be used to calculate a utilization rate for the replicated state machine useable in block allocation operations.
-
公开(公告)号:US09495478B2
公开(公告)日:2016-11-15
申请号:US14231095
申请日:2014-03-31
Applicant: Amazon Technologies, Inc.
Inventor: Joshua Samuel Hendrickson , Matti Juhani Oikarinen , Alex Haugland , Pradeep Vincent , Matteo Frigo , Xiaobin Wu
IPC: G06F17/30
CPC classification number: G06F17/30958 , G06F17/30097
Abstract: A directed acyclic graph (DAG) is generated to represent a namespace of a directory. In response to a request to create a new object with a specified name, a hash value bit sequence is computed for the name. A plurality of levels of the DAG are navigated using successive subsequences of the bit sequence to identify a candidate node for storing a new entry corresponding to the specified name. If the candidate node meets a split criterion, the new entry and at least a selected subset of entries of the candidate node's list of entries are distributed among a plurality of DAG nodes, including at least one new DAG node, using respective bit sequences obtained by applying the hash function for each distributed entry.
Abstract translation: 生成有向无环图(DAG)来表示目录的命名空间。 响应于创建具有指定名称的新对象的请求,为该名称计算哈希值比特序列。 使用比特序列的连续子序列导航多个DAG级别,以识别用于存储与指定名称相对应的新条目的候选节点。 如果候选节点满足分割标准,则候选节点的条目列表的新条目和至少一个选定的条目子集在包括至少一个新的DAG节点的多个DAG节点之间分配,使用由 对每个分布式条目应用散列函数。
-
-
-
-
-
-
-
-
-