公开(公告)号：US08683112B2

公开(公告)日：2014-03-25

申请号：US13907531

申请日：2013-05-31

Applicant: Google Inc.

Inventor： Alexandre Drobychev ,  Alexander Kesselman

IPC: G06F13/00

CPC classification number: G06F17/30578 ,  G06F17/30091 ,  G06F17/30212

Abstract: A method performed by two or more devices of a group of devices in a distributed data replication system may include receiving a group of chunks having a same unique temporary identifier, the group of chunks comprising an object to be uploaded; creating an entry for the object in a replicated index, the entry being keyed by the unique temporary identifier, and the replicated index being replicated at each of the two or more devices; and determining, by an initiating device of the two or more devices, that a union of the group of chunks contains all data of the object. The method may also include calculating a content-based identifier to the object; creating another entry for the object in the replicated index, the other entry being keyed by the content-based identifier; and updating the replicated index to point from the unique temporary identifier to the content-based identifier.

Abstract translation: 分布式数据复制系统中的一组设备的两个或多个设备执行的方法可以包括接收具有相同唯一临时标识符的一组块，该组块包括要上传的对象; 在复制索引中创建对象的条目，该条目由唯一临时标识符键入，并且复制索引被复制在两个或更多个设备中的每一个上; 以及由所述两个或多个设备的发起设备确定所述一组组合的联合包含所述对象的所有数据。 该方法还可以包括：向对象计算基于内容的标识符; 为复制索引中的对象创建另一个条目，另一个条目由基于内容的标识符键入; 并将所复制的索引更新为从唯一临时标识符指向基于内容的标识符。

公开(公告)号：US20140032200A1

公开(公告)日：2014-01-30

申请号：US14037298

申请日：2013-09-25

Applicant: Google Inc.

Inventor： Yonatan Zunger ,  Alexandre Drobychev ,  Alexander Kesselman ,  Rebekah C. Vickrey ,  Frank Clare Dachille ,  George Datuashvili

IPC: G06G7/62

CPC classification number: G06F17/30215 ,  G06F11/2094 ,  G06F11/3457 ,  H04L67/1095 ,  H04L67/1097

Abstract: A distributed storage system has a plurality of instances. A computer system simulates the state of the distributed storage system. The system obtains a current state of the distributed storage system and replication policies for objects in the distributed storage system. Each replication policy specifies criteria for placing copies of the relevant objects among the plurality of instances. The system receives proposed modifications to the state of the distributed storage system and simulates the state of the distributed storage system over time based on the current state of the distributed storage system, current statistical trends in the state of the distributed storage system, the replication policies for the objects in the distributed storage system, and the proposed modifications to the state of the distributed storage system. One or more reports are generated relating to time evolution of the state of the distributed storage system based on the simulation.

Abstract translation: 分布式存储系统具有多个实例。 计算机系统模拟分布式存储系统的状态。 系统获取分布式存储系统的当前状态和分布式存储系统中对象的复制策略。 每个复制策略指定在多个实例中放置相关对象的副本的标准。 系统接收对分布式存储系统状态的建议修改，并根据分布式存储系统的当前状态，分布式存储系统状态下的当前统计趋势，复制策略，模拟分布式存储系统随时间的状态 对于分布式存储系统中的对象，以及对分布式存储系统状态的建议修改。 基于模拟，生成与分布式存储系统的状态的时间演化相关的一个或多个报告。

公开(公告)号：US09330107B1

公开(公告)日：2016-05-03

申请号：US13761115

申请日：2013-02-06

Applicant: Google Inc.

Inventor： Alexander Kesselman ,  Angus Lees ,  Dmitry Broyde ,  Michael O'Reilly

IPC: G06F17/30

CPC classification number: G06F17/30194

Abstract: A system, computer-readable storage medium storing at least one program, and a computer-implemented method for storing metadata for a file in a distributed storage system is presented. Metadata for a file is generated. A subset of metadata clusters in a distributed storage system is identified to form a quorum of metadata clusters. Requests to store the metadata for the file on the metadata clusters in the distributed storage system are issued, where the requests include high priority requests to store the metadata for the file on at least one of the metadata clusters in the quorum of metadata clusters and normal priority requests to store the metadata in the remaining metadata clusters, the metadata being stored on the at least one of the metadata clusters in the quorum of metadata clusters before being stored on the remaining metadata clusters.

Abstract translation: 提出了存储至少一个程序的系统，计算机可读存储介质和用于存储分布式存储系统中的文件的元数据的计算机实现的方法。 生成文件的元数据。 识别分布式存储系统中的元数据集群的子集，以形成元数据集群的数量。 发布将文件的元数据存储在分布式存储系统中的元数据集群上的请求，其中请求包括将文件的元数据存储在元数据集群的正常数量的仲裁中的至少一个元数据集群上的高优先级请求 优先权请求将元数据存储在剩余的元数据集群中，元数据被存储在元数据集群的数量集中的元数据集群中的至少一个之前被存储在剩余的元数据集群上。

公开(公告)号：US09158472B2

公开(公告)日：2015-10-13

申请号：US14142706

申请日：2013-12-27

Applicant: Google Inc.

Inventor： Alexander Kesselman ,  Michael O'Reilly ,  George Datuashvili ,  Alexandre Drobychev

IPC: G06F12/00 ,  G06F3/06

CPC classification number: G06F17/30575 ,  G06F3/0614 ,  G06F3/065 ,  G06F3/067 ,  G06F17/30321 ,  G06F17/30342

Abstract: Placement of object replicas in a distributed storage system includes, at a first instance, opening a journal for storage of object chunks. Each journal is associated with a single placement policy. An object is received, which comprises a chunk. The object has a placement policy, and the chunk comprises a plurality of storage blocks. The blocks are stored in a journal that matches the placement policy. Global metadata for the object is stored, which includes a list of chunks for the object. Local metadata for the chunk is stored, which includes a block list identifying each block of the plurality of blocks. The local metadata is associated with the journal. The journal is later closed. The journal is subsequently replicated to a second instance according to the placement policy. The global metadata is updated to reflect the replication, whereas the local metadata is unchanged by the replication.

Abstract translation: 在分布式存储系统中放置对象副本包括首先打开用于存储对象块的日志。 每个日记帐都与单一的刊登位置政策相关联。 接收到一个包含块的对象。 对象具有放置策略，并且该块包括多个存储块。 这些区块存储在与展示位置策略匹配的日记帐中。 存储对象的全局元数据，其中包含对象的块列表。 存储块的本地元数据，其包括标识多个块的每个块的块列表。 本地元数据与日记相关联。 该杂志以后关闭。 该日记随后根据放置政策复制到第二个实例。 全局元数据被更新以反映复制，而复制的本地元数据不变。

公开(公告)号：US20140304240A1

公开(公告)日：2014-10-09

申请号：US14293966

申请日：2014-06-02

Applicant: Google Inc.

Inventor： Yonatan Zunger ,  Alexandre Drobychev ,  Alexander Kesselman ,  Rebekah C. Vickrey ,  Frank C. Dachille ,  George Datuashvili

IPC: G06F17/30

CPC classification number: G06F17/30215 ,  G06F11/2094 ,  G06F11/3457 ,  H04L67/1095 ,  H04L67/1097

Abstract: A method allocates object replicas in a distributed storage system. The method identifies a plurality of objects in the distributed storage system. Each object has an associated storage policy that specifies a target number of object replicas stored at distinct instances of the distributed storage system. The method identifies an object of the plurality of objects whose number of object replicas exceeds the target number of object replicas specified by the storage policy associated with the object. The method selects a first replica of the object for removal based on last access times for replicas of the object, and transmits a request to a first instance of the distributed storage system that stores the first replica. The request instructs the first instance to remove the first replica of the object.

Abstract translation: 一种方法在分布式存储系统中分配对象副本。 该方法识别分布式存储系统中的多个对象。 每个对象具有关联的存储策略，其指定存储在分布式存储系统的不同实例处的对象副本的目标数量。 该方法识别多个对象的对象，其对象副本的数量超过与对象相关联的存储策略指定的对象副本的目标数量。 该方法基于对象的副本的最后访问时间选择要删除的对象的第一副本，并将请求发送到存储第一副本的分布式存储系统的第一实例。 请求指示第一个实例删除对象的第一个副本。