公开(公告)号：US11979336B1

公开(公告)日：2024-05-07

申请号：US18078289

申请日：2022-12-09

Applicant: Google LLC

Inventor： Priyanka Gupta ,  Sean Quinlan ,  Lawrence E. Greenfield

IPC: G06F9/46 ,  G06F9/50 ,  H04L47/70 ,  H04L47/762 ,  H04L12/28 ,  H04L12/911

CPC classification number: H04L47/762 ,  H04L47/821

Abstract: The present disclosure relates to dynamically scheduling resource requests in a distributed system based on usage quotas. One example method includes identifying usage information for a distributed system including atoms, each atom representing a distinct item used by users of the distributed system; determining that a usage quota associated with the distributed system has been exceeded based on the usage information, the usage quota representing an upper limit for a particular type of usage of the distributed system; receiving a first request for a particular atom requiring invocation of the particular type of usage represented by the usage quota; determining that a second request for a different type of usage of the particular atom is waiting to be processed; and processing the second request for the particular atom before processing the first request.

公开(公告)号：US11789938B1

公开(公告)日：2023-10-17

申请号：US17875780

申请日：2022-07-28

Applicant: Google LLC

Inventor： Wilson Cheng-Yi Hsieh ,  Alexander Lloyd ,  Peter Hochschild ,  Michael James Boyer Epstein ,  Sean Quinlan

IPC: G06F16/23 ,  G06F16/14 ,  G06F9/46 ,  G06F16/2458

CPC classification number: G06F16/2379 ,  G06F9/466 ,  G06F16/148 ,  G06F16/2322 ,  G06F16/2477

Abstract: The present technology proposes techniques for ensuring globally consistent transactions. This technology may allow distributed systems to ensure the causal order of read and write transactions across different partitions of a distributed database. By assigning causally generated timestamps to the transactions based on one or more globally coherent time services, the timestamps can be used to preserve and represent the causal order of the transactions in the distributed system. In this regard, certain transactions may wait for a period of time after choosing a timestamp in order to delay the start of any second transaction that might depend on it. The wait may ensure that the effects of the first transaction are not made visible until its timestamp is guaranteed to be in the past. This may ensure that a consistent snapshot of the distributed database can be determined for any past timestamp.

公开(公告)号：US20230236935A1

公开(公告)日：2023-07-27

申请号：US18191371

申请日：2023-03-28

Applicant: Google LLC

Inventor： Robert Cypher ,  Sean Quinlan ,  Steven Robert Schirripa

IPC: G06F11/14 ,  G06F16/182 ,  G06F16/27 ,  G06F16/174

CPC classification number: G06F11/1435 ,  G06F16/182 ,  G06F16/278 ,  G06F16/1748

Abstract: A method of distributing data in a distributed storage system includes receiving a file, dividing the received file into chunks, and determining a distribution of the chunks among storage devices of the distributed storage system based on a maintenance hierarchy of the distributed storage system. The maintenance hierarchy includes maintenance levels, and each maintenance level includes one or more maintenance units. Each maintenance unit has an active state and an inactive state. Moreover, each storage device is associated with a maintenance unit. The determining of the distribution of the chunks includes identifying a random selection of the storage devices matching a number of chunks of the file and being capable of maintaining accessibility of the file when one or more maintenance units are in an inactive state. The method also includes distributing the chunks to storage devices of the distributed storage system according to the determined distribution.

公开(公告)号：US11539636B1

公开(公告)日：2022-12-27

申请号：US17181601

申请日：2021-02-22

Applicant: Google LLC

Inventor： Lawrence E. Greenfield ,  Sean Quinlan ,  Priyanka Gupta

IPC: H04L1/18 ,  H04L27/26 ,  H04L12/26 ,  H04L47/762 ,  H04L47/70 ,  H04W28/16

Abstract: The present disclosure relates to dynamically scheduling resource requests in a distributed system based on usage quotas. One example method includes identifying usage information for a distributed system including atoms, each atom representing a distinct item used by users of the distributed system; determining that a usage quota associated with the distributed system has been exceeded based on the usage information, the usage quota representing an upper limit for a particular type of usage of the distributed system; receiving a first request for a particular atom requiring invocation of the particular type of usage represented by the usage quota; determining that a second request for a different type of usage of the particular atom is waiting to be processed; and processing the second request for the particular atom before processing the first request.

公开(公告)号：US11442925B1

公开(公告)日：2022-09-13

申请号：US16023326

申请日：2018-06-29

Applicant: Google LLC

Inventor： Wilson Cheng-Yi Hsieh ,  Alexander Lloyd ,  Peter Hochschild ,  Michael James Boyer Epstein ,  Sean Quinlan

IPC: G06F16/23 ,  G06F16/14 ,  G06F9/46 ,  G06F16/2458

Abstract: The present technology proposes techniques for ensuring globally consistent transactions. This technology may allow distributed systems to ensure the causal order of read and write transactions across different partitions of a distributed database. By assigning causally generated timestamps to the transactions based on one or more globally coherent time services, the timestamps can be used to preserve and represent the causal order of the transactions in the distributed system. In this regard, certain transactions may wait for a period of time after choosing a timestamp in order to delay the start of any second transaction that might depend on it. The wait may ensure that the effects of the first transaction are not made visible until its timestamp is guaranteed to be in the past. This may ensure that a consistent snapshot of the distributed database can be determined for any past timestamp.

公开(公告)号：US10318384B2

公开(公告)日：2019-06-11

申请号：US15180896

申请日：2016-06-13

Applicant: Google LLC

Inventor： Robert Cypher ,  Sean Quinlan ,  Steven Robert Schirripa

IPC: H04L12/00 ,  G06F13/00 ,  G06F11/14 ,  G06F16/182 ,  G06F16/27 ,  G06F16/174

Abstract: A method of distributing data in a distributed storage system includes receiving a file, dividing the received file into chunks, and determining a distribution of the chunks among storage devices of the distributed storage system based on a maintenance hierarchy of the distributed storage system. The maintenance hierarchy includes maintenance levels, and each maintenance level includes one or more maintenance units. Each maintenance unit has an active state and an inactive state. Moreover, each storage device is associated with a maintenance unit. The determining of the distribution of the chunks includes identifying a random selection of the storage devices matching a number of chunks of the file and being capable of maintaining accessibility of the file when one or more maintenance units are in an inactive state. The method also includes distributing the chunks to storage devices of the distributed storage system according to the determined distribution.

公开(公告)号：US11275743B2

公开(公告)日：2022-03-15

申请号：US15799939

申请日：2017-10-31

Applicant: Google LLC

Inventor： Robert C. Pike ,  Sean Quinlan ,  Sean M. Dorward ,  Jeffrey Dean ,  Sanjay Ghemawat

IPC: G06F16/18 ,  G06F16/2455 ,  G06F16/28 ,  G06F16/2458 ,  G06F11/14

Abstract: Systems and methods for analyzing input data records are provided in which a master process initiates a plurality of concurrent first processes each of which comprises, for each data record in at least a subset of a plurality of input data records, creating a parsed representation of the data record and independently applying a procedural language query to the parsed representation to extract one or more values. A respective emit operator is applied to at least one of the extracted one or more values thereby adding corresponding information to a respective intermediate data structure. The respective emit operator implements one of a predefined set of statistical information processing functions. The master process also initiates a plurality of second processes each of which aggregates information from a corresponding subset of intermediate data structures to produce aggregated data that is, in turn, combined to produce output data.

公开(公告)号：US11113150B2

公开(公告)日：2021-09-07

申请号：US16880513

申请日：2020-05-21

Applicant: Google LLC

Inventor： Robert Cypher ,  Sean Quinlan ,  Steven Robert Schirripa

IPC: H04L12/00 ,  G06F11/14 ,  G06F16/182 ,  G06F16/27 ,  G06F16/174

Abstract: A method of distributing data in a distributed storage system includes receiving a file, dividing the received file into chunks, and determining a distribution of the chunks among storage devices of the distributed storage system based on a maintenance hierarchy of the distributed storage system. The maintenance hierarchy includes maintenance levels, and each maintenance level includes one or more maintenance units. Each maintenance unit has an active state and an inactive state. Moreover, each storage device is associated with a maintenance unit. The determining of the distribution of the chunks includes identifying a random selection of the storage devices matching a number of chunks of the file and being capable of maintaining accessibility of the file when one or more maintenance units are in an inactive state. The method also includes distributing the chunks to storage devices of the distributed storage system according to the determined distribution.

公开(公告)号：US20200285541A1

公开(公告)日：2020-09-10

申请号：US16880513

申请日：2020-05-21

Applicant: Google LLC

Inventor： Robert Cypher ,  Sean Quinlan ,  Steven Robert Schirripa

IPC: G06F11/14 ,  G06F16/182 ,  G06F16/27 ,  G06F16/174

Abstract: A method of distributing data in a distributed storage system includes receiving a file, dividing the received file into chunks, and determining a distribution of the chunks among storage devices of the distributed storage system based on a maintenance hierarchy of the distributed storage system. The maintenance hierarchy includes maintenance levels, and each maintenance level includes one or more maintenance units. Each maintenance unit has an active state and an inactive state. Moreover, each storage device is associated with a maintenance unit. The determining of the distribution of the chunks includes identifying a random selection of the storage devices matching a number of chunks of the file and being capable of maintaining accessibility of the file when one or more maintenance units are in an inactive state. The method also includes distributing the chunks to storage devices of the distributed storage system according to the determined distribution.

公开(公告)号：US10678647B2

公开(公告)日：2020-06-09

申请号：US16392904

申请日：2019-04-24

Applicant: Google LLC

Inventor： Robert Cypher ,  Sean Quinlan ,  Steven Robert Schirripa

IPC: H04L12/00 ,  G06F17/00 ,  G06F11/14 ,  G06F16/182 ,  G06F16/27 ,  G06F16/174

Abstract: A method of distributing data in a distributed storage system includes receiving a file, dividing the received file into chunks, and determining a distribution of the chunks among storage devices of the distributed storage system based on a maintenance hierarchy of the distributed storage system. The maintenance hierarchy includes maintenance levels, and each maintenance level includes one or more maintenance units. Each maintenance unit has an active state and an inactive state. Moreover, each storage device is associated with a maintenance unit. The determining of the distribution of the chunks includes identifying a random selection of the storage devices matching a number of chunks of the file and being capable of maintaining accessibility of the file when one or more maintenance units are in an inactive state. The method also includes distributing the chunks to storage devices of the distributed storage system according to the determined distribution.