公开(公告)号：US11507594B2

公开(公告)日：2022-11-22

申请号：US15640887

申请日：2017-07-03

Applicant: Amazon Technologies, Inc.

Inventor： Alexander Clark Prater ,  Victor Gregory Estevez ,  Matthew Rice Carlson ,  Nipun Dureja ,  Yung-Chun Lin ,  Udit Madan

IPC: G06F16/00 ,  G06F16/27 ,  G06F11/14 ,  G06F16/20 ,  G06F3/06

Abstract: A bulk data distribution system in which, instead of multiple data consumers contending to access the same data sets from a primary data store, one or more producers capture snapshots (states of the data sets at particular points in time) and upload the snapshots to an intermediate data store for access by snapshot consumers. The snapshot consumers may download the snapshots to generate local versions of the data sets for access by one or more data processing applications or processes. A snapshot producer may periodically generate full snapshots of a data set, and may generate one or more incremental snapshots of the data set between full snapshots. A snapshot consumer may bootstrap a local data set from a full snapshot and one or more incrementals, and may maintain state of the local data set by accessing new snapshots uploaded by the producer.

公开(公告)号：US20170300551A1

公开(公告)日：2017-10-19

申请号：US15640887

申请日：2017-07-03

Applicant: Amazon Technologies, Inc.

Inventor： Alexander Clark Prater ,  Victor Gregory Estevez ,  Matthew Rice Carlson ,  Nipun Dureja ,  Yung-Chun Lin ,  Udit Madan

IPC: G06F17/30 ,  G06F11/14

CPC classification number: G06F16/27 ,  G06F3/061 ,  G06F3/065 ,  G06F3/067 ,  G06F11/1451 ,  G06F16/00 ,  G06F16/20

Abstract: A bulk data distribution system in which, instead of multiple data consumers contending to access the same data sets from a primary data store, one or more producers capture snapshots (states of the data sets at particular points in time) and upload the snapshots to an intermediate data store for access by snapshot consumers. The snapshot consumers may download the snapshots to generate local versions of the data sets for access by one or more data processing applications or processes. A snapshot producer may periodically generate full snapshots of a data set, and may generate one or more incremental snapshots of the data set between full snapshots. A snapshot consumer may bootstrap a local data set from a full snapshot and one or more incrementals, and may maintain state of the local data set by accessing new snapshots uploaded by the producer.

公开(公告)号：US09697268B1

公开(公告)日：2017-07-04

申请号：US14488022

申请日：2014-09-16

Applicant: Amazon Technologies, Inc.

Inventor： Alexander Clark Prater ,  Victor Gregory Estevez ,  Matthew Rice Carlson ,  Nipun Dureja ,  Yung-Chun Lin ,  Udit Madan

IPC: G06F17/30 ,  G06F11/14

CPC classification number: G06F17/30575 ,  G06F3/061 ,  G06F3/065 ,  G06F3/067 ,  G06F11/1451 ,  G06F17/30 ,  G06F17/30286

Abstract: A bulk data distribution system in which, instead of multiple data consumers contending to access the same data sets from a primary data store, one or more producers capture snapshots (states of the data sets at particular points in time) and upload the snapshots to an intermediate data store for access by snapshot consumers. The snapshot consumers may download the snapshots to generate local versions of the data sets for access by one or more data processing applications or processes. A snapshot producer may periodically generate full snapshots of a data set, and may generate one or more incremental snapshots of the data set between full snapshots. A snapshot consumer may bootstrap a local data set from a full snapshot and one or more incrementals, and may maintain state of the local data set by accessing new snapshots uploaded by the producer.