-
公开(公告)号:US10795905B2
公开(公告)日:2020-10-06
申请号:US15858480
申请日:2017-12-29
Applicant: Amazon Technologies, Inc.
Inventor: Marvin Michael Theimer , Gaurav D. Ghare , John David Dunagan , Gregory M. Burgess , Ying Xiong
Abstract: A programmatic interface is implemented, enabling a client of a stream management service to select a data ingestion policy for a data stream. A client request selecting an at-least-once ingestion policy is received. In accordance with the at-least-once policy, a client may transmit an indication of a data record one or more times to the service until a positive acknowledgement is received. In response to receiving a plurality of transmissions indicating a particular data record, respective positive acknowledgements are sent to the client. Based on a persistence policy selected for the stream, copies of the data record are stored at one or more storage locations in response to one particular transmission of the plurality of transmissions.
-
公开(公告)号:US20180189367A1
公开(公告)日:2018-07-05
申请号:US15858480
申请日:2017-12-29
Applicant: Amazon Technologies, Inc.
Inventor: Marvin Michael Theimer , Gaurav D. Ghare , John David Dunagan , Greg Burgess , Ying Xiong
IPC: G06F17/30
CPC classification number: G06F16/254 , G06F16/113
Abstract: A programmatic interface is implemented, enabling a client of a stream management service to select a data ingestion policy for a data stream. A client request selecting an at-least-once ingestion policy is received. In accordance with the at-least-once policy, a client may transmit an indication of a data record one or more times to the service until a positive acknowledgement is received. In response to receiving a plurality of transmissions indicating a particular data record, respective positive acknowledgements are sent to the client. Based on a persistence policy selected for the stream, copies of the data record are stored at one or more storage locations in response to one particular transmission of the plurality of transmissions.
-
公开(公告)号:US10691716B2
公开(公告)日:2020-06-23
申请号:US15665103
申请日:2017-07-31
Applicant: Amazon Technologies, Inc.
Inventor: Marvin Michael Theimer , Gaurav D. Ghare , John David Dunagan , Gregory M. Burgess , Ying Xiong
Abstract: A partitioning policy, comprising an indication of an initial mapping of data records of a stream to a plurality of partitions, is selected to distribute data records of a data stream among a plurality of nodes of a stream management service. Data ingestion nodes and storage nodes are configured according to the initial mapping. In response to a determination that a triggering criterion for dynamically repartitioning the data stream has been met, a modified mapping is generated, and a different set of ingestion and storage nodes are configured. For at least some time during which arriving data records are stored in accordance with the modified mapping, data records stored at the first set of storage nodes in accordance with the initial mapping are retained.
-
公开(公告)号:US09720989B2
公开(公告)日:2017-08-01
申请号:US14077171
申请日:2013-11-11
Applicant: Amazon Technologies, Inc.
Inventor: Marvin Michael Theimer , Gaurav D. Ghare , John David Dunagan , Greg Burgess , Ying Xiong
IPC: G06F17/30 , G06F15/173 , G06F17/00
CPC classification number: G06F17/30569 , G06F17/30
Abstract: A partitioning policy, comprising an indication of an initial mapping of data records of a stream to a plurality of partitions, is selected to distribute data records of a data stream among a plurality of nodes of a stream management service. Data ingestion nodes and storage nodes are configured according to the initial mapping. In response to a determination that a triggering criterion for dynamically repartitioning the data stream has been met, a modified mapping is generated, and a different set of ingestion and storage nodes are configured. For at least some time during which arriving data records are stored in accordance with the modified mapping, data records stored at the first set of storage nodes in accordance with the initial mapping are retained.
-
公开(公告)号:US09794135B2
公开(公告)日:2017-10-17
申请号:US14077173
申请日:2013-11-11
Applicant: Amazon Technologies, Inc.
Inventor: Marvin Michael Theimer , Gaurav D. Ghare , John David Dunagan , Greg Burgess , Ying Xiong
IPC: H04L12/24 , G06F17/30 , G06F15/16 , G06F15/173
CPC classification number: H04L41/24 , G06F17/30516 , H04L41/0668 , H04L41/18 , H04L41/5051
Abstract: A control node of a multi-tenant stream management service receives a request to initialize a data stream to be comprised of a plurality of data records. The control node determines, based on a partitioning policy, parameters to be used to configure subsystems for ingestion, storage and retrieval of the records. The control node identifies resources to be used for a node of retrieval subsystem The retrieval node is configured to implement programmatic record retrieval interfaces, including respective interfaces to implement non-sequential and sequential access patterns. The control node configures the retrieval node using the selected resources.
-
公开(公告)号:US09276959B2
公开(公告)日:2016-03-01
申请号:US14077158
申请日:2013-11-11
Applicant: Amazon Technologies, Inc.
Inventor: Marvin Michael Theimer , Gaurav D. Ghare , John David Dunagan , Greg Burgess , Ying Xiong
CPC classification number: H04L63/20 , H04L67/1004 , H04L67/1027
Abstract: A configuration request comprising a security option selected for a particular data stream is received. Nodes of a plurality of functional categories, such as a data ingestion category and a data retrieval category are to be configured for the stream. The security option indicates a security profile of a resource to be used for nodes of at least one functional category. In accordance with the configuration request, a node of a first functional category is configured at a resource with a first security profile, and configuration of a node of a second functional category is initiated at a different resource with a different security profile.
Abstract translation: 接收包括为特定数据流选择的安全选项的配置请求。 要为流配置多个功能类别的节点,例如数据摄取类别和数据检索类别。 安全选项指示要用于至少一个功能类别的节点的资源的安全配置文件。 根据配置请求,在具有第一安全简档的资源上配置第一功能类别的节点,并且在具有不同安全性配置文件的不同资源上启动第二功能类别的节点的配置。
-
公开(公告)号:US09858322B2
公开(公告)日:2018-01-02
申请号:US14077162
申请日:2013-11-11
Applicant: Amazon Technologies, Inc.
Inventor: Marvin Michael Theimer , Gaurav D. Ghare , John David Dunagan , Greg Burgess , Ying Xiong
IPC: G06F17/30 , G06F15/173 , G06F17/00
CPC classification number: G06F17/30563 , G06F17/30073
Abstract: A programmatic interface is implemented, enabling a client of a stream management service to select a data ingestion policy for a data stream. A client request selecting an at-least-once ingestion policy is received. In accordance with the at-least-once policy, a client may transmit an indication of a data record one or more times to the service until a positive acknowledgement is received. In response to receiving a plurality of transmissions indicating a particular data record, respective positive acknowledgements are sent to the client. Based on a persistence policy selected for the stream, copies of the data record are stored at one or more storage locations in response to one particular transmission of the plurality of transmissions.
-
公开(公告)号:US20170357703A1
公开(公告)日:2017-12-14
申请号:US15665103
申请日:2017-07-31
Applicant: Amazon Technologies, Inc.
Inventor: Marvin Michael Theimer , Gaurav D. Ghare , John David Dunagan , Gregory M. Burgess , Ying Xiong
IPC: G06F17/30
Abstract: A partitioning policy, comprising an indication of an initial mapping of data records of a stream to a plurality of partitions, is selected to distribute data records of a data stream among a plurality of nodes of a stream management service. Data ingestion nodes and storage nodes are configured according to the initial mapping. In response to a determination that a triggering criterion for dynamically repartitioning the data stream has been met, a modified mapping is generated, and a different set of ingestion and storage nodes are configured. For at least some time during which arriving data records are stored in accordance with the modified mapping, data records stored at the first set of storage nodes in accordance with the initial mapping are retained.
-
-
-
-
-
-
-