公开(公告)号：US20150134796A1

公开(公告)日：2015-05-14

申请号：US14077171

申请日：2013-11-11

Applicant: AMAZON TECHNOLOGIES, INC.

Inventor： MARVIN MICHAEL THEIMER ,  GAURAV D. GHARE ,  JOHN DAVID DUNAGAN ,  GREG BURGESS ,  YING XIONG

IPC: H04L12/24

CPC classification number: G06F17/30569 ,  G06F17/30

Abstract: A partitioning policy, comprising an indication of an initial mapping of data records of a stream to a plurality of partitions, is selected to distribute data records of a data stream among a plurality of nodes of a stream management service. Data ingestion nodes and storage nodes are configured according to the initial mapping. In response to a determination that a triggering criterion for dynamically repartitioning the data stream has been met, a modified mapping is generated, and a different set of ingestion and storage nodes are configured. For at least some time during which arriving data records are stored in accordance with the modified mapping, data records stored at the first set of storage nodes in accordance with the initial mapping are retained.

Abstract translation: 选择包括流到多个分区的数据记录的初始映射的指示的分区策略，以在流管理服务的多个节点之间分配数据流的数据记录。 数据摄取节点和存储节点根据初始映射进行配置。 响应于确定已经满足用于动态地重新分配数据流的触发标准，生成修改的映射，并且配置不同的摄取和存储节点。 在至少一段时间内，根据修改的映射来存储到达的数据记录，保留根据初始映射存储在第一组存储节点处的数据记录。

公开(公告)号：US20150135255A1

公开(公告)日：2015-05-14

申请号：US14077158

申请日：2013-11-11

Applicant: Amazon Technologies, Inc.

Inventor： MARVIN MICHAEL THEIMER ,  GAURAV D. GHARE ,  JOHN DAVID DUNAGAN ,  GREG BURGESS ,  YING XIONG

IPC: H04L29/06

CPC classification number: H04L63/20 ,  H04L67/1004 ,  H04L67/1027

Abstract: A configuration request comprising a security option selected for a particular data stream is received. Nodes of a plurality of functional categories, such as a data ingestion category and a data retrieval category are to be configured for the stream. The security option indicates a security profile of a resource to be used for nodes of at least one functional category. In accordance with the configuration request, a node of a first functional category is configured at a resource with a first security profile, and configuration of a node of a second functional category is initiated at a different resource with a different security profile.

Abstract translation: 接收包括为特定数据流选择的安全选项的配置请求。 要为流配置多个功能类别的节点，例如数据摄取类别和数据检索类别。 安全选项指示要用于至少一个功能类别的节点的资源的安全配置文件。 根据配置请求，在具有第一安全简档的资源上配置第一功能类别的节点，并且在具有不同安全性配置文件的不同资源上启动第二功能类别的节点的配置。

公开(公告)号：US20150134795A1

公开(公告)日：2015-05-14

申请号：US14077162

申请日：2013-11-11

Applicant: Amazon Technologies, Inc.

Inventor： MARVIN MICHAEL THEIMER ,  GAURAV D. GHARE ,  JOHN DAVID DUNAGAN ,  GREG BURGESS ,  YING XIONG

IPC: H04L12/24

CPC classification number: G06F17/30563 ,  G06F17/30073

Abstract: A programmatic interface is implemented, enabling a client of a stream management service to select a data ingestion policy for a data stream. A client request selecting an at-least-once ingestion policy is received. In accordance with the at-least-once policy, a client may transmit an indication of a data record one or more times to the service until a positive acknowledgement is received. In response to receiving a plurality of transmissions indicating a particular data record, respective positive acknowledgements are sent to the client. Based on a persistence policy selected for the stream, copies of the data record are stored at one or more storage locations in response to one particular transmission of the plurality of transmissions.

Abstract translation: 实现编程接口，使得流管理服务的客户端能够选择数据流的数据摄取策略。 接收到选择至少一次摄取策略的客户端请求。 根据至少一次策略，客户端可以向服务发送一次或多次数据记录的指示，直到接收到肯定的确认。 响应于接收到指示特定数据记录的多个传输，相应的肯定确认被发送给客户端。 基于为流选择的持久性策略，响应于多个传输的一个特定传输，将数据记录的副本存储在一个或多个存储位置。

公开(公告)号：US20150134626A1

公开(公告)日：2015-05-14

申请号：US14077167

申请日：2013-11-11

Applicant: Amazon Technologies, Inc.

Inventor： MARVIN MICHAEL THEIMER ,  GAURAV D. GHARE ,  JOHN DAVID DUNAGAN ,  GREG BURGESS ,  YING XIONG

IPC: G06F17/30

CPC classification number: G06F16/21 ,  G06F9/5066 ,  G06F11/1469 ,  G06F11/1482 ,  G06F11/2028 ,  G06F11/3006 ,  G06F11/3055 ,  G06F11/3433

Abstract: A control node of a multi-tenant stream processing service receives a request indicating an operation to be performed on data records of a particular data stream. Based on a stream partitioning policy, the control node determines an initial number of worker nodes to be used. The control node configures a worker node to perform the operation on received records. In response to a determination that the worker node is in an unhealthy state, the control node configures a replacement worker node.

Abstract translation: 多租户流处理服务的控制节点接收指示对特定数据流的数据记录执行的操作的请求。 基于流分配策略，控制节点确定要使用的工作节点的初始数量。 控制节点配置工作节点对接收到的记录执行操作。 响应于确定工作节点处于不健康状态，控制节点配置替换工作者节点。

公开(公告)号：US20170289240A1

公开(公告)日：2017-10-05

申请号：US15084347

申请日：2016-03-29

Applicant: Amazon Technologies, Inc.

Inventor： GAURAV D. GHARE ,  ROGER Shane BARGA

IPC: H04L29/08 ,  G06F17/30 ,  H04L29/06

CPC classification number: H04L67/10 ,  G06F17/30283 ,  G06F17/30516 ,  H04L65/4069

Abstract: Managed function execution for processing data streams in real time may be. A function that describes one or more operations to be performed with respect to one or more data streams may be received via programmatic interface for a managed stream processing system. Stream processing nodes capable of applying the function may be determined and execution of the one or more operations may be initiated at the stream processing nodes as data records of the data stream are received. Results of the application of the processing function may be provided to one or more destinations specified for the function. Performance metrics may also be collected for the execution of the function and provided to a client that submitted the function.

公开(公告)号：US20150134797A1

公开(公告)日：2015-05-14

申请号：US14077173

申请日：2013-11-11

Applicant: AMAZON TECHNOLOGIES, INC.

Inventor： MARVIN MICHAEL THEIMER ,  GAURAV D. GHARE ,  JOHN DAVID DUNAGAN ,  GREG BURGESS ,  YING XIONG

IPC: H04L12/24

CPC classification number: H04L41/24 ,  G06F17/30516 ,  H04L41/0668 ,  H04L41/18 ,  H04L41/5051

Abstract: A control node of a multi-tenant stream management service receives a request to initialize a data stream to be comprised of a plurality of data records. The control node determines, based on a partitioning policy, parameters to be used to configure subsystems for ingestion, storage and retrieval of the records. The control node identifies resources to be used for a node of retrieval subsystem The retrieval node is configured to implement programmatic record retrieval interfaces, including respective interfaces to implement non-sequential and sequential access patterns. The control node configures the retrieval node using the selected resources.

Abstract translation: 多租户流管理服务的控制节点接收初始化要由多个数据记录组成的数据流的请求。 控制节点基于分区策略确定要用于配置子系统以进行记录的摄取，存储和检索的参数。 控制节点识别要用于检索子系统的节点的资源。检索节点被配置为实现程序化记录检索接口，包括相应的接口以实现非顺序和顺序访问模式。 控制节点使用所选择的资源配置检索节点。