-
公开(公告)号:US11232106B1
公开(公告)日:2022-01-25
申请号:US15934691
申请日:2018-03-23
Applicant: Amazon Technologies, Inc.
Inventor: Praveen Kumar Gattu , Nagesh V. Honnalli , Gaurav D. Ghare , Christopher James Bradfield , Ryan Nienhuis
IPC: G06F16/2455 , H04L12/24 , H04L12/26 , G06F16/248 , G06F16/28
Abstract: Windowed queries with event-based open times are described for use in data analytics for streaming data. A window semantic may be used to define windowed queries of an analytics application that, when executed against an incoming data stream, open a window at a time when a first event associated with a grouping key is received by a windowed query. The window remains open for a predefined duration specified in the windowed query, and query results may be generated based on any events associated with the grouping key that are received while the window remains open. After a lapse of the predefined duration, the window closes, and the query results may be output in association with the window.
-
公开(公告)号:US10560465B2
公开(公告)日:2020-02-11
申请号:US16186314
申请日:2018-11-09
Applicant: Amazon Technologies, Inc.
Inventor: Gaurav D. Ghare , Roger Shane Barga
IPC: H04L29/06
Abstract: Real time detection of anomalies may be implemented for a data stream. A data stream may receive data records as input. An evaluation of the data records may be performed as the data records are received utilizing an anomaly detection model that is dynamically generated for a time period that includes the data records. A responsive action may be performed in response to detecting various anomalies, such as providing a notification of the anomaly, requesting a corrective action, invoking further analysis of the detected anomaly. Evaluations of the data record and responsive actions may be performed as specified by a user, in some embodiments.
-
公开(公告)号:US09794135B2
公开(公告)日:2017-10-17
申请号:US14077173
申请日:2013-11-11
Applicant: Amazon Technologies, Inc.
Inventor: Marvin Michael Theimer , Gaurav D. Ghare , John David Dunagan , Greg Burgess , Ying Xiong
IPC: H04L12/24 , G06F17/30 , G06F15/16 , G06F15/173
CPC classification number: H04L41/24 , G06F17/30516 , H04L41/0668 , H04L41/18 , H04L41/5051
Abstract: A control node of a multi-tenant stream management service receives a request to initialize a data stream to be comprised of a plurality of data records. The control node determines, based on a partitioning policy, parameters to be used to configure subsystems for ingestion, storage and retrieval of the records. The control node identifies resources to be used for a node of retrieval subsystem The retrieval node is configured to implement programmatic record retrieval interfaces, including respective interfaces to implement non-sequential and sequential access patterns. The control node configures the retrieval node using the selected resources.
-
公开(公告)号:US20170235645A1
公开(公告)日:2017-08-17
申请号:US15583823
申请日:2017-05-01
Applicant: Amazon Technologies, Inc.
Inventor: Marvin Michael Theimer , Benjamin Warren Mercier , Nathaniel Martin Glass , Gaurav D. Ghare
Abstract: A replication chain comprising one or more replication nodes of a multi-tenant stream management system is assigned to store data records of a partition of a particular data stream. A data record of the partition is received at a selected replication node of the replication chain. In a sequential order, a respective replica of the data record is stored at each replication node of the chain. An acknowledgement of a successful storage of the data record is provided after the replications are completed.
-
公开(公告)号:US09276959B2
公开(公告)日:2016-03-01
申请号:US14077158
申请日:2013-11-11
Applicant: Amazon Technologies, Inc.
Inventor: Marvin Michael Theimer , Gaurav D. Ghare , John David Dunagan , Greg Burgess , Ying Xiong
CPC classification number: H04L63/20 , H04L67/1004 , H04L67/1027
Abstract: A configuration request comprising a security option selected for a particular data stream is received. Nodes of a plurality of functional categories, such as a data ingestion category and a data retrieval category are to be configured for the stream. The security option indicates a security profile of a resource to be used for nodes of at least one functional category. In accordance with the configuration request, a node of a first functional category is configured at a resource with a first security profile, and configuration of a node of a second functional category is initiated at a different resource with a different security profile.
Abstract translation: 接收包括为特定数据流选择的安全选项的配置请求。 要为流配置多个功能类别的节点,例如数据摄取类别和数据检索类别。 安全选项指示要用于至少一个功能类别的节点的资源的安全配置文件。 根据配置请求,在具有第一安全简档的资源上配置第一功能类别的节点,并且在具有不同安全性配置文件的不同资源上启动第二功能类别的节点的配置。
-
公开(公告)号:US11836533B2
公开(公告)日:2023-12-05
申请号:US17813009
申请日:2022-07-15
Applicant: Amazon Technologies, Inc.
Inventor: Gaurav D. Ghare , Roger Shane Barga
IPC: G06F9/50 , G06F16/2455
CPC classification number: G06F9/505 , G06F16/24568
Abstract: Automated reconfiguration of real time data stream processing may be implemented. A processing function that describes one or more operations to be performed with respect to one or more data streams may be executed at one or more processing nodes. Performance metrics describing the performance of the processing function at the processing nodes may be collected and monitored. A reconfiguration event may be detected for the processing function. A different execution configuration for the processing function may be determined and initiated in response to detecting the reconfiguration event.
-
公开(公告)号:US20190082005A1
公开(公告)日:2019-03-14
申请号:US16179339
申请日:2018-11-02
Applicant: Amazon Technologies, Inc.
Inventor: Gaurav D. Ghare , Roger Shane Barga
Abstract: Managed function execution for processing data streams in real time may be. A function that describes one or more operations to be performed with respect to one or more data streams may be received via programmatic interface for a managed stream processing system. Stream processing nodes capable of applying the function may be determined and execution of the one or more operations may be initiated at the stream processing nodes as data records of the data stream are received. Results of the application of the processing function may be provided to one or more destinations specified for the function. Performance metrics may also be collected for the execution of the function and provided to a client that submitted the function.
-
公开(公告)号:US20180121254A1
公开(公告)日:2018-05-03
申请号:US15858491
申请日:2017-12-29
Applicant: Amazon Technologies, Inc.
Inventor: Gaurav D. Ghare
CPC classification number: G06F9/5083
Abstract: A stream management system may implement dynamic management of a data stream. Utilization data of different partitions of a data stream may be tracked. When routing a data record received at the stream management system, a partition may be dynamically identified for the data recorded. The data record may then be directed to the identified partition. Other management operations, such as repartitioning the data stream or reassigning resources for processing data records in the data stream may be performed based on the utilization data tracked for the partitions.
-
公开(公告)号:US09858322B2
公开(公告)日:2018-01-02
申请号:US14077162
申请日:2013-11-11
Applicant: Amazon Technologies, Inc.
Inventor: Marvin Michael Theimer , Gaurav D. Ghare , John David Dunagan , Greg Burgess , Ying Xiong
IPC: G06F17/30 , G06F15/173 , G06F17/00
CPC classification number: G06F17/30563 , G06F17/30073
Abstract: A programmatic interface is implemented, enabling a client of a stream management service to select a data ingestion policy for a data stream. A client request selecting an at-least-once ingestion policy is received. In accordance with the at-least-once policy, a client may transmit an indication of a data record one or more times to the service until a positive acknowledgement is received. In response to receiving a plurality of transmissions indicating a particular data record, respective positive acknowledgements are sent to the client. Based on a persistence policy selected for the stream, copies of the data record are stored at one or more storage locations in response to one particular transmission of the plurality of transmissions.
-
公开(公告)号:US11178197B2
公开(公告)日:2021-11-16
申请号:US16825404
申请日:2020-03-20
Applicant: Amazon Technologies, Inc.
Inventor: Gaurav D. Ghare
Abstract: Idempotent processing of data may be implemented for data records retrieved from a data stream. A data stream may receive data records as input and distribute the ingestion, storage, and processing of the data records amongst one or more partitions of the data stream. Partition metadata may be maintained which includes checkpoint metadata for retrieving, processing, and sending data records in the data stream to a specified destination. When assigned a partition for processing, checkpoint metadata for partition may be accessed to determine whether a pending checkpoint for the partition exists. If not pending checkpoint exists, new data records may be retrieved, processed, and sent from the partition of the data stream to a specified destination. If a checkpoint is pending, then the data records identified by the checkpoint metadata as pending may be retrieved, processed, and sent to the specified destination.
-
-
-
-
-
-
-
-
-