公开(公告)号：US11075984B1

公开(公告)日：2021-07-27

申请号：US16143245

申请日：2018-09-26

Applicant: Amazon Technologies, Inc.

Inventor： Benjamin Warren Mercier ,  Sayantan Chakravorty ,  Yasemin Avcular

IPC: H04L29/08 ,  H04L12/24 ,  H04L12/26

Abstract: At a platform at which stream data retrieval requests directed to a stream management service are processed, respective metrics of reads being performed using a first set of one or more persistent network connections established between the platform and one or more stream processing applications are collected. A request to read additional stream data requests is directed to the platform based at least in part on a decision made by a load balancer. The request is rejected based on a determination that the collected metrics satisfy a threshold.

公开(公告)号：US20210021653A1

公开(公告)日：2021-01-21

申请号：US17062435

申请日：2020-10-02

Applicant: Amazon Technologies, Inc.

Inventor： Benjamin Warren Mercier ,  Sayantan Chakravorty ,  Yasemin Avcular ,  Charlie Paucard

IPC: H04L29/06

Abstract: From a stream management service, indications of transmissions of data records of a first data stream via a persistent network connection associated with a subscription are provided to a stream processing application. In response to detecting that a transfer throttling condition associated with the subscription has been met, a delay is introduced before additional data records are transmitted via the same persistent network connection. After a subscription expiration period has elapsed, an indication of the expiration is provided to the application via the persistent connection.

公开(公告)号：US10666569B1

公开(公告)日：2020-05-26

申请号：US15275296

申请日：2016-09-23

Applicant: Amazon Technologies, Inc.

Inventor： Christopher Richard Jacques de Kadt ,  Benjamin Warren Mercier ,  Carlos Vara Callau ,  Timothy Daniel Cole ,  Aaron Gifford Freshwater ,  Sayantan Chakravorty ,  Allan Henry Vermeulen

IPC: G06F11/14 ,  G06F16/18 ,  G06F16/23 ,  H04L12/911 ,  H04L29/08

Abstract: A journal service manages a journal that holds a number of journal entries. Journal entries are provided to the journal service by one or more producer services, and the journal service provides journal entries to one or more consumer services. Lead consumers retrieve journal entries from the journal service by specifying a named cursor which identifies a location within the journal from which the journal entries are retrieved. The journal service may select a lead producer and a lead consumer from a collection of candidate producers and consumers using a fitness score determined for each candidate producer and consumer. The fitness score may be based at least in part on the processing capability, predicted reliability, and cost of each candidate producer and consumer. The journal service may replace a lead consumer or producer with an alternate consumer or producer in response to a variety of events.

公开(公告)号：US11811839B2

公开(公告)日：2023-11-07

申请号：US17073161

申请日：2020-10-16

Applicant: Amazon Technologies, Inc.

Inventor： Sayantan Chakravorty ,  Benjamin Warren Mercier

IPC: G06F15/16 ,  H04L65/61 ,  H04L67/10 ,  H04L67/306 ,  H04L65/75

CPC classification number: H04L65/61 ,  H04L65/762 ,  H04L67/10 ,  H04L67/306

Abstract: Configuration information indicating that one or more stream consumers are granted read-only access to contents of a shared-access data stream is stored at a stream management service. A virtual stream associated with the shared-access stream may be established. In response to a read request directed to the virtual stream, contents of a particular record of the shared-access data stream are provided.

公开(公告)号：US10956246B1

公开(公告)日：2021-03-23

申请号：US16143361

申请日：2018-09-26

Applicant: Amazon Technologies, Inc.

Inventor： Sumeetkumar Bagde ,  Benjamin Warren Mercier ,  Sayantan Chakravorty ,  Rohit Unnikrishnan Nair ,  Yasemin Avcular ,  Charlie Paucard

IPC: G06F11/07

Abstract: In response to a first programmatic request, metadata indicating that a first isolated read channel associated with a data stream has been established is stored at a stream management service. A read request that indicates the first isolated read channel and the stream is obtained. In response to the read request, contents of data records of the stream are provided.

公开(公告)号：US20210037072A1

公开(公告)日：2021-02-04

申请号：US17073161

申请日：2020-10-16

Applicant: Amazon Technologies, Inc.

Inventor： Sayantan Chakravorty ,  Benjamin Warren Mercier

IPC: H04L29/06 ,  H04L29/08

Abstract: Configuration information indicating that one or more stream consumers are granted read-only access to contents of a shared-access data stream is stored at a stream management service. A virtual stream associated with the shared-access stream may be established. In response to a read request directed to the virtual stream, contents of a particular record of the shared-access data stream are provided.

公开(公告)号：US10855754B1

公开(公告)日：2020-12-01

申请号：US16143340

申请日：2018-09-26

Applicant: Amazon Technologies, Inc.

Inventor： Benjamin Warren Mercier ,  Sayantan Chakravorty ,  Yasemin Avcular ,  Charlie Paucard

IPC: G06F15/173 ,  H04L29/08 ,  G06F16/28

Abstract: In response to a first programmatic request, metadata indicating that a first isolated read channel of a real-time category has been associated with a first target stream is stored at a stream management service. In response to another request, metadata indicating that a second isolated read channel of a non-real-time category has been associated with a second target stream is stored. In response to a read request indicating the first channel or the second channel, one or more data records of the corresponding target streams are provided.

公开(公告)号：US10805238B1

公开(公告)日：2020-10-13

申请号：US15275300

申请日：2016-09-23

Applicant: Amazon Technologies, Inc.

Inventor： Christopher Richard Jacques de Kadt ,  Benjamin Warren Mercier ,  Carlos Vara Callau ,  Timothy Daniel Cole ,  Aaron Gifford Freshwater ,  Sayantan Chakravorty ,  Allan Henry Vermeulen

IPC: H04L12/911 ,  H04L29/08

Abstract: A journal service maintains a journal that holds a number of journal entries. Access to the journal entries is controlled by providing a lead resource with access to a named role associated with the journal. A pool of alternate resources containing potential replacements for the lead resource is maintained. A succession order may be established for the pool of alternate resources, and individual resources in the pool of alternate resources may be managed based at least in part on their position within the succession order. The succession order may be established using various criteria that are indicative of a likelihood that a particular alternate resource will be available if the lead resource fails, such as the presence of shared points of failure between the current lead resource and an alternate resource. The lead resource may be replaced with an alternate resource as a result of various impeachment events.

公开(公告)号：US10798140B1

公开(公告)日：2020-10-06

申请号：US16143344

申请日：2018-09-26

Applicant: Amazon Technologies, Inc.

Inventor： Benjamin Warren Mercier ,  Sayantan Chakravorty ,  Yasemin Avcular ,  Charlie Paucard

IPC: H04L29/06 ,  H04L29/08

Abstract: From a stream management service, indications of transmissions of data records of a first data stream via a persistent network connection associated with a subscription are provided to a stream processing application. In response to detecting that a transfer throttling condition associated with the subscription has been met, a delay is introduced before additional data records are transmitted via the same persistent network connection. After a subscription expiration period has elapsed, an indication of the expiration is provided to the application via the persistent connection.

公开(公告)号：US11968279B1

公开(公告)日：2024-04-23

申请号：US17810299

申请日：2022-06-30

Applicant: Amazon Technologies, Inc.

Inventor： Sayantan Chakravorty ,  Nagarjuna Koduru ,  Nabanita Maji ,  Vijaya Rama Reddy Kistampalli ,  Sankalp Bhatia ,  Sahil Dorwat

IPC: H04L67/562 ,  H04L9/40 ,  H04L41/50 ,  H04L65/60

CPC classification number: H04L67/562 ,  H04L41/509 ,  H04L63/08 ,  H04L65/60

Abstract: Various embodiments of systems and methods for providing virtualized (e.g., serverless) broker clusters for a data streaming service are disclosed. A data streaming service uses a front-end proxy layer and a back-end broker layer to provide virtualized broker clusters, for example in a Kafka-based streaming service. Resources included in a virtualized broker cluster are monitored and automatically scaled-up, scaled-down, or re-balanced in a way that is transparent to data producing and/or data consuming clients of the data streaming service.