公开(公告)号：US20190138367A1

公开(公告)日：2019-05-09

申请号：US16200360

申请日：2018-11-26

Applicant: salesforce.com, inc.

Inventor： Elden Gregory BISHOP ,  Jeffrey Chao

IPC: G06F9/50

Abstract: The technology disclosed herein relates to method, system, and computer program product (computer-readable storage device) embodiments for managing resource allocation in a stream processing framework. An embodiment operates by configuring an allocation of a task sequence and machine resources to a container, and by running the task sequence, wherein the task sequence is configured to be run continuously as a plurality of units of work corresponding to the task sequence. Some embodiments further include changing the allocation responsive to a determination of an increase in data volume. A query may be taken from the task sequence and processed. Responsive to the query, a real-time result may be returned. Query processing may involve continuously applying a rule to the data stream, in real time or near real time. The rule may be set via a query language. Additionally, the data stream may be partitioned into batches for parallel processing.

公开(公告)号：US20190163539A1

公开(公告)日：2019-05-30

申请号：US16200365

申请日：2018-11-26

Applicant: salesforce.com, inc.

Inventor： Elden Gregory BISHOP ,  Jeffrey CHAO

IPC: G06F9/50

Abstract: The technology disclosed herein relates to method, system, and computer program product (computer-readable storage device) embodiments for managing resource allocation in a stream processing framework. An embodiment operates by configuring an allocation of a task sequence and machine resources to a container, partitioning a data stream into a plurality of batches arranged for parallel processing by the container via the machine resources allocated to the container, and running the task sequence, running at least one batch of the plurality of batches. Some embodiments may also include changing the allocation responsive to a determination of an increase in data volume, and may further include changing the allocation to a previous state of the allocation, responsive to a determination of a decrease in data volume. Additionally, time-based throughput of the data stream may be monitored for a given worker node configured to run a batch of the plurality of batches.

公开(公告)号：US20180074852A1

公开(公告)日：2018-03-15

申请号：US15265817

申请日：2016-09-14

Applicant: salesforce.com, inc.

Inventor： Elden Gregory BISHOP ,  Jeffrey CHAO

IPC: G06F9/48 ,  G06F9/50 ,  G06F9/455 ,  G06F9/44

CPC classification number: G06F9/4881 ,  G06F8/71 ,  G06F9/44521 ,  G06F9/45504 ,  G06F9/4552 ,  G06F9/5072

Abstract: The technology disclosed provides a novel and innovative technique for compact deployment of application code to stream processing systems. In particular, the technology disclosed relates to obviating the need of accompanying application code with its dependencies during deployment (i.e., creating fat jars) by operating a stream processing system within a container defined over worker nodes of whole machines and initializing the worker nodes with precompiled dependency libraries having precompiled classes. Accordingly, the application code is deployed to the container without its dependencies, and, once deployed, the application code is linked with the locally stored precompiled dependencies at runtime. In implementations, the application code is deployed to the container running the stream processing system between 300 milliseconds and 6 seconds. This is drastically faster than existing deployment techniques that take anywhere between 5 to 15 minutes for deployment.