公开(公告)号：US20190163539A1

公开(公告)日：2019-05-30

申请号：US16200365

申请日：2018-11-26

Applicant: salesforce.com, inc.

Inventor： Elden Gregory BISHOP ,  Jeffrey CHAO

IPC: G06F9/50

Abstract: The technology disclosed herein relates to method, system, and computer program product (computer-readable storage device) embodiments for managing resource allocation in a stream processing framework. An embodiment operates by configuring an allocation of a task sequence and machine resources to a container, partitioning a data stream into a plurality of batches arranged for parallel processing by the container via the machine resources allocated to the container, and running the task sequence, running at least one batch of the plurality of batches. Some embodiments may also include changing the allocation responsive to a determination of an increase in data volume, and may further include changing the allocation to a previous state of the allocation, responsive to a determination of a decrease in data volume. Additionally, time-based throughput of the data stream may be monitored for a given worker node configured to run a batch of the plurality of batches.

公开(公告)号：US20180074852A1

公开(公告)日：2018-03-15

申请号：US15265817

申请日：2016-09-14

Applicant: salesforce.com, inc.

Inventor： Elden Gregory BISHOP ,  Jeffrey CHAO

IPC: G06F9/48 ,  G06F9/50 ,  G06F9/455 ,  G06F9/44

CPC classification number: G06F9/4881 ,  G06F8/71 ,  G06F9/44521 ,  G06F9/45504 ,  G06F9/4552 ,  G06F9/5072

Abstract: The technology disclosed provides a novel and innovative technique for compact deployment of application code to stream processing systems. In particular, the technology disclosed relates to obviating the need of accompanying application code with its dependencies during deployment (i.e., creating fat jars) by operating a stream processing system within a container defined over worker nodes of whole machines and initializing the worker nodes with precompiled dependency libraries having precompiled classes. Accordingly, the application code is deployed to the container without its dependencies, and, once deployed, the application code is linked with the locally stored precompiled dependencies at runtime. In implementations, the application code is deployed to the container running the stream processing system between 300 milliseconds and 6 seconds. This is drastically faster than existing deployment techniques that take anywhere between 5 to 15 minutes for deployment.