DATASTORE AWARE STREAMS COMPUTING

    公开(公告)号:US20220405301A1

    公开(公告)日:2022-12-22

    申请号:US17353928

    申请日:2021-06-22

    IPC分类号: G06F16/27 G06F11/34

    摘要: A stream of tuples is processed by a stream application. The stream application includes a plurality of processing elements that operate on one or more compute nodes, each processing element includes one or more stream operators. One or more databases that are capable of communicating with the stream application are monitored during the processing of the stream of tuples. A potential performance condition of a first database of the one or more databases is detected based on the monitoring. An output adjustment is performed, in response to the potential performance condition.

    SELECTIVE ENCRYPTION OF DATA IN MOTION

    公开(公告)号:US20220030012A1

    公开(公告)日:2022-01-27

    申请号:US16939389

    申请日:2020-07-27

    IPC分类号: H04L29/06

    摘要: Provided is a method for disabling encryption of data in motion in response to an event. The method includes a service processing data. The service may process the data while in a public mode, in which the service is configured to encrypt data in motion. The method further comprises detecting an event that triggers the service to go into a protected mode. The method further comprises isolating the service from one or more public systems in response to detecting the event. The method further comprises deactivating encryption of data in motion, and processing the data without encrypting the data while in motion.

    Placement of services in stream computing applications

    公开(公告)号:US10528568B2

    公开(公告)日:2020-01-07

    申请号:US15593762

    申请日:2017-05-12

    摘要: A stream of tuples to be processed by a plurality processing elements executing on two or more compute nodes is received. Each compute node stores one or more of the processing elements having one or more stream operators. It is determined whether an overhead parameter associated with a first streams service located at a first stream operator is outside of a first overhead criterion. The first streams service is ended at the first stream operator and a second streams service is instantiated at a second stream operator when the overhead parameter associated with the first streams service is outside of the first overhead criterion. The second stream operator is different from the first stream operator. The method may include determining whether the first streams service samples a first data attribute of tuples or measures performance.

    Processing element management in a streaming data system

    公开(公告)号:US10296386B2

    公开(公告)日:2019-05-21

    申请号:US15817426

    申请日:2017-11-20

    IPC分类号: G06F9/50 G06F9/44 G06F9/48

    摘要: Stream applications may inefficiently use the hardware resources that execute the processing elements of the data stream. For example, a compute node may host four processing elements and execute each using a CPU. However, other CPUs on the compute node may sit idle. To take advantage of these available hardware resources, a stream programmer may identify one or more processing elements that may be cloned. The cloned processing elements may be used to generate a different execution path that is parallel to the execution path that includes the original processing elements. Because the cloned processing elements contain the same operators as the original processing elements, the data stream that was previously flowing through only the original processing element may be split and sent through both the original and cloned processing elements. In this manner, the parallel execution path may use underutilized hardware resources to increase the throughput of the data stream.

    Debug management in a distributed batch data processing environment

    公开(公告)号:US10169201B2

    公开(公告)日:2019-01-01

    申请号:US15401011

    申请日:2017-01-07

    摘要: Disclosed aspects relate to debug management in a distributed batch data processing environment which uses a shared pool of configurable computing resources. A debug configuration to fire a breakpoint based on an achievement of a debug criterion may be initiated in the distributed batch data processing environment. A data block may be detected in the distributed batch data processing environment. The data block may be analyzed with respect to the debug criterion by a debug management engine. Achievement of the debug criterion by the data block may be determined by the debug management engine. In response to determining the achievement of the debug criterion by the data block, the breakpoint may be fired based on the achievement of the debug criterion.

    Processing a stream of tuples
    7.
    发明授权

    公开(公告)号:US09971851B2

    公开(公告)日:2018-05-15

    申请号:US14749073

    申请日:2015-06-24

    摘要: A stream of tuples may be processed by receiving at a first stream operator a first tuple from a stream of tuples. In response to receiving the first tuple, port mutability conditions for a first stream operator and a second stream operator may be analyzed. In response to identifying the port mutability conditions for a first stream operator and a second stream operator, a first set of attribute mutability conditions for the first tuple received at the first stream operator may be identified. Based on the first set of attribute mutability conditions, a reference of an attribute from the first tuple may be generated where the reference is added to a second tuple passing from the first stream operator to the second stream operator.

    MONITORING STATES OF PROCESSING ELEMENTS
    8.
    发明申请

    公开(公告)号:US20170168748A1

    公开(公告)日:2017-06-15

    申请号:US14969952

    申请日:2015-12-15

    IPC分类号: G06F3/06 G06F3/0481

    摘要: Managing a streaming environment of an operator graph by performing corrective actions based on a threshold of changes in state being reached. An operator graph includes states of information stored within a memory of a first processing element configured to process a set of tuples. The memory of the first processing element is monitored. A change in the information from a first state to a second state is identified, based on the monitoring. The change from the first state to the second state is recorded. A determination is made if the change from the first state to the second state has caused a threshold of changes between the states of the information to be reached. A corrective action is performed that modifies a configuration of the operator graph in response to the threshold of changes between the states of the information being reached.

    Processing element management in a streaming data system
    9.
    发明授权
    Processing element management in a streaming data system 有权
    流数据系统中的处理元件管理

    公开(公告)号:US09535707B2

    公开(公告)日:2017-01-03

    申请号:US13709405

    申请日:2012-12-10

    IPC分类号: G06F15/16 G06F9/44 G06F9/50

    摘要: Stream applications may inefficiently use the hardware resources that execute the processing elements of the data stream. For example, a compute node may host four processing elements and execute each using a CPU. However, other CPUs on the compute node may sit idle. To take advantage of these available hardware resources, a stream programmer may identify one or more processing elements that may be cloned. The cloned processing elements may be used to generate a different execution path that is parallel to the execution path that includes the original processing elements. Because the cloned processing elements contain the same operators as the original processing elements, the data stream that was previously flowing through only the original processing element may be split and sent through both the original and cloned processing elements. In this manner, the parallel execution path may use underutilized hardware resources to increase the throughput of the data stream.

    摘要翻译: 流应用可能会低效地使用执行数据流处理元素的硬件资源。 例如,计算节点可以托管四个处理元件,并使用CPU执行每个处理元件。 但是,计算节点上的其他CPU可能会处于空闲状态。 为了利用这些可用的硬件资源,流程序员可以识别可被克隆的一个或多个处理元件。 克隆的处理元件可以用于生成与包括原始处理元件的执行路径平行的不同执行路径。 因为克隆的处理元素包含与原始处理元素相同的操作符,所以先前仅流过原始处理元素的数据流可以被分割并通过原始和克隆处理元素发送。 以这种方式,并行执行路径可以使用未充分利用的硬件资源来增加数据流的吞吐量。

    Operator graph changes in response to dynamic connections in stream computing applications
    10.
    发明授权
    Operator graph changes in response to dynamic connections in stream computing applications 有权
    响应于流计算应用中的动态连接的运算符图更改

    公开(公告)号:US09135057B2

    公开(公告)日:2015-09-15

    申请号:US13780800

    申请日:2013-02-28

    摘要: A stream computing application may permit one job to connect to a data stream of a different job. As more and more jobs dynamically connect to the data stream, the connections may have a negative impact on the performance of the job that generates the data stream. Accordingly, a variety of metrics and statistics (e.g., CPU utilization or tuple rate) may be monitored to determine if the dynamic connections are harming performance. If so, the stream computing system may be optimized to mitigate the effects of the dynamic connections. For example, particular operators may be unfused from a processing element and moved to a compute node that has available computing resources. Additionally, the stream computing application may clone the data stream in order to distribute the workload of transmitting the data stream to the connected jobs.

    摘要翻译: 流计算应用可以允许一个作业连接到不同作业的数据流。 随着越来越多的作业动态地连接到数据流,连接可能对生成数据流的作业的性能产生负面影响。 因此,可以监视各种指标和统计(例如,CPU利用率或元组速率),以确定动态连接是否损害性能。 如果是这样,可以优化流计算系统以减轻动态连接的影响。 例如,特定的运算符可以从处理元件中被引用,并且被移动到具有可用计算资源的计算节点。 此外,流计算应用可以克隆数据流,以便将发送数据流的工作负载分配给所连接的作业。