摘要:
A stream of tuples is processed by a stream application. The stream application includes a plurality of processing elements that operate on one or more compute nodes, each processing element includes one or more stream operators. One or more databases that are capable of communicating with the stream application are monitored during the processing of the stream of tuples. A potential performance condition of a first database of the one or more databases is detected based on the monitoring. An output adjustment is performed, in response to the potential performance condition.
摘要:
Provided is a method for disabling encryption of data in motion in response to an event. The method includes a service processing data. The service may process the data while in a public mode, in which the service is configured to encrypt data in motion. The method further comprises detecting an event that triggers the service to go into a protected mode. The method further comprises isolating the service from one or more public systems in response to detecting the event. The method further comprises deactivating encryption of data in motion, and processing the data without encrypting the data while in motion.
摘要:
A stream of tuples to be processed by a plurality processing elements executing on two or more compute nodes is received. Each compute node stores one or more of the processing elements having one or more stream operators. It is determined whether an overhead parameter associated with a first streams service located at a first stream operator is outside of a first overhead criterion. The first streams service is ended at the first stream operator and a second streams service is instantiated at a second stream operator when the overhead parameter associated with the first streams service is outside of the first overhead criterion. The second stream operator is different from the first stream operator. The method may include determining whether the first streams service samples a first data attribute of tuples or measures performance.
摘要:
Stream applications may inefficiently use the hardware resources that execute the processing elements of the data stream. For example, a compute node may host four processing elements and execute each using a CPU. However, other CPUs on the compute node may sit idle. To take advantage of these available hardware resources, a stream programmer may identify one or more processing elements that may be cloned. The cloned processing elements may be used to generate a different execution path that is parallel to the execution path that includes the original processing elements. Because the cloned processing elements contain the same operators as the original processing elements, the data stream that was previously flowing through only the original processing element may be split and sent through both the original and cloned processing elements. In this manner, the parallel execution path may use underutilized hardware resources to increase the throughput of the data stream.
摘要:
Disclosed aspects relate to debug management in a distributed batch data processing environment which uses a shared pool of configurable computing resources. A debug configuration to fire a breakpoint based on an achievement of a debug criterion may be initiated in the distributed batch data processing environment. A data block may be detected in the distributed batch data processing environment. The data block may be analyzed with respect to the debug criterion by a debug management engine. Achievement of the debug criterion by the data block may be determined by the debug management engine. In response to determining the achievement of the debug criterion by the data block, the breakpoint may be fired based on the achievement of the debug criterion.
摘要:
Disclosed aspects relate to debug management in a distributed batch data processing environment which uses a shared pool of configurable computing resources. A debug configuration to fire a breakpoint based on an achievement of a debug criterion may be initiated in the distributed batch data processing environment. A data block may be detected in the distributed batch data processing environment. The data block may be analyzed with respect to the debug criterion by a debug management engine. Achievement of the debug criterion by the data block may be determined by the debug management engine. In response to determining the achievement of the debug criterion by the data block, the breakpoint may be fired based on the achievement of the debug criterion.
摘要:
A stream of tuples may be processed by receiving at a first stream operator a first tuple from a stream of tuples. In response to receiving the first tuple, port mutability conditions for a first stream operator and a second stream operator may be analyzed. In response to identifying the port mutability conditions for a first stream operator and a second stream operator, a first set of attribute mutability conditions for the first tuple received at the first stream operator may be identified. Based on the first set of attribute mutability conditions, a reference of an attribute from the first tuple may be generated where the reference is added to a second tuple passing from the first stream operator to the second stream operator.
摘要:
Managing a streaming environment of an operator graph by performing corrective actions based on a threshold of changes in state being reached. An operator graph includes states of information stored within a memory of a first processing element configured to process a set of tuples. The memory of the first processing element is monitored. A change in the information from a first state to a second state is identified, based on the monitoring. The change from the first state to the second state is recorded. A determination is made if the change from the first state to the second state has caused a threshold of changes between the states of the information to be reached. A corrective action is performed that modifies a configuration of the operator graph in response to the threshold of changes between the states of the information being reached.
摘要:
Stream applications may inefficiently use the hardware resources that execute the processing elements of the data stream. For example, a compute node may host four processing elements and execute each using a CPU. However, other CPUs on the compute node may sit idle. To take advantage of these available hardware resources, a stream programmer may identify one or more processing elements that may be cloned. The cloned processing elements may be used to generate a different execution path that is parallel to the execution path that includes the original processing elements. Because the cloned processing elements contain the same operators as the original processing elements, the data stream that was previously flowing through only the original processing element may be split and sent through both the original and cloned processing elements. In this manner, the parallel execution path may use underutilized hardware resources to increase the throughput of the data stream.
摘要:
A stream computing application may permit one job to connect to a data stream of a different job. As more and more jobs dynamically connect to the data stream, the connections may have a negative impact on the performance of the job that generates the data stream. Accordingly, a variety of metrics and statistics (e.g., CPU utilization or tuple rate) may be monitored to determine if the dynamic connections are harming performance. If so, the stream computing system may be optimized to mitigate the effects of the dynamic connections. For example, particular operators may be unfused from a processing element and moved to a compute node that has available computing resources. Additionally, the stream computing application may clone the data stream in order to distribute the workload of transmitting the data stream to the connected jobs.