Dynamic runtime choosing of processing communication methods

    公开(公告)号:US09148496B2

    公开(公告)日:2015-09-29

    申请号:US13681820

    申请日:2012-11-20

    IPC分类号: G06F15/16 H04L29/06

    CPC分类号: H04L69/08 H04L69/18

    摘要: Techniques are described for assigning and changing communication protocols for a pair of processing elements. The communication protocol determines how the pair of processing elements transmits data in a stream application. The pair may be assigned a communication protocol (e.g., TCP/IP or a protocol that uses a relational database, shared file system, or shared memory) before the operator graph begins to stream data. This assignment may be based on a priority of the processing elements and/or a priority of the communication protocols. After the operator graph begins to stream data, the pair of processing elements may switch to a different communication protocol. The decision to switch the communication protocol may be based on whether the pair of processing elements or assigned communication protocol is meeting established performance standards for the stream application.

    Analysis of operator graph and dynamic reallocation of a resource to improve performance
    3.
    发明授权
    Analysis of operator graph and dynamic reallocation of a resource to improve performance 有权
    分析运算符图和动态重新分配资源以提高性能

    公开(公告)号:US08997108B2

    公开(公告)日:2015-03-31

    申请号:US13679308

    申请日:2012-11-16

    IPC分类号: G06F9/50 G06F11/07 G06F9/52

    摘要: An operator graph analysis mechanism analyzes an operator graph corresponding to an application for problems as the application runs, and determines potential reallocations from a reallocation policy. The reallocation policy may specify potential reallocations depending on whether one or more operators in the operator graph are compute bound, memory bound, communication bound, or storage bound. The operator graph analysis mechanism includes a resource reallocation mechanism that can dynamically change allocation of resources in the system at runtime to address problems detected in the operator graph. The operator graph analysis mechanism thus allows an application represented by an operator graph to dynamically evolve over time to optimize its performance at runtime.

    摘要翻译: 运算符图分析机制分析应用程序运行时与应用程序相对应的运算符图,并确定重新分配策略的潜在重新分配。 重新分配策略可以根据运算符图中的一个或多个运算符是计算边界,存储器边界,通信限制还是存储绑定来指定潜在的重新分配。 运营商图分析机制包括资源重分配机制,可以在运行时动态地改变系统中资源的分配,以解决运营商图中检测到的问题。 因此,运算符图分析机制允许由运算符图表示的应用随时间动态演变以优化运行时的性能。

    Using predictive determinism within a streaming environment
    4.
    发明授权
    Using predictive determinism within a streaming environment 有权
    在流媒体环境中使用预测决定论

    公开(公告)号:US08954713B2

    公开(公告)日:2015-02-10

    申请号:US13681919

    申请日:2012-11-20

    IPC分类号: G06F15/17 H04L29/06 G06F9/06

    CPC分类号: H04L65/608 G06F9/06 G06F15/17

    摘要: Techniques are described for transmitting predicted output data on a processing element in a stream computing application instead of processing currently received input data. The stream computing application monitors the output of a processing element and determines whether its output is predictable, for example, if the previously transmitted output values are within a predefined range or if one or more input values correlate with the same one or more output values. The application may then generate a predicted output value to transmit from the processing element instead of transmitting a processed output value based on current input values. The predicted output value may be, for example, an average of the previously transmitted output values or a previously transmitted output value that was transmitted in response to a previously received input value that is similar to a currently received input value.

    摘要翻译: 描述了用于在流计算应用中的处理元件上传送预测输出数据而不是处理当前接收到的输入数据的技术。 流计算应用程序监视处理元件的输出并确定其输出是否可预测,例如,如果先前发送的输出值在预定范围内,或者如果一个或多个输入值与相同的一个或多个输出值相关联。 然后,应用程序可以生成预测的输出值以从处理元件发送,而不是基于当前输入值发送经处理的输出值。 预测的输出值可以是例如先前发送的输出值的平均值或响应于与当前接收到的输入值相似的先前接收到的输入值而发送的先前发送的输出值。

    Streams debugging within a windowing condition
    5.
    发明授权
    Streams debugging within a windowing condition 有权
    在开窗状态下流调试

    公开(公告)号:US08924940B2

    公开(公告)日:2014-12-30

    申请号:US13770212

    申请日:2013-02-19

    IPC分类号: G06F9/44 G06F11/36

    CPC分类号: G06F11/362 G06F11/3636

    摘要: Method product for performing an operation, the operation including providing a plurality of processing elements comprising one or more operators, the operators configured to process streaming data tuples. The operation then establishes an operator graph of multiple operators, the operator graph defining at least one execution path in which a first operator of the plurality of operators is configured to receive data tuples from at least one upstream operator and transmit data tuples to at least one downstream operator. The operation then defines a breakpoint, the breakpoint comprising a condition, the condition based on attribute values of data tuples in a window of at least one operator, the window comprising a plurality of data tuples in an operator. The operation, upon detecting occurrence of the condition, triggers the breakpoint to halt processing by each of the plurality of operators in the operator graph.

    摘要翻译: 用于执行操作的方法产品,所述操作包括提供包括一个或多个操作者的多个处理元件,所述操作器被配置为处理流数据元组。 操作然后建立多个运算符的运算符图,运算符图定义至少一个执行路径,其中多个运算符中的第一运算符被配置为从至少一个上游运算符接收数据元组,并将数据元组传输到至少一个 下游运营商。 操作然后定义断点,断点包括条件,基于至少一个操作者的窗口中的数据元组的属性值的条件,该窗口包括操作者中的多个数据元组。 该操作在检测到条件的发生时触发断点以停止运算符图中的多个运算符中的每一个的处理。

    Handling out-of-sequence data in a streaming environment
    6.
    发明授权
    Handling out-of-sequence data in a streaming environment 有权
    处理流媒体环境中的乱序数据

    公开(公告)号:US08903866B2

    公开(公告)日:2014-12-02

    申请号:US13775714

    申请日:2013-02-25

    IPC分类号: G06F17/30

    CPC分类号: G06F17/30958 G06F17/30516

    摘要: Computer-implemented method, system, and computer program product for processing data in an out-of-order manner in a streams computing environment. A windowing condition is defined such that incoming data tuples are processed within a specified time or count of each other. Additionally, the windowing condition may be based on a specified attribute of the data tuples. If the tuples are not processed within the constraints specified by the windowing condition, the unprocessed tuples may be discarded, i.e., not processed, to optimize operator performance.

    摘要翻译: 用于在流计算环境中以无序方式处理数据的计算机实现的方法,系统和计算机程序产品。 定义窗口条件,使得输入数据元组在指定的时间或彼此计数之间进行处理。 此外,窗口状态可以基于数据元组的指定属性。 如果元组在窗口条件指定的约束条件下未被处理,那么未处理的元组可能被丢弃,即不被处理,以优化操作者的性能。

    Distributed Control of a Heterogeneous Video Surveillance Network
    7.
    发明申请
    Distributed Control of a Heterogeneous Video Surveillance Network 有权
    异构视频监控网络的分布式控制

    公开(公告)号:US20140132763A1

    公开(公告)日:2014-05-15

    申请号:US13751013

    申请日:2013-01-25

    IPC分类号: H04N7/16

    摘要: A surveillance video broker arbitrates access by multiple clients to multiple surveillance video sources. Both clients and sources register with the broker. Each source independently specifies respective clients permitted real-time access to its video and conditions of access, if any. Preferably, the video source is a local surveillance domain having one or more cameras, one or more sensors, and a local controller, the source specifying clients or client groups permitted access, and independently specifying conditions of access for each client or client group, where conditions may include scheduled events, non-scheduled events, such as alarms or emergencies, and/or physical proximity. The broker automatically authorizes real-time access according to pre-specified conditions. Preferably, the broker can also arbitrate alert notifications to the clients based on pre-specified notification criteria.

    摘要翻译: 监控视频代理对多个客户端访问多个监控视频源进行仲裁。 客户端和来源都向代理注册。 每个源独立地指定允许实时访问其视频和访问条件的相应客户端(如果有的话)。 优选地,视频源是具有一个或多个照相机,一个或多个传感器和本地控制器的本地监视域,指定客户端或客户端组允许的访问的源,以及独立地指定每个客户端或客户端组的访问条件,其中 条件可以包括调度事件,非计划事件,例如警报或紧急情况,和/或物理接近。 经纪人根据预先指定的条件自动授权实时访问。 优选地,代理还可以基于预先指定的通知标准来仲裁向客户端的警报通知。

    MONITORING STREAMS BUFFERING TO OPTIMIZE OPERATOR PROCESSING

    公开(公告)号:US20130290489A1

    公开(公告)日:2013-10-31

    申请号:US13710570

    申请日:2012-12-11

    IPC分类号: H04L29/08

    CPC分类号: H04L67/104 H04L47/25

    摘要: Method, system and computer program product for performing an operation, including providing a plurality of processing elements comprising one or more operators, the operators configured to process streaming data tuples, establishing an operator graph of multiple operators, the operator graph defining at least one execution path in which a first operator is configured to receive data tuples from at least one upstream operator and transmit data tuples to at least one downstream operator, providing each operator a buffer configured to hold data tuples requiring processing by the respective operator, wherein the buffer is a first-in-first-out buffer, receiving a plurality of data tuples in a buffer associated with an operator, the data tuples comprising at least one attribute, selecting at least one data tuple from the first buffer, examining an attribute of the selected data tuples to identify a candidate tuple, and performing a second operation on the candidate tuple.

    PROCESSING ELEMENT MANAGEMENT IN A STREAMING DATA SYSTEM
    9.
    发明申请
    PROCESSING ELEMENT MANAGEMENT IN A STREAMING DATA SYSTEM 有权
    流程数据系统中的处理元素管理

    公开(公告)号:US20130198489A1

    公开(公告)日:2013-08-01

    申请号:US13709405

    申请日:2012-12-10

    IPC分类号: G06F9/44

    摘要: Stream applications may inefficiently use the hardware resources that execute the processing elements of the data stream. For example, a compute node may host four processing elements and execute each using a CPU. However, other CPUs on the compute node may sit idle. To take advantage of these available hardware resources, a stream programmer may identify one or more processing elements that may be cloned. The cloned processing elements may be used to generate a different execution path that is parallel to the execution path that includes the original processing elements. Because the cloned processing elements contain the same operators as the original processing elements, the data stream that was previously flowing through only the original processing element may be split and sent through both the original and cloned processing elements. In this manner, the parallel execution path may use underutilized hardware resources to increase the throughput of the data stream.

    摘要翻译: 流应用可能会低效地使用执行数据流处理元素的硬件资源。 例如,计算节点可以托管四个处理元件,并使用CPU执行每个处理元件。 但是,计算节点上的其他CPU可能会处于空闲状态。 为了利用这些可用的硬件资源,流程序员可以识别可被克隆的一个或多个处理元件。 克隆的处理元件可以用于生成与包括原始处理元件的执行路径平行的不同执行路径。 因为克隆的处理元素包含与原始处理元素相同的操作符,所以先前仅流过原始处理元素的数据流可以被分割并通过原始和克隆处理元素发送。 以这种方式,并行执行路径可以使用未充分利用的硬件资源来增加数据流的吞吐量。

    USING PREDICTIVE DETERMINISM WITHIN A STREAMING ENVIRONMENT
    10.
    发明申请
    USING PREDICTIVE DETERMINISM WITHIN A STREAMING ENVIRONMENT 有权
    在流动环境中使用预测性决定

    公开(公告)号:US20130080653A1

    公开(公告)日:2013-03-28

    申请号:US13681919

    申请日:2012-11-20

    IPC分类号: H04L29/06

    CPC分类号: H04L65/608 G06F9/06 G06F15/17

    摘要: Techniques are described for transmitting predicted output data on a processing element in a stream computing application instead of processing currently received input data. The stream computing application monitors the output of a processing element and determines whether its output is predictable, for example, if the previously transmitted output values are within a predefined range or if one or more input values correlate with the same one or more output values. The application may then generate a predicted output value to transmit from the processing element instead of transmitting a processed output value based on current input values. The predicted output value may be, for example, an average of the previously transmitted output values or a previously transmitted output value that was transmitted in response to a previously received input value that is similar to a currently received input value.

    摘要翻译: 描述了用于在流计算应用中的处理元件上传送预测输出数据而不是处理当前接收到的输入数据的技术。 流计算应用程序监视处理元件的输出并确定其输出是否可预测,例如,如果先前发送的输出值在预定范围内,或者如果一个或多个输入值与相同的一个或多个输出值相关联。 然后,应用程序可以生成预测的输出值以从处理元件发送,而不是基于当前输入值发送经处理的输出值。 预测的输出值可以是例如先前发送的输出值的平均值或响应于与当前接收到的输入值相似的先前接收到的输入值而发送的先前发送的输出值。