Handling modifications in programming of an iterative message processing system

    公开(公告)号:US11474673B1

    公开(公告)日:2022-10-18

    申请号:US16945657

    申请日:2020-07-31

    Applicant: Splunk Inc.

    Abstract: Systems and methods are described for graphical user interfaces that enable users to graphically design nodes and interconnections of pipelines for preprocessing data later ingested into an indexing system. The preprocessing can include receiving messages published to a first publish-subscribe messaging system, the messages containing raw machine data generated by one or more components in an information technology environment, performing one or more processing operations on at least some of the messages to generate preprocessed messages, republishing the preprocessed messages to a second publish-subscribe messaging system, and providing to the indexing system, a subset of the messages from the second publish-subscribe messaging system. Nodes in the pipeline can specify the preprocessing operations, and interconnections in the pipeline can represent data flow through the nodes of the pipeline.

    Iterative message-based data processing including streaming analytics

    公开(公告)号:US11269939B1

    公开(公告)日:2022-03-08

    申请号:US15967581

    申请日:2018-04-30

    Applicant: Splunk Inc.

    Abstract: Systems and methods are described for processing data by interactions with a publish-subscribe messaging system. The processing may include receiving messages published to at least one publish-subscribe messaging system, the messages containing data collected during operation of one or more source computing systems and iteratively processing the received messages through the publish-subscribe messaging system, wherein iteratively processing a message comprises serially republishing the message to the at least one publish-subscribe messaging system and altering the message between individual republishings according to a set of rules, and wherein a final republishing of the message during the serial republishing makes a version of the message available on the at least one publish-subscribe messaging system to a downstream component.

    Assisted visual programming for iterative message processing system

    公开(公告)号:US11194552B1

    公开(公告)日:2021-12-07

    申请号:US16945590

    申请日:2020-07-31

    Applicant: Splunk Inc.

    Abstract: Systems and methods are described for graphical user interfaces that enable users to graphically design nodes and interconnections of pipelines for preprocessing data later ingested into an indexing system. The preprocessing can include receiving messages published to a first publish-subscribe messaging system, the messages containing raw machine data generated by one or more components in an information technology environment, performing one or more processing operations on at least some of the messages to generate preprocessed messages, republishing the preprocessed messages to a second publish-subscribe messaging system, and providing to the indexing system, a subset of the messages from the second publish-subscribe messaging system. Nodes in the pipeline can specify the preprocessing operations, and interconnections in the pipeline can represent data flow through the nodes of the pipeline.

    Unified data processing across streaming and indexed data sets

    公开(公告)号:US10936585B1

    公开(公告)日:2021-03-02

    申请号:US16177234

    申请日:2018-10-31

    Applicant: Splunk Inc.

    Abstract: Systems and methods are described for unified processing of indexed and streaming data. A system enables users to query indexed data or specify processing pipelines to be applied to streaming data. In some instances, a user may specify a query intended to be run against indexed data, but may specify criteria that includes not-yet-indexed data (e.g., a future time frame). The system may convert the query into a data processing pipeline applied to not-yet-indexed data, thus increasing the efficiency of the system. Similarly, in some instances, a user may specify a data processing pipeline to be applied to a data stream, but specify criteria including data items outside the data stream. For example, a user may wish to apply the pipeline retroactively, to data items that have already exited the data stream. The system can convert the pipeline into a query against indexed data to satisfy the users processing requirements.

    DYNAMIC QUERY PROCESSOR FOR STREAMING AND BATCH QUERIES

    公开(公告)号:US20190236194A1

    公开(公告)日:2019-08-01

    申请号:US15885645

    申请日:2018-01-31

    Applicant: Splunk Inc.

    CPC classification number: G06F16/24568 G06F16/24542

    Abstract: Operational machine components of an information technology (IT) or other microprocessor- or microcontroller-permeated environment generate disparate forms of machine data. Network connections are established between these components and processors of data intake and query system (DIQS). The DIQS conducts network transactions on a periodic and/or continuous basis with the machine components to receive disparate data and ingest certain of the data as entries of a data store that is searchable for DIQS query processing. The DIQS may receive queries to process against the received and ingested data via an exposed network interface. In one example embodiment, the DIQS receives a query identifying data to be processed, dynamically generates a query processing scheme based on the state of the data to be processed, such as streaming or at rest, and dynamically communicates the query processing scheme to a query executor based on the state of the data to be processed.

    Data stream generation based on sourcetypes associated with messages

    公开(公告)号:US11853303B1

    公开(公告)日:2023-12-26

    申请号:US17411357

    申请日:2021-08-25

    Applicant: SPLUNK Inc.

    Abstract: As described herein, a portion of machine data of a message may be analyzed to infer, using an inference model, a sourcetype of the message. The portion of machine data may be generated by one or more components in an information technology environment. Based on the inference, a set of extraction rules associated with the sourcetype may be selected. Each extraction rule may define criteria for identifying a sub-portion of text from the portion of machine data of the message to produce a value. The set of extraction rules may be applied to the portion of machine data of the message to produce a result set that indicates a number of values identified using the set of extraction rules. Based on the result set, at least one action may be performed on one or more of inference data associated with the inference model and one or more messages.

    Feedback on inferred sourcetypes
    9.
    发明授权

    公开(公告)号:US11748358B2

    公开(公告)日:2023-09-05

    申请号:US16175642

    申请日:2018-10-30

    Applicant: Splunk, Inc.

    Abstract: As described herein, a portion of machine data of a message may be analyzed to infer, using an inference model, a sourcetype of the message. The portion of machine data may be generated by one or more components in an information technology environment. Based on the inference, a set of extraction rules associated with the sourcetype may be selected. Each extraction rule may define criteria for identifying a sub-portion of text from the portion of machine data of the message to produce a value. The set of extraction rules may be applied to the portion of machine data of the message to produce a result set that indicates a number of values identified using the set of extraction rules. Based on the result set, at least one action may be performed on one or more of inference data associated with the inference model and one or more messages.

    Visual programming for iterative publish-subscribe message processing system

    公开(公告)号:US10776441B1

    公开(公告)日:2020-09-15

    申请号:US16148840

    申请日:2018-10-01

    Applicant: Splunk Inc.

    Abstract: Systems and methods are described for graphical user interfaces that enable users to graphically design nodes and interconnections of pipelines for preprocessing data later ingested into an indexing system. The preprocessing can include receiving messages published to a first publish-subscribe messaging system, the messages containing raw machine data generated by one or more components in an information technology environment, performing one or more processing operations on at least some of the messages to generate preprocessed messages, republishing the preprocessed messages to a second publish-subscribe messaging system, and providing to the indexing system, a subset of the messages from the second publish-subscribe messaging system. Nodes in the pipeline can specify the preprocessing operations, and interconnections in the pipeline can represent data flow through the nodes of the pipeline.

Patent Agency Ranking