公开(公告)号：US20190258637A1

公开(公告)日：2019-08-22

申请号：US16397970

申请日：2019-04-29

Applicant: Splunk Inc.

Inventor： Arindam Bhattacharjee ,  Sourav Pal ,  Wayne Patterson ,  Srinivas Bobba

IPC: G06F16/2453 ,  G06F16/22 ,  G06F16/27

Abstract: Systems and methods are described for partitioning and reducing records at ingest of a worker node. The worker node receives chunks of data from one or more indexers of a data intake and query system based on the execution of a query by the data intake and query system. The worker node assigns records to different record groups based on the content of the records. The system also assigns the record to a partition of a group of partitions. Record data of the records in a particular partition is combined. The system processes the partitions based on the query.

公开(公告)号：US11989194B2

公开(公告)日：2024-05-21

申请号：US16657867

申请日：2019-10-18

Applicant: Splunk Inc.

Inventor： Arindam Bhattacharjee ,  Sourav Pal ,  Srinivas Bobba

IPC: G06F16/2458 ,  G06F16/27

CPC classification number: G06F16/2471 ,  G06F16/278

Abstract: Systems and methods are described for distributed processing a query in a first query language utilizing a query execution engine intended for single-device execution. While distributed processing provides numerous benefits over single-device processing, distributed query execution engines can be significantly more difficult to develop that single-device engines. Embodiments of this disclosure enable the use of a single-device engine to support distributed processing, by dividing a query into multiple stages, each of which can be executed by multiple, concurrent executions of a single-device engine. Between stages, data can be shuffled between executions of the engine, such that individual executions of the engine are provided with a complete set of records needed to implement an individual stage. Because single-device engines can be significantly less difficult to develop, use of the techniques described herein can enable a distributed system to rapidly support multiple query languages.

公开(公告)号：US20240320231A1

公开(公告)日：2024-09-26

申请号：US18626007

申请日：2024-04-03

Applicant: Splunk Inc.

Inventor： Arindam Bhattacharjee ,  Sourav Pal ,  Srinivas Bobba

IPC: G06F16/2458 ,  G06F16/27

CPC classification number: G06F16/2471 ,  G06F16/278

Abstract: Systems and methods are described for distributed processing a query in a first query language utilizing a query execution engine intended for single-device execution. While distributed processing provides numerous benefits over single-device processing, distributed query execution engines can be significantly more difficult to develop that single-device engines. Embodiments of this disclosure enable the use of a single-device engine to support distributed processing, by dividing a query into multiple stages, each of which can be executed by multiple, concurrent executions of a single-device engine. Between stages, data can be shuffled between executions of the engine, such that individual executions of the engine are provided with a complete set of records needed to implement an individual stage. Because single-device engines can be significantly less difficult to develop, use of the techniques described herein can enable a distributed system to rapidly support multiple query languages.

公开(公告)号：US20200065303A1

公开(公告)日：2020-02-27

申请号：US16657867

申请日：2019-10-18

Applicant: Splunk Inc.

Inventor： Arindam Bhattacharjee ,  Sourav Pal ,  Srinivas Bobba

IPC: G06F16/2458 ,  G06F16/242 ,  G06F16/27 ,  G06F11/07 ,  G06F11/30 ,  G06F9/50

Abstract: Systems and methods are described for distributed processing a query in a first query language utilizing a query execution engine intended for single-device execution. While distributed processing provides numerous benefits over single-device processing, distributed query execution engines can be significantly more difficult to develop that single-device engines. Embodiments of this disclosure enable the use of a single-device engine to support distributed processing, by dividing a query into multiple stages, each of which can be executed by multiple, concurrent executions of a single-device engine. Between stages, data can be shuffled between executions of the engine, such that individual executions of the engine are provided with a complete set of records needed to implement an individual stage. Because single-device engines can be significantly less difficult to develop, use of the techniques described herein can enable a distributed system to rapidly support multiple query languages.

公开(公告)号：US11593377B2

公开(公告)日：2023-02-28

申请号：US16397922

申请日：2019-04-29

Applicant: Splunk Inc.

Inventor： Arindam Bhattacharjee ,  Sourav Pal ,  Srinivas Bobba

IPC: G06F16/00 ,  G06F16/2455 ,  G06F16/2458

Abstract: Systems and methods are described for assigning a processing task from one component of a data intake and query system to a different component of the data intake and query system. As part of processing a query, the system can determine that a particular processing task is to be executed by a particular component of the data intake and query system. Based on the characteristics of the component that is to execute the processing task, the system can assign the task or a supplemental task to one or more other components of the data intake and query system.

公开(公告)号：US11586627B2

公开(公告)日：2023-02-21

申请号：US16397970

申请日：2019-04-29

Applicant: Splunk Inc.

Inventor： Arindam Bhattacharjee ,  Sourav Pal ,  Wayne Patterson ,  Srinivas Bobba

IPC: G06F16/00 ,  G06F16/2453 ,  G06F16/27 ,  G06F16/22

Abstract: Systems and methods are described for partitioning and reducing records at ingest of a worker node. The worker node receives chunks of data from one or more indexers of a data intake and query system based on the execution of a query by the data intake and query system. The worker node assigns records to different record groups based on the content of the records. The system also assigns the record to a partition of a group of partitions. Record data of the records in a particular partition is combined. The system processes the partitions based on the query.

公开(公告)号：US20190272271A1

公开(公告)日：2019-09-05

申请号：US16397922

申请日：2019-04-29

Applicant: Splunk Inc.

Inventor： Arindam Bhattacharjee ,  Sourav Pal ,  Srinivas Bobba

IPC: G06F16/2455 ,  G06F16/2458

Abstract: Systems and methods are described for assigning a processing task from one component of a data intake and query system to a different component of the data intake and query system. As part of processing a query, the system can determine that a particular processing task is to be executed by a particular component of the data intake and query system. Based on the characteristics of the component that is to execute the processing task, the system can assign the task or a supplemental task to one or more other components of the data intake and query system.