公开(公告)号：US20180007180A1

公开(公告)日：2018-01-04

申请号：US15703209

申请日：2017-09-13

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Christopher Madden Pride

IPC: H04L29/08

CPC classification number: H04L69/329 ,  H04L67/02 ,  H04L67/2804 ,  H04L67/2819 ,  H04L69/326

Abstract: Systems and methods for priority-based processing of messages received from multiple servers. An example method comprises: receiving a plurality of network packets from one or more servers; processing the plurality of network packets to produce a first message associated with a first timestamp and a second message associated with a second timestamp; writing the first message to a first message queue of a plurality of message queues; writing the second message to a second message queue of the plurality of message queues; and retrieving, from the plurality of message queues, the first message and the second message in an order of their respective associated timestamps.

公开(公告)号：US20220382755A1

公开(公告)日：2022-12-01

申请号：US17652620

申请日：2022-02-25

Applicant: Splunk Inc.

Inventor： Alexandros Batsakis ,  Scott Calvert ,  Alexander Douglas James ,  Bei Li ,  Ashish Mathew ,  James Monschke ,  Sogol Moshtaghi ,  Christopher Madden Pride ,  Xiaowei Wang

IPC: G06F16/2453 ,  G06F16/13

Abstract: Systems and methods are disclosed for mapping search nodes to a search head in a data intake and query system based on a tenant identifier in order to execute a query received by the data intake and query system. The mapping may allow same or similar search nodes to be used to execute queries that are associated with a particular tenant identifier, in order to take advantage of caching and local data stored with those search nodes. In some cases, search nodes can be mapped based on the tenant identifier using a hashing algorithm, such as a consistent hashing algorithm.

公开(公告)号：US20220156267A1

公开(公告)日：2022-05-19

申请号：US17586590

申请日：2022-01-27

Applicant: Splunk Inc.

Inventor： Alexander Douglas James ,  Scott Calvert ,  Manu Jose ,  Andrew Peters ,  Christopher Madden Pride ,  Arun Ramani

IPC: G06F16/2457 ,  G06F16/907 ,  G06F16/2455 ,  G06F40/30

Abstract: Systems and methods are disclosed for annotating a metadata catalog in a data intake and query system based on a query received by the data intake and query system. The metadata catalog can store information about datasets associated with the data intake and query system, including dataset configuration records of the datasets, which can be used to process queries for execution by the data intake and query system. The data intake and query system can receive a query identifying a set of data to be processed and a manner of processing the set of data. The data intake and query system can parse the query to identify datasets and/or data fields associated with the query. Based on the identified datasets and/or fields, the data intake and query system can generate one or more annotations, and use the annotations to update the metadata catalog.

公开(公告)号：US11238049B1

公开(公告)日：2022-02-01

申请号：US16264019

申请日：2019-01-31

Applicant: Splunk Inc.

Inventor： Alexander Douglas James ,  Scott Calvert ,  Manu Jose ,  Andrew Peters ,  Christopher Madden Pride ,  Arun Ramani

IPC: G06F16/00 ,  G06F16/2457 ,  G06F16/907 ,  G06F16/2455 ,  G06F40/30

Abstract: Systems and methods are disclosed for annotating a metadata catalog in a data intake and query system based on a query received by the data intake and query system. The metadata catalog can store information about datasets associated with the data intake and query system, including dataset configuration records of the datasets, which can be used to process queries for execution by the data intake and query system. The data intake and query system can receive a query identifying a set of data to be processed and a manner of processing the set of data. The data intake and query system can parse the query to identify datasets and/or data fields associated with the query. Based on the identified datasets and/or fields, the data intake and query system can generate one or more annotations, and use the annotations to update the metadata catalog.

公开(公告)号：US11157497B1

公开(公告)日：2021-10-26

申请号：US16513555

申请日：2019-07-16

Applicant: Splunk Inc.

Inventor： Alexandros Batsakis ,  Scott Calvert ,  Alexander Douglas James ,  Bei Li ,  Ashish Mathew ,  James Monschke ,  Sogol Moshtaghi ,  Christopher Madden Pride ,  Xiaowei Wang

IPC: G06F16/00 ,  G06F16/2453

Abstract: Systems and methods are disclosed for dynamically assigning a search head or search nodes in a data intake and query system for a query received by the data intake and query system. Existing search heads and search nodes can periodically report their status to the data intake and query system, which can use that information to help determine the need to provision additional search heads and search nodes. The data intake and query system can receive a query identifying a set of data to be processed and a manner of processing the set of data. The data intake and query system can use the status information for existing search heads and search nodes to dynamically assign a search head and search nodes for the query. Dynamically assigning the search head and search nodes in this manner may provide many benefits, including improved load balancing and resource utilization.

公开(公告)号：US11093564B1

公开(公告)日：2021-08-17

申请号：US16147129

申请日：2018-09-28

Applicant: Splunk Inc.

Inventor： Alexander Douglas James ,  Manu Jose ,  Sourav Pal ,  Christopher Madden Pride ,  Nicholas Robert Romito ,  Igor Braylovskiy ,  Arun Ramani ,  Ankit Jain

IPC: G06F16/00 ,  G06F16/9535 ,  G06F9/54 ,  G06F16/242 ,  G06F40/205

Abstract: Systems and methods are disclosed for processing and executing queries in a data intake and query system. The data intake and query system receives a query identifying a set of data to be processed and a manner of processing the set of data. The data intake and query system parses the query and uses a metadata catalog to dynamically identify configuration parameters of datasets and/or rules associated with the query. The identified configuration parameters are communicated to a query processing component of the data intake and query system for use in executing the query.

公开(公告)号：US20200167395A1

公开(公告)日：2020-05-28

申请号：US16777602

申请日：2020-01-30

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Christopher Madden Pride ,  Arindam Bhattacharjee ,  Xiaowei Wang ,  James Alasdair Robert Hodge ,  Mustafa Ahamed

IPC: G06F16/951 ,  G06F16/25 ,  G06F16/2455 ,  G06F16/27 ,  G06F16/2458 ,  G06F16/248 ,  G06F16/21 ,  G06F16/903 ,  G06F16/9038 ,  G06F16/901 ,  G06F16/904

Abstract: Disclosed is a data fabric service system that can be implemented in a distributed computer network, such as a data intake and query system. The data index and query system can receive a search query and define a search scheme for applying the search query on distributed data storage systems including internal data storage and external data storage. The data index and query system may provide a portion of the search scheme to a search service of the data fabric service system, which can cause worker nodes of the data fabric service system to perform various functions—including applying the search query to the external data storage based on the portion of the search scheme in order to obtain search results.

公开(公告)号：US20200068051A1

公开(公告)日：2020-02-27

申请号：US16668808

申请日：2019-10-30

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Christopher Madden Pride

IPC: H04L29/08

Abstract: Systems and methods for multi-thread processing of messages. An example method comprises: receiving, by a first processing thread, a plurality of network packets from a server; processing the plurality of network packets to produce a message; writing the message to a message queue; retrieving, by a second processing thread, the message from the message queue; producing a memory data structure based on the message; placing the memory data structure into a result queue; and responsive to determining that a total size of messages in the message queue exceeds a certain threshold, causing the first processing thread to suspend receiving network packets.

公开(公告)号：US09813528B2

公开(公告)日：2017-11-07

申请号：US14448928

申请日：2014-07-31

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Christopher Madden Pride

IPC: G06F15/16 ,  H04L29/08

CPC classification number: H04L69/329 ,  H04L67/02 ,  H04L67/2804 ,  H04L67/2819 ,  H04L69/326

Abstract: Systems and methods for priority-based processing of messages received from multiple servers. An example method may comprise: receiving, by a first processing thread, in a non-blocking mode, a plurality of sub-application layer protocol packets from a plurality of servers; processing one or more sub-application layer protocol packets received from a first server of the plurality of servers, to produce a first application layer message; writing the first application layer message to a first message queue of a plurality of message queues associated with the plurality of servers, the first message queue corresponding to the first server; processing one or more sub-application layer protocol packets received from a second server of the plurality of servers, to produce a second application layer message; writing the second application layer message to a second message queue of a plurality of message queues associated with the plurality of servers, the second message queue corresponding to the second server; and reading, by a second processing thread, an application layer message having a most recent timestamp among a plurality of application layer messages in the plurality of message queues, the plurality of application layer messages including the first application layer message and the second application layer message.

公开(公告)号：US20160036716A1

公开(公告)日：2016-02-04

申请号：US14448928

申请日：2014-07-31

Applicant: Splunk Inc.

Inventor： Sourav Pal ,  Christopher Madden Pride

IPC: H04L12/863 ,  H04L29/08 ,  H04L12/865

CPC classification number: H04L69/329 ,  H04L67/02 ,  H04L67/2804 ,  H04L67/2819 ,  H04L69/326

Abstract: Systems and methods for priority-based processing of messages received from multiple servers. An example method may comprise: receiving, by a first processing thread, in a non-blocking mode, a plurality of sub-application layer protocol packets from a plurality of servers; processing one or more sub-application layer protocol packets received from a first server of the plurality of servers, to produce a first application layer message; writing the first application layer message to a first message queue of a plurality of message queues associated with the plurality of servers, the first message queue corresponding to the first server; processing one or more sub-application layer protocol packets received from a second server of the plurality of servers, to produce a second application layer message; writing the second application layer message to a second message queue of a plurality of message queues associated with the plurality of servers, the second message queue corresponding to the second server; and reading, by a second processing thread, an application layer message having a most recent timestamp among a plurality of application layer messages in the plurality of message queues, the plurality of application layer messages including the first application layer message and the second application layer message.

Abstract translation: 用于基于优先级处理从多个服务器接收的消息的系统和方法。 示例性方法可以包括：由第一处理线程以非阻塞模式从多个服务器接收多个子应用层协议分组; 处理从所述多个服务器的第一服务器接收的一个或多个子应用层协议分组，以产生第一应用层消息; 将所述第一应用层消息写入与所述多个服务器相关联的多个消息队列的第一消息队列，所述第一消息队列对应于所述第一服务器; 处理从所述多个服务器的第二服务器接收的一个或多个子应用层协议分组，以产生第二应用层消息; 将所述第二应用层消息写入到与所述多个服务器相关联的多个消息队列的第二消息队列中，所述第二消息队列对应于所述第二服务器; 以及通过第二处理线程读取在所述多个消息队列中的多个应用层消息中具有最新时间戳的应用层消息，所述多个应用层消息包括所述第一应用层消息和所述第二应用层消息 。