-
公开(公告)号：US20200068051A1
公开(公告)日：2020-02-27
申请号：US16668808
申请日：2019-10-30
申请人： Splunk Inc.
IPC分类号： H04L29/08
摘要： Systems and methods for multi-thread processing of messages. An example method comprises: receiving, by a first processing thread, a plurality of network packets from a server; processing the plurality of network packets to produce a message; writing the message to a message queue; retrieving, by a second processing thread, the message from the message queue; producing a memory data structure based on the message; placing the memory data structure into a result queue; and responsive to determining that a total size of messages in the message queue exceeds a certain threshold, causing the first processing thread to suspend receiving network packets.
-
公开(公告)号：US20200050612A1
公开(公告)日：2020-02-13
申请号：US16657916
申请日：2019-10-18
申请人： Splunk Inc.
发明人： Arindam Bhattacharjee , Sourav Pal , Timothy Tully
IPC分类号： G06F16/28 , G06F16/2452
摘要： Systems and methods are described for distributed processing a query in a first query language utilizing a query execution engine intended for single-device execution. While distributed processing provides numerous benefits over single-device processing, distributed query execution engines can be significantly more difficult to develop that single-device engines. Embodiments of this disclosure enable the use of a single-device engine to support distributed processing, by dividing a query into multiple stages, each of which can be executed by multiple, concurrent executions of a single-device engine. Between stages, data can be shuffled between executions of the engine, such that individual executions of the engine are provided with a complete set of records needed to implement an individual stage. Because single-device engines can be significantly less difficult to develop, use of the techniques described herein can enable a distributed system to rapidly support multiple query languages.
-
公开(公告)号：US20190310977A1
公开(公告)日：2019-10-10
申请号：US16398038
申请日：2019-04-29
申请人： Splunk Inc.
发明人： Sourav Pal , Arindam Bhattacharjee , Asha Andrade , Nikhil Roy
IPC分类号： G06F16/2455 , G06F16/22 , G06F9/50
摘要： Systems and methods are described for exporting bucket data from one or more buckets to one or more worker nodes. The system can identify data from different bucket data from buckets stored in a data intake and query system that is to be processed by one or more worker nodes. The system can allocate one or more execution resources, such as a processing pipeline, to process and export the bucket data from the buckets. The system can assign bucket data corresponding to individual buckets to the execution resource based on a bucket distribution policy. The indexer can export the bucket data to the worker nodes for further processing based on the bucket data-execution resource assignment.
-
24.发明申请公开(公告)号：US20190258636A1
公开(公告)日：2019-08-22
申请号：US16397968
申请日：2019-04-29
申请人： Splunk Inc.
IPC分类号： G06F16/2453 , G06F16/22
摘要： Systems and methods are described for processing records associated with a query that identifies an association between two data fields. The system can obtain a chunk of data that includes multiple records based on a query received by a data intake and query system. At least one record can include multiple sub-records that share a field value for at least one field. The system can generate a record from each sub-record and assign the generated records to one or more groups of partitions. The system can combine record data of generated records assigned to one partition of a group of partitions and then combine record data across the group of partitions. The system can process the results of the combination of records across the group of partitions based on the query.
-
公开(公告)号：US20190171677A1
公开(公告)日：2019-06-06
申请号：US16264441
申请日：2019-01-31
申请人： Splunk Inc.
发明人： Sourav Pal , Christopher Pride , Arindam Bhattacharjee , Xiaowei Wang , James Alasdair Robert Hodge , Mustafa Ahamed
IPC分类号： G06F16/951 , G06F16/21 , G06F16/903 , G06F16/9038 , G06F16/904 , G06F16/25 , G06F16/901
摘要： Disclosed is a technique that can be performed in a distributed network. The technique can include a search service system that receives an indication of at least a portion of a search scheme to cause worker nodes to obtain search results from distributed data storage systems. The search scheme is defined by a data intake and query system. The search service system defines a search process based on the at least a portion of the search scheme and executes the search process to cause the worker nodes to obtain search results from the distributed data storage systems. The search service system receives a combination of search results based on the search results obtained by the worker nodes from the distributed data storage systems, and causes an output based on the combination of search results obtained by the data intake and query system in accordance with the search scheme.
-
公开(公告)号：US20190163822A1
公开(公告)日：2019-05-30
申请号：US15339833
申请日：2016-10-31
申请人： Splunk Inc.
IPC分类号： G06F17/30
CPC分类号： G06F16/951 , G06F16/211 , G06F16/212 , G06F16/2455 , G06F16/2471 , G06F16/248 , G06F16/252 , G06F16/258 , G06F16/27 , G06F16/9024 , G06F16/90335 , G06F16/9038 , G06F16/904
摘要： The disclosed embodiments include techniques to obtain ordered search results based on partial search results from across multiple diverse internal and/or external data sources. The ordering of the search results may be with respect to a parameter associated with the partial search results. An example of a parameter includes time. As such, the disclosed technique can provide a time-ordered search result based on partial search results obtained from across multiple internal and/or external data sources. Moreover, the disclosed technique can provide time-ordered search results regardless of whether the partial search results obtained from the diverse data sources are timestamped.
-
公开(公告)号：US20190147085A1
公开(公告)日：2019-05-16
申请号：US16146990
申请日：2018-09-28
申请人： Splunk Inc.
发明人： Sourav Pal , Arindam Bhattacharjee
IPC分类号： G06F17/30
摘要： Systems and methods are disclosed for receiving, at a data intake and query system, a query that includes a subquery that is to be executed at an external data system that supports a different query language than the data intake and query system. The data intake and query system converts the subquery from the query language supported by the external data system to the query language supported by the data intake and query system. The data intake and query system then processes the query including the translated subquery. The translated subquery is then translated back to the language supported by the external data system including any processing or optimizations performed with respect to the subquery.
-
公开(公告)号：US20190138638A1
公开(公告)日：2019-05-09
申请号：US16051300
申请日：2018-07-31
申请人： Splunk Inc.
发明人： Sourav Pal , Arindam Bhattacharjee
摘要： Systems and methods are disclosed for processing data chunks from different data sources at an execution node in a distributed execution environment. The execution node receives data chunks from different sources and combines data from groups of data chunks into partitions based on an associated data source. The execution node executes the partitions using one or more processors.
-
公开(公告)号：US09813528B2
公开(公告)日：2017-11-07
申请号：US14448928
申请日：2014-07-31
申请人： Splunk Inc.
CPC分类号： H04L69/329 , H04L67/02 , H04L67/2804 , H04L67/2819 , H04L69/326
摘要： Systems and methods for priority-based processing of messages received from multiple servers. An example method may comprise: receiving, by a first processing thread, in a non-blocking mode, a plurality of sub-application layer protocol packets from a plurality of servers; processing one or more sub-application layer protocol packets received from a first server of the plurality of servers, to produce a first application layer message; writing the first application layer message to a first message queue of a plurality of message queues associated with the plurality of servers, the first message queue corresponding to the first server; processing one or more sub-application layer protocol packets received from a second server of the plurality of servers, to produce a second application layer message; writing the second application layer message to a second message queue of a plurality of message queues associated with the plurality of servers, the second message queue corresponding to the second server; and reading, by a second processing thread, an application layer message having a most recent timestamp among a plurality of application layer messages in the plurality of message queues, the plurality of application layer messages including the first application layer message and the second application layer message.
-
公开(公告)号：US20160036716A1
公开(公告)日：2016-02-04
申请号：US14448928
申请日：2014-07-31
申请人： Splunk Inc.
IPC分类号： H04L12/863 , H04L29/08 , H04L12/865
CPC分类号： H04L69/329 , H04L67/02 , H04L67/2804 , H04L67/2819 , H04L69/326
摘要： Systems and methods for priority-based processing of messages received from multiple servers. An example method may comprise: receiving, by a first processing thread, in a non-blocking mode, a plurality of sub-application layer protocol packets from a plurality of servers; processing one or more sub-application layer protocol packets received from a first server of the plurality of servers, to produce a first application layer message; writing the first application layer message to a first message queue of a plurality of message queues associated with the plurality of servers, the first message queue corresponding to the first server; processing one or more sub-application layer protocol packets received from a second server of the plurality of servers, to produce a second application layer message; writing the second application layer message to a second message queue of a plurality of message queues associated with the plurality of servers, the second message queue corresponding to the second server; and reading, by a second processing thread, an application layer message having a most recent timestamp among a plurality of application layer messages in the plurality of message queues, the plurality of application layer messages including the first application layer message and the second application layer message.
摘要翻译： 用于基于优先级处理从多个服务器接收的消息的系统和方法。 示例性方法可以包括：由第一处理线程以非阻塞模式从多个服务器接收多个子应用层协议分组; 处理从所述多个服务器的第一服务器接收的一个或多个子应用层协议分组，以产生第一应用层消息; 将所述第一应用层消息写入与所述多个服务器相关联的多个消息队列的第一消息队列，所述第一消息队列对应于所述第一服务器; 处理从所述多个服务器的第二服务器接收的一个或多个子应用层协议分组，以产生第二应用层消息; 将所述第二应用层消息写入到与所述多个服务器相关联的多个消息队列的第二消息队列中，所述第二消息队列对应于所述第二服务器; 以及通过第二处理线程读取在所述多个消息队列中的多个应用层消息中具有最新时间戳的应用层消息，所述多个应用层消息包括所述第一应用层消息和所述第二应用层消息 。
-
-
-
-
-
-
-
-
-
搜索结果
159 条记录 (耗时 0.102 秒)
