Abstract:
The present disclosure is directed to providing dynamic indexer discovery. An index manager, which may also be known as a cluster master, is configured to track the statuses and capabilities of indexers and provide the statuses and capabilities obtained from the indexers to data collectors, such as forwarders. The data collectors may use the statuses and capabilities associated with the indexers to load balance transmission of data to the indexers. Dynamic indexer discovery may eliminate the need to manually reconfigure data collectors when the status of an indexer changes because the information may be obtained from the index manager without the need to reinitialize the data collectors.
Abstract:
Provided are systems and methods for verifying user credentials for performing a search. In one embodiment, a method can be provided that includes receiving a request to perform a search of machine generated data comprising time stamped events that is associated with a user, determining whether a set of cached user credentials has been updated within a period of time, querying, in response to determining that the credentials for the user have not been updated within the period of time, an identity provider server for a current set of user credentials associated with the user, receiving the current set of user credentials, determining whether the user has privileges to perform the search based at least in part on the set of user credentials, and causing, in response to determining that the user has privileges to perform the search, the search to be performed to identify one or more of the events that are responsive to the search.
Abstract:
The present invention is related to a method for providing dynamic indexer discovery. The method comprises receiving, from an index manager, a status indication associated with a plurality of indexers, wherein each of the plurality of indexers indexes events of raw machine-generated data received from a plurality of data collectors. The method further comprises determining a weight associated with each of the plurality of indexers and selecting an indexer from the plurality of indexers. Subsequently, the method comprises allocating data to the indexer in accordance with a respective weight assigned to the indexer and transmitting the allocated data to the indexer.
Abstract:
Disclosed herein is a data estimation technique for a data intake and query system. The system receives user inputs indicative that a first data source is to be the subject of a storage related estimate. The system receives a first plurality of events generated by the first data source. The system indexes only a sample of the received first plurality of events, based on a sampling criterion, where the sample is fewer than all of the first plurality of events. The system generates the storage related estimate based on at least some of the first plurality of events, and causes an indication of the estimate to be output to a user.
Abstract:
The present invention is related to a method for providing dynamic indexer discovery. The method comprises receiving, from an index manager, a status indication associated with a plurality of indexers, wherein each of the plurality of indexers indexes events of raw machine-generated data received from a plurality of data collectors. The method further comprises determining a weight associated with each of the plurality of indexers and selecting an indexer from the plurality of indexers. Subsequently, the method comprises allocating data to the indexer in accordance with a respective weight assigned to the indexer and transmitting the allocated data to the indexer.
Abstract:
Disclosed herein is a data estimation technique for a data intake and query system. The system receives user inputs indicative that a first data source is to be the subject of a storage related estimate. The system receives a first plurality of events generated by the first data source. The system indexes only a sample of the received first plurality of events, based on a sampling criterion, where the sample is fewer than all of the first plurality of events. The system generates the storage related estimate based on at least some of the first plurality of events, and causes an indication of the estimate to be output to a user.
Abstract:
The present disclosure is directed to providing dynamic indexer discovery. An index manager, which may also be known as a cluster master, is configured to track the statuses and capabilities of indexers and provide the statuses and capabilities obtained from the indexers to data collectors, such as forwarders. The data collectors may use the statuses and capabilities associated with the indexers to load balance transmission of data to the indexers. Dynamic indexer discovery may eliminate the need to manually reconfigure data collectors when the status of an indexer changes because the information may be obtained from the index manager without the need to reinitialize the data collectors.
Abstract:
Disclosed herein is a data estimation technique for a data intake and query system. The system receives user inputs indicative that a first data source is to be the subject of a storage related estimate. The system receives a first plurality of events generated by the first data source. The system indexes only a sample of the received first plurality of events, based on a sampling criterion, where the sample is fewer than all of the first plurality of events. The system generates the storage related estimate based on at least some of the first plurality of events, and causes an indication of the estimate to be output to a user.
Abstract:
Provided are systems and methods for verifying user credentials for performing a search. In one embodiment, a method can be provided that includes receiving a request to perform a search of machine generated data comprising time stamped events that is associated with a user, determining whether a set of cached user credentials has been updated within a period of time, querying, in response to determining that the credentials for the user have not been updated within the period of time, an identity provider server for a current set of user credentials associated with the user, receiving the current set of user credentials, determining whether the user has privileges to perform the search based at least in part on the set of user credentials, and causing, in response to determining that the user has privileges to perform the search, the search to be performed to identify one or more of the events that are responsive to the search.
Abstract:
According to various embodiments, techniques are described for managing data within a multi-site clustered data intake and query system. A data intake and query system as described herein generally refers to a system for collecting, retrieving, and analyzing data. In this context, a clustered data intake and query system generally refers to a system environment that is configured to provide data redundancy and other features that improve the availability of data stored by the system. For example, a clustered data intake and query system may be configured to store multiple copies of data stored by the system across multiple components such that recovery from a failure of one or more of the components is possible by using copies of the data stored elsewhere in the cluster.