-
公开(公告)号:US10459915B2
公开(公告)日:2019-10-29
申请号:US12977594
申请日:2010-12-23
申请人: Craig W. Stanfill , John Maclean
发明人: Craig W. Stanfill , John Maclean
IPC分类号: G06F16/30 , G06F16/2453
摘要: Managing queries performed on one or more data sources includes: storing at least a first query in a storage medium; selecting the first query for processing; instructing a query engine to process the first query on a first portion of data in the one or more data sources for a first query interval; receiving result data from the query engine based on processing the first query on the first portion of data; saving a state of the first query in the storage medium after the first query interval; instructing the query engine to process a second query during a second query interval after the first query interval; and instructing the query engine to process the first query on a second portion of data in the one or more data sources during a third query interval after the second query interval.
-
公开(公告)号:US09977659B2
公开(公告)日:2018-05-22
申请号:US13281039
申请日:2011-10-25
CPC分类号: G06F8/34
摘要: Managing data set objects for graph-based data processing includes: storing a group of one or more data set objects in a data storage system, the data set objects each representing a respective data set; and generating an association between at least a first data set object in the group and at least a first node of a dataflow graph for processing data in a data processing system, the first node representing a source or sink of data in a flow of data represented by a link in the dataflow graph, and the first data set object including a plurality of modes in which different transformational logic is applied to data processed by the first node.
-
公开(公告)号:US08706667B2
公开(公告)日:2014-04-22
申请号:US12180141
申请日:2008-07-25
CPC分类号: G06N99/005 , G06F9/4494 , G06F9/466
摘要: Processing transactions using graph-based computations includes determining that at least one of a plurality of graph elements of a computation graph of a set of one or more computation graphs includes a computation to be performed for a given transaction, associating the given transaction with an instance of the computation graph that includes reusable computation elements associated with respective graph elements, and executing the graph to perform the computation.
摘要翻译: 使用基于图的计算的处理事务包括确定一个或多个计算图的集合的计算图的多个图元素中的至少一个图元素包括要为给定事务执行的计算,将给定事务与实例相关联 包括与各个图形元素相关联的可重复使用的计算元素的计算图形,以及执行图形以执行计算。
-
公开(公告)号:US20120271862A1
公开(公告)日:2012-10-25
申请号:US13540125
申请日:2012-07-02
IPC分类号: G06F17/30
CPC分类号: G06F17/30312 , G06F17/30321 , G06F17/30418
摘要: Managing data by: receiving a group of individually accessible data units, each data unit identified by a key value, with key values determined such that the key value identifying a first data unit received before a second data unit occurs earlier in a sort order than the key value identifying the second data unit; and processing the data units for storage in a data storage system. The processing includes: storing blocks of data, the blocks being generated by combining a plurality of the data units; providing an index with entries that enable location, based on a provided key value, of a block that includes a data unit corresponding to the provided key value; and generating one or more screening data structures associated with the blocks for determining, based on a given key value, whether to search the stored blocks for a data unit corresponding to the given key value.
摘要翻译: 通过以下操作来管理数据:接收一组单独可访问的数据单元,每个数据单元由键值标识,其中键值被确定为使得该键值识别在第二数据单元之前接收的第一数据单元以比该 识别第二数据单元的键值; 并处理数据单元以存储在数据存储系统中。 该处理包括:存储数据块,通过组合多个数据单元生成块; 向所述索引提供具有条目的条目,所述条目基于所提供的密钥值,所述条目包括与所提供的密钥值对应的数据单元的块; 以及生成与所述块相关联的一个或多个筛选数据结构,用于基于给定的密钥值来确定是否搜索所存储的块以获得对应于给定密钥值的数据单元。
-
公开(公告)号:US20120056892A1
公开(公告)日:2012-03-08
申请号:US13295216
申请日:2011-11-14
申请人: Craig W. Stanfill
发明人: Craig W. Stanfill
IPC分类号: G06F15/80
CPC分类号: G06F8/45
摘要: An approach to automatically specifying, or assisting with the specification of, a parallel computation graph involves determining data processing characteristics of the linking elements that couple data processing elements of the graph. The characteristics of the linking elements are determined according to the characteristics of the upstream and/or downstream data processing elements associated with the linking element, for example, to enable computation by the parallel computation graph that is equivalent to computation of an associated serial graph.
摘要翻译: 自动指定或协助规划并行计算图的方法涉及确定耦合图的数据处理元件的链接元件的数据处理特性。 根据与链接元件相关联的上游和/或下游数据处理元件的特性来确定连接元件的特性,例如,能够通过并行计算图进行计算,其等效于相关联的串行图的计算。
-
公开(公告)号:US20120056888A1
公开(公告)日:2012-03-08
申请号:US13295206
申请日:2011-11-14
申请人: Craig W. Stanfill
发明人: Craig W. Stanfill
IPC分类号: G06T11/20
CPC分类号: G06F8/45
摘要: An approach to automatically specifying, or assisting with the specification of, a parallel computation graph involves determining data processing characteristics of the linking elements that couple data processing elements of the graph. The characteristics of the linking elements are determined according to the characteristics of the upstream and/or downstream data processing elements associated with the linking element, for example, to enable computation by the parallel computation graph that is equivalent to computation of an associated serial graph.
-
公开(公告)号:US08078820B2
公开(公告)日:2011-12-13
申请号:US12960648
申请日:2010-12-06
CPC分类号: G06F9/466 , G06F9/5016 , G06F9/526 , G06F9/546 , G06F2209/5016 , G06F2209/508
摘要: A method, and corresponding system and software, is described for writing data to a plurality of queues, each portion of the data being written to a corresponding one of the queues. The method includes, without requiring concurrent locking of more than one queue, determining if a space is available in each queue for writing a corresponding portion of the data, and if available, reserving the spaces in the queues. The method includes writing each portion of the data to a corresponding one of the queues.
-
公开(公告)号:US07885932B2
公开(公告)日:2011-02-08
申请号:US12721214
申请日:2010-03-10
CPC分类号: G06F17/30312 , G06F17/30321 , G06F17/30418
摘要: Managing data includes: receiving at least one group of individually accessible data units over an input device or port, each data unit identified by a key value, with key values of the received data units being sorted such that the key value identifying a given first data unit that is received before a given second data unit occurs earlier in a sort order than the key value identifying the given second data unit; and processing the data units for storage in a data storage system. The processing includes: storing a plurality of blocks of data, each of one or more of the blocks being generated by combining a plurality of the data units; providing an index that includes an entry for each of the blocks, wherein one or more of the entries enable location, based on a provided key value, of a block that includes data units corresponding to a range of key values that includes the provided key value; and generating one or more screening data structures associated with the stored blocks for determining a possibility that a data unit that includes a given key value was included in the group of individually accessible data units.
摘要翻译: 管理数据包括:通过输入设备或端口接收至少一组单独可访问的数据单元,每个数据单元通过键值标识,所接收的数据单元的键值被排序,使得键值识别给定的第一数据 在给定的第二数据单元之前接收的单元以比识别给定的第二数据单元的键值更高的排序顺序发生; 并处理数据单元以存储在数据存储系统中。 该处理包括:存储多个数据块,通过组合多个数据单元来生成一个或多个块中的每一个块; 提供包括每个块的条目的索引,其中所述条目中的一个或多个使得能够基于提供的密钥值来确定包括与包括所提供的密钥值的密钥值的范围相对应的数据单元的块的位置 ; 以及生成与所存储的块相关联的一个或多个筛选数据结构,以确定包括给定键值的数据单元被包括在可单独访问的数据单元组中的可能性。
-
公开(公告)号:US20100169274A1
公开(公告)日:2010-07-01
申请号:US12721214
申请日:2010-03-10
CPC分类号: G06F17/30312 , G06F17/30321 , G06F17/30418
摘要: Managing data includes: receiving at least one group of individually accessible data units over an input device or port, each data unit identified by a key value, with key values of the received data units being sorted such that the key value identifying a given first data unit that is received before a given second data unit occurs earlier in a sort order than the key value identifying the given second data unit; and processing the data units for storage in a data storage system. The processing includes: storing a plurality of blocks of data, each of one or more of the blocks being generated by combining a plurality of the data units; providing an index that includes an entry for each of the blocks, wherein one or more of the entries enable location, based on a provided key value, of a block that includes data units corresponding to a range of key values that includes the provided key value; and generating one or more screening data structures associated with the stored blocks for determining a possibility that a data unit that includes a given key value was included in the group of individually accessible data units.
摘要翻译: 管理数据包括:通过输入设备或端口接收至少一组单独可访问的数据单元,每个数据单元通过键值标识,所接收的数据单元的键值被排序,使得键值识别给定的第一数据 在给定的第二数据单元之前接收的单元以比识别给定的第二数据单元的键值更高的排序顺序发生; 并处理数据单元以存储在数据存储系统中。 该处理包括:存储多个数据块,通过组合多个数据单元来生成一个或多个块中的每一个块; 提供包括每个块的条目的索引,其中所述条目中的一个或多个使得能够基于所提供的密钥值来确定包括与包括所提供的密钥值的密钥值的范围相对应的数据单元的块的位置 ; 以及生成与所存储的块相关联的一个或多个筛选数据结构,以确定包括给定键值的数据单元被包括在可单独访问的数据单元组中的可能性。
-
公开(公告)号:US07167850B2
公开(公告)日:2007-01-23
申请号:US10268509
申请日:2002-10-10
申请人: Craig W. Stanfill
发明人: Craig W. Stanfill
IPC分类号: G06F17/00
CPC分类号: G06F9/50 , G06F9/4494
摘要: An approach to performing graph-based computation uses one or both of an efficient startup approach and efficient control using process pools. Efficient startup of a graph-based computation involves precomputing data representing a runtime structure of a computation graph such that an instance of the computation graph is formed using the precomputed data for the required type of graph to form the runtime data structure for the instance of the computation graph. Pools of processes that are each suitable for performing computations associated with one or more vertices of the computation graphs are formed such that at runtime, members of these pools of processes are dynamically assigned to particular vertices of instances of computation graphs when inputs are available for processing at those vertices.
摘要翻译: 执行基于图形的计算的方法使用有效启动方法中的一个或两者,并使用进程池进行有效的控制。 基于图的计算的有效启动涉及预计算表示计算图的运行时结构的数据,使得使用所需类型的图的预计算数据形成计算图的实例,以形成用于实例的运行时数据结构 计算图。 各个适合于执行与计算图的一个或多个顶点相关联的计算的处理池形成为使得在运行时,当输入可用于处理时,这些进程池的成员被动态地分配给计算图的实例的特定顶点 在那些顶点。
-
-
-
-
-
-
-
-
-