Interactively building previews of extract, transform, load (ETL) graphs using cached previews of subgraphs

    公开(公告)号:US11841872B2

    公开(公告)日:2023-12-12

    申请号:US17385393

    申请日:2021-07-26

    CPC classification number: G06F16/254 G06F3/0486 G06F16/24552 G06F16/26

    Abstract: Disclosed are some implementations of systems, apparatus, methods and computer program products for executing a process flow represented by a graph or portion thereof using cached subgraphs. A first request to execute a first portion of a process flow is processed, where the first portion of the process flow is represented by a first subgraph of a graph representing the process flow and a final node of the first subgraph corresponds to a set of computer-readable instructions. The first portion of the process flow is executed such that a first output of executing the first portion of the process flow is obtained. The first subgraph is stored in association with the first output in a first cache entry of a cache. A second request to execute a second portion of the process flow is processed, where the second portion of the process flow is represented by a second subgraph of the graph. At least one cache entry for which a corresponding subgraph matches at least a portion of the second subgraph is identified in the cache, where the at least one cache entry includes the first cache entry. The first output is retrieved from the first cache entry, a node of the second subgraph to which the final node of the first subgraph is connected is identified, and the second portion of the process flow is executed by providing the first output as input to the identified node of the second subgraph without executing the set of computer-readable instructions.

    Interactive dataflow preview
    2.
    发明授权

    公开(公告)号:US11755608B2

    公开(公告)日:2023-09-12

    申请号:US16740918

    申请日:2020-01-13

    CPC classification number: G06F16/254 G06F16/258

    Abstract: DESCRIBED HEREIN ARE SYSTEMS, APPARATUS, METHODS AND COMPUTER PROGRAM PRODUCTS FOR IMPLEMENTING DESIGN TIME AND BATCH TIME FOR AN EXTRACT, TRANSFORM, LOAD (ETL) PROCESS. WHEN A SESSION IS ESTABLISHED, A KUBERNETES POD INSTANCE MAY BE EXCLUSIVELY ASSOCIATED WITH A USER FOR THE USER'S SESSIONS. DESIGN TIME AND BATCH TIME MAY BOTH BE PERFORMED WITHIN THE KUBERNETES POD INSTANCE. AS SUCH, A SERVICE PROVIDER MAY PROVIDE A SECURE DATAFLOW PREVIEW. FURTHERMORE, THE DATAFLOW MAY BE CACHED. CACHING OF THE DATAFLOW ALLOWS THE SERVICE PROVIDER TO MORE QUICKLY PROVIDE FOLLOW UP PREVIEWS, DECREASING LATENCY.

Patent Agency Ranking