Invention Publication
- Patent Title: PREVIEW DATA LINEAGE RELATIONSHIP TO REDUCE ETL
-
Application No.: US18125882Application Date: 2023-03-24
-
Publication No.: US20240320234A1Publication Date: 2024-09-26
- Inventor: Yi Yang Ren , Chun Hua Sun , Xu Bin Cai , Wei Wang , Jian Ling Shi , Chun Leng , Pin Lv
- Applicant: International Business Machines Corporation
- Applicant Address: US NY Armonk
- Assignee: International Business Machines Corporation
- Current Assignee: International Business Machines Corporation
- Current Assignee Address: US NY Armonk
- Main IPC: G06F16/25
- IPC: G06F16/25 ; G06F16/215

Abstract:
An approach is disclosed that receives a new ETL job. The job includes a number of intermediate database files descriptors corresponding to a plurality of intermediate database files that are used to accomplish the new ETL. A new data lineage graph is created that pertains to the new ETL job. The new data lineage graph is compared to a number of existing data lineage graphs with each of the existing data lineage graphs corresponding to an existing ETL job. The approach substitutes existing database files found in the existing data lineage graphs for one or more intermediate database files found in the new data lineage graph. The new ETL job is then run by utilizing the substituted database files, the result being a new final database file.
Information query