公开(公告)号：US20180150529A1

公开(公告)日：2018-05-31

申请号：US15385787

申请日：2016-12-20

Applicant: Amazon Technologies, Inc.

Inventor： GEORGE STEVEN MCPHERSON ,  MEHUL A. SHAH ,  PRAJAKTA DATTA DAMLE ,  GOPINATH DUDDI ,  ANURAG WINDLASS GUPTA

IPC: G06F17/30 ,  G06F9/54

CPC classification number: G06F16/254 ,  G06F9/542

Abstract: Extract, Transform, Load (ETL) processing may be initiated by detected events. A trigger event may be associated with an ETL process apply one or more transformations to a source data object. The trigger event may be detected for the ETL process and evaluated with respect to one or more execution conditions for the ETL process. If the execution conditions for the ETL process are satisfied, then the ETL process may be executed. At least some of the source data object may be obtained, the one or more transformations of the ETL process may be applied, and one or more transformed data objects may be stored.

公开(公告)号：US20180150528A1

公开(公告)日：2018-05-31

申请号：US15385764

申请日：2016-12-20

Applicant: Amazon Technologies, Inc.

Inventor： MEHUL A. SHAH ,  GEORGE STEVEN MCPHERSON ,  PRAJAKTA DATTA DAMLE ,  GOPINATH DUDDI ,  ANURAG WINDLASS GUPTA ,  BENJAMIN ALBERT SOWELL ,  BOHOU LI

IPC: G06F17/30

CPC classification number: G06F16/254 ,  G06F16/282

Abstract: Data transformation workflows may be generated to transform data objects. A source data schema for a data object and a target data format or target data schema for a data object may be identified. A comparison of the source data schema and the target data format or schema may be made to determine what transformations can be performed to transform the data object into the target data format or schema. Code to execute the transformation operations may then be generated. The code may be stored for subsequent modification or execution.

公开(公告)号：US20180173774A1

公开(公告)日：2018-06-21

申请号：US15385789

申请日：2016-12-20

Applicant: Amazon Technologies, Inc.

Inventor： GEORGE STEVEN MCPHERSON ,  MEHUL A. SHAH ,  PRAJAKTA DATTA DAMLE ,  GOPINATH DUDDI ,  ANURAG WINDLASS GUPTA

IPC: G06F17/30

Abstract: History for data objects may be maintained to detect data events. An indication of an Extract, Transform, Load (ETL) process applied to one or more source data objects to generate one or more transformed data objects may be received. History for the source data objects may be updated to include the transformed data objects and the ETL process that generated the transformed data objects. An evaluation of the update may be performed to determine whether an event associated with the data lineage is triggered. If the event is triggered, a notification of the event may be sent to one or more subscribers for the event.

公开(公告)号：US20180150548A1

公开(公告)日：2018-05-31

申请号：US15385772

申请日：2016-12-20

Applicant: Amazon Technologies, Inc.

Inventor： MEHUL A. SHAH ,  GEORGE STEVEN MCPHERSON ,  PRAJAKTA DATTA DAMLE ,  GOPINATH DUDDI ,  ANURAG WINDLASS GUPTA

IPC: G06F17/30

CPC classification number: G06F16/285 ,  G06F16/13 ,  G06F16/211 ,  G06F16/254 ,  G06F16/289

Abstract: Recognizing unknown data objects may be implemented for data objects stored in a data store. Data objects that are identified as unknown may be accessed to retrieve a portion of the data object. Different representations of the data object may be generated for recognizing different data schemas. An analysis of the representations may be performed to identify a data schema for the unknown data object. The data schema may be stored in a metadata store for the unknown data object.