Export data from tables into partitioned folders on an external data lake

    公开(公告)号:US11347775B2

    公开(公告)日:2022-05-31

    申请号:US17463313

    申请日:2021-08-31

    申请人: Snowflake Inc.

    摘要: A database export system exports data using a plurality of nodes that process the data to generate structured result files that are partitioned by an export parameter in an export request. The database export system distributes the data and merges the files to avoid small file creation and increase processing speed via parallelism. The database export system generates the result files of a specified maximum size in a final format, where the files are processed merged in a temporary file format. The parallel processing is optimized and constrained per the amount of processing nodes, available memory, requested final file sizes, and operation based ordering to complete data exports in a scalable multi-stage approach.

    EXPORT DATA FROM TABLES INTO PARTITIONED FOLDERS ON AN EXTERNAL DATA LAKE

    公开(公告)号:US20220121683A1

    公开(公告)日:2022-04-21

    申请号:US17463313

    申请日:2021-08-31

    申请人: Snowflake Inc.

    摘要: A database export system exports data using a plurality of nodes that process the data to generate structured result files that are partitioned by an export parameter in an export request. The database export system distributes the data and merges the files to avoid small file creation and increase processing speed via parallelism. The database export system generates the result files of a specified maximum size in a final format, where the files are processed merged in a temporary file format. The parallel processing is optimized and constrained per the amount of processing nodes, available memory, requested final file sizes, and operation based ordering to complete data exports in a scalable multi-stage approach.

    Export data from tables into partitioned folders on an external data lake

    公开(公告)号:US11354331B2

    公开(公告)日:2022-06-07

    申请号:US17086221

    申请日:2020-10-30

    申请人: Snowflake Inc.

    摘要: A database export system exports data using a plurality of nodes that process the data to generate structured result files that are partitioned by an export parameter in an export request. The database export system distributes the data and merges the files to avoid small file creation and increase processing speed via parallelism. The database export system generates the result files of a specified maximum size in a final format, where the files are processed merged in a temporary file format. The parallel processing is optimized and constrained per the amount of processing nodes, available memory, requested final file sizes, and operation based ordering to complete data exports in a scalable multi-stage approach.

    EXPORT DATA FROM TABLES INTO PARTITIONED FOLDERS ON AN EXTERNAL DATA LAKE

    公开(公告)号:US20220121673A1

    公开(公告)日:2022-04-21

    申请号:US17086221

    申请日:2020-10-30

    申请人: Snowflake Inc.

    摘要: A database export system exports data using a plurality of nodes that process the data to generate structured result files that are partitioned by an export parameter in an export request. The database export system distributes the data and merges the files to avoid small file creation and increase processing speed via parallelism. The database export system generates the result files of a specified maximum size in a final format, where the files are processed merged in a temporary file format. The parallel processing is optimized and constrained per the amount of processing nodes, available memory, requested final file sizes, and operation based ordering to complete data exports in a scalable multi-stage approach.

    Export data from tables into partitioned folders on an external data lake

    公开(公告)号:US11138232B1

    公开(公告)日:2021-10-05

    申请号:US17086215

    申请日:2020-10-30

    申请人: Snowflake Inc.

    摘要: A database export system exports data using a plurality of nodes that process the data to generate structured result files that are partitioned by an export parameter in an export request. The database export system distributes the data and merges the files to avoid small file creation and increase processing speed via parallelism. The database export system generates the result files of a specified maximum size in a final format, where the files are processed merged in a temporary file format. The parallel processing is optimized and constrained per the amount of processing nodes, available memory, requested final file sizes, and operation based ordering to complete data exports in a scalable multi-stage approach.