公开(公告)号：US20230315893A1

公开(公告)日：2023-10-05

申请号：US18130632

申请日：2023-04-04

Applicant: Google LLC

Inventor： Justin Levandoski ,  Anoop Kochummen Johnson ,  Gaurav Saxena ,  Thibaud Hottelier ,  Yuri Volobuev ,  Garrett Casto

IPC: G06F21/62 ,  G06F21/60

CPC classification number: G06F21/6227 ,  G06F21/604 ,  G06F2221/2141 ,  G06F2221/2113

Abstract: The present disclosure provides a storage engine that unifies data warehouses and lakes, by providing uniform fine-grained access control, performance acceleration across multi-cloud storage, and open formats. It provides an application programming interface (API) for query engines spanning across data warehouse and open source runtimes to access distributed data with consistent security and governance controls. Access is evaluated at the API layer, separate from the query engine, and is uniformly enforced across query engines.

公开(公告)号：US20240394273A1

公开(公告)日：2024-11-28

申请号：US18201243

申请日：2023-05-24

Applicant: Google LLC

Inventor： Zhou Fang ,  Thibaud Hottelier ,  Anoop Kochummen Johnson ,  Micah Kornfield ,  Justin Levandoski ,  Yuri Volobuev

IPC: G06F16/25 ,  G06F16/22

Abstract: Aspects of the disclosure are directed to a runtime catalog for a cloud storage engine that unifies data lakes and data warehouses. The runtime catalog can expose a single universe of cloud storage tables through an endpoint for query engines for data lakes and another endpoint for query engines for data warehouses. The runtime catalog can allow the query engines for data lakes and the query engines for data warehouses to query any cloud storage table by representing data warehouse native tables in a format compatible with data lakes and representing data lake native tables in a format compatible with data warehouses.

公开(公告)号：US20250077477A1

公开(公告)日：2025-03-06

申请号：US18389331

申请日：2023-11-14

Applicant: Google LLC

Inventor： Thibaud Hottelier ,  Anoop Kochummen Johnson ,  Justin Levandoski ,  Gaurav Saxena ,  Yuri Volobuev

IPC: G06F16/18 ,  G06F16/22 ,  G06F16/23 ,  G06F16/28

Abstract: Aspects of the disclosure are directed to merging data lake openness with scalable metadata for managed tables in a cloud database platform, allowing for atomicity, consistency, isolation, and durability (ACID) transactions, performant data manipulation language (DML), higher throughput stream ingestion, data consistency, schema evolution, time travel, clustering, fine-grained security, and/or automatic storage optimization. Table data is stored in various open-source file formats in cloud storage while physical metadata of the table data is stored in a scalable metadata storage system.

公开(公告)号：US20240378204A1

公开(公告)日：2024-11-14

申请号：US18195577

申请日：2023-05-10

Applicant: Google LLC

Inventor： Thibaud Hottelier ,  Anoop Kochummen Johnson ,  Justin Levandoski ,  Deepak Choudhary Nettem ,  Yuri Volobuev

IPC: G06F16/2455 ,  G06F16/2453 ,  G06F16/25

Abstract: Aspects of the disclosure are directed to a metadata cache for extending data warehouse features to data lakes. The metadata cache can accelerate query execution by directly accessing unmanaged data from the data lake rather than accessing the data through the data warehouse. The metadata cache can allow for filtering the unmanaged data to improve the speed of retrieving data for executing a query.

公开(公告)号：US20250077478A1

公开(公告)日：2025-03-06

申请号：US18389337

申请日：2023-11-14

Applicant: Google LLC

Inventor： Victor Sergeyevich Agababov ,  Shuang Guan ,  Thibaud Hottelier ,  Anoop Kochummen Johnson ,  Justin Levandoski ,  Bigang Li ,  Yuri Volobuev

IPC: G06F16/18 ,  G06F12/02 ,  G06F16/23 ,  G06F16/28

Abstract: Aspects of the disclosure are directed to merging data lake openness with scalable metadata for managed tables in a cloud database platform, allowing for atomicity, consistency, isolation, and durability (ACID) transactions, performant data manipulation language (DML), higher throughput stream ingestion, data consistency, schema evolution, time travel, clustering, fine-grained security, and/or automatic storage optimization. Table data is stored in various open-source file formats in cloud storage while physical metadata of the table data is stored in a scalable metadata storage system.

公开(公告)号：US20240193295A1

公开(公告)日：2024-06-13

申请号：US18080178

申请日：2022-12-13

Applicant: Google LLC

Inventor： Thibaud Hottelier ,  Brian Lee Welcker ,  Jonah Tang Soon Yuen ,  Neil Martin Devine

IPC: G06F21/62 ,  G06F16/2458

CPC classification number: G06F21/6227 ,  G06F16/2471

Abstract: Aspects of the disclosure relate to managing access to published data by different groups of users through linked datasets. A subscriber system generates a linked dataset that links the subscriber system to a source dataset published by a publisher system. The subscriber system queries the linked dataset. Queries to the linked dataset are redirected to the source dataset. The subscriber system manages access control to the linked dataset instead of the publisher system managing access control for the subscriber system directly to the source dataset. The source dataset does not need to be copied to the subscriber system. From the perspective of the subscriber system, changes to the source dataset appear instantly, as subscribers may query the source dataset through the linked dataset without waiting for copies of the source dataset to propagate.