Optimizing serverless computing using a distributed computing framework

    公开(公告)号:US10678444B2

    公开(公告)日:2020-06-09

    申请号:US15943640

    申请日:2018-04-02

    Abstract: Aspects of the technology provide improvements to a Serverless Computing (SLC) workflow by determining when and how to optimize SLC jobs for computing in a Distributed Computing Framework (DCF). DCF optimization can be performed by abstracting SLC tasks into different workflow configurations to determined optimal arrangements for execution in a DCF environment. A process of the technology can include steps for receiving an SLC job including one or more SLC tasks, executing one or more of the tasks to determine a latency metric and a throughput metric for the SLC tasks, and determining if the SLC tasks should be converted to a Distributed Computing Framework (DCF) format based on the latency metric and the throughput metric. Systems and machine-readable media are also provided.

    PERFORMANCE OF OBJECT STORAGE SYSTEMS
    22.
    发明申请

    公开(公告)号:US20170371558A1

    公开(公告)日:2017-12-28

    申请号:US15192255

    申请日:2016-06-24

    Abstract: Approaches are disclosed for improving performance of logical disks. A logical disk can comprise several storage devices. In an object storage system (OSS), when a logical disk stores a file, fragments of the file are stored distributed across the storage devices. Each of the fragments of the file is asymmetrically stored in (write) and retrieved from (read) the storage devices. The performance of the logical disk is improved by reconfiguring one or more of the storage devices based on an influence that each of the storage devices has on performance of the logical disk and the asymmetric read and write operations of each of the storage devices. For example, latency of the logical disk can be reduced by reconfiguring one or more of the plurality of storage disks based on a proportion of the latency of the logical device that is attributable to each of the plurality of storage devices.

    SMART STORAGE RECOVERY IN A DISTRIBUTED STORAGE SYSTEM
    23.
    发明申请
    SMART STORAGE RECOVERY IN A DISTRIBUTED STORAGE SYSTEM 有权
    分布式存储系统中的SMART STORAGE恢复

    公开(公告)号:US20160335166A1

    公开(公告)日:2016-11-17

    申请号:US14712762

    申请日:2015-05-14

    Abstract: Embodiments include obtaining at least one system metric of a distributed storage system, generating one or more recovery parameters based on the at least one system metric, identifying at least one policy associated with data stored in a storage node of a plurality of storage nodes in the distributed storage system, and generating a recovery plan for the data based on the one or more recovery parameters and the at least one policy. In more specific embodiments, the recovery plan includes a recovery order for recovering the data. Further embodiments include initiating a recovery process to copy replicas of the data from a second storage node to a new storage node, wherein the replicas of the data are copied according to the recovery order indicated in the recovery plan.

    Abstract translation: 实施例包括获得分布式存储系统的至少一个系统度量,基于至少一个系统度量来生成一个或多个恢复参数,识别与存储在多个存储节点的存储节点中的数据相关联的至少一个策略 分布式存储系统,以及基于所述一个或多个恢复参数和所述至少一个策略来生成用于所述数据的恢复计划。 在更具体的实施例中,恢复计划包括用于恢复数据的恢复顺序。 进一步的实施例包括发起恢复过程以将数据的副本从第二存储节点复制到新的存储节点,其中根据恢复计划中指示的恢复顺序复制数据的副本。

    TENANT-LEVEL SHARDING OF DISKS WITH TENANT-SPECIFIC STORAGE MODULES TO ENABLE POLICIES PER TENANT IN A DISTRIBUTED STORAGE SYSTEM
    24.
    发明申请
    TENANT-LEVEL SHARDING OF DISKS WITH TENANT-SPECIFIC STORAGE MODULES TO ENABLE POLICIES PER TENANT IN A DISTRIBUTED STORAGE SYSTEM 审中-公开
    具有特定存储模块的磁盘的潜在级别保护,以便在分布式存储系统中实现每个优先级的策略

    公开(公告)号:US20160334998A1

    公开(公告)日:2016-11-17

    申请号:US14713851

    申请日:2015-05-15

    Abstract: Embodiments include receiving an indication of a data storage module to be associated with a tenant of a distributed storage system, allocating a partition of a disk for data of the tenant, creating a first association between the data storage module and the disk partition, creating a second association between the data storage module and the tenant, and creating rules for the data storage module based on one or more policies configured for the tenant. Embodiments further include receiving an indication of a type of subscription model selected for the tenant, and selecting the disk partition to be allocated based, at least in part, on the subscription model selected for the tenant. More specific embodiments include generating a storage map indicating the first association between the data storage module and the disk partition and indicating the second association between the data storage module and the tenant.

    Abstract translation: 实施例包括接收与分布式存储系统的租户相关联的数据存储模块的指示,为租户的数据分配磁盘的分区,在数据存储模块和磁盘分区之间创建第一关联,创建一个 数据存储模块和租户之间的第二关联,以及基于为租户配置的一个或多个策略来创建数据存储模块的规则。 实施例还包括接收为租户选择的订阅模式的类型的指示,以及至少部分地基于为租户选择的订阅模型来选择要分配的磁盘分区。 更具体的实施例包括生成指示数据存储模块和磁盘分区之间的第一关联并指示数据存储模块和租户之间的第二关联的存储映射。

    Accelerating data replication using multicast and non-volatile memory enabled nodes

    公开(公告)号:US11595474B2

    公开(公告)日:2023-02-28

    申请号:US15856716

    申请日:2017-12-28

    Abstract: A method for accelerating data operations across a plurality of nodes of one or more clusters of a distributed computing environment. Rack awareness information characterizing the plurality of nodes is retrieved and a non-volatile memory (NVM) capability of each node is determined. A write operation is received at a management node of the plurality of nodes and one or more of the rack awareness information and the NVM capability of the plurality of nodes are analyzed to select one or more nodes to receive at least a portion of the write operation, wherein at least one of the selected nodes has an NVM capability. A multicast group for the write operation is then generated wherein the selected nodes are subscribers of the multicast group, and the multicast group is used to perform hardware accelerated read or write operations at one or more of the selected nodes.

    Accessing composite data structures in tiered storage across network nodes

    公开(公告)号:US10938581B2

    公开(公告)日:2021-03-02

    申请号:US15907018

    申请日:2018-02-27

    Abstract: Aspects of the disclosed technology relate to ways to determine the optimal storage of data structures across different memory device is associated with physically disparate network nodes. In some aspects, a process of the technology can include steps for receiving a first retrieval request for a first object, searching a local PMEM device for the first object based on the first retrieval request, in response to a failure to find the first object on the local PMEM device, transmitting a second retrieval request to a remote node, wherein the second retrieval request is configured to cause the remote node to retrieve the first object from a remote PMEM device. Systems and machine-readable media are also provided.

    TENANT-LEVEL SHARDING OF DISKS WITH TENANT-SPECIFIC STORAGE MODULES TO ENABLE POLICIES PER TENANT IN A DISTRIBUTED STORAGE SYSTEM

    公开(公告)号:US20200285396A1

    公开(公告)日:2020-09-10

    申请号:US16879612

    申请日:2020-05-20

    Abstract: Embodiments include receiving an indication of a data storage module to be associated with a tenant of a distributed storage system, allocating a partition of a disk for data of the tenant, creating a first association between the data storage module and the disk partition, creating a second association between the data storage module and the tenant, and creating rules for the data storage module based on one or more policies configured for the tenant. Embodiments further include receiving an indication of a type of subscription model selected for the tenant, and selecting the disk partition to be allocated based, at least in part, on the subscription model selected for the tenant. More specific embodiments include generating a storage map indicating the first association between the data storage module and the disk partition and indicating the second association between the data storage module and the tenant.

Patent Agency Ranking