-
公开(公告)号:US20150227392A1
公开(公告)日:2015-08-13
申请号:US14176679
申请日:2014-02-10
Applicant: International Business Machines Corporation
Inventor: Nicholas C. Fuller , Minkyong Kim , Min Li , Shicong Meng , Jian Tan , Liangzhao Zeng , Li Zhang
IPC: G06F9/50
CPC classification number: G06F9/5066 , G06F9/466 , G06F9/4881 , G06F11/34
Abstract: Methods, systems, and articles of manufacture for enabling dynamic task-level configuration in MapReduce are provided herein. A method includes generating a first set of configurations for a currently executing MapReduce job, wherein said set of configurations comprises job-level configurations and task-level configurations; dynamically modifying configurations associated with a mapper component and/or a reducer component associated with at least one ongoing map task and/or ongoing reduce task of the MapReduce job based on the generated first set of configurations; and deploying said first set of configurations to the mapper component and/or the reducer component associated with the MapReduce job.
Abstract translation: 本文提供了在MapReduce中实现动态任务级配置的方法,系统和制造。 一种方法包括为当前执行的MapReduce作业生成第一组配置,其中所述一组配置包括作业级配置和任务级配置; 基于所生成的第一组配置,动态地修改与映射器组件和/或与至少一个正在进行的映射任务相关联的还原器组件和/或正在进行的MapReduce作业的减少任务; 以及将所述第一组配置部署到与所述MapReduce作业相关联的所述映射器组件和/或所述reducer组件。
-
公开(公告)号:US11474697B2
公开(公告)日:2022-10-18
申请号:US15155473
申请日:2016-05-16
Applicant: INTERNATIONAL BUSINESS MACHINES CORPORATION
Inventor: John A. Bivens , Min Li , Ruchi Mahindru , HariGovind V. Ramasamy , Yaoping Ruan , Valentina Salapura , Eugen Schenfeld
IPC: G06F3/06
Abstract: Various embodiments for optimizing memory bandwidth in a disaggregated computing system, by a processor device, are provided. Respective memory devices are assigned to respective processor devices in the disaggregated computing system, the disaggregated computing system having at least a pool of the memory devices and a pool of the processor devices. An analytic function is performed on data resident in the pool of the memory devices using memory bandwidth not currently committed to a primary compute task.
-
公开(公告)号:US20220100722A1
公开(公告)日:2022-03-31
申请号:US17031996
申请日:2020-09-25
Applicant: International Business Machines Corporation
Inventor: Sheng Yan Sun , Min Li , Shuo Li , Xiaobo Wang , Jian Xu
Abstract: In an approach for database self-diagnosis and self-healing, a processor receives a problem description related to a database. A processor classifies the problem description into a natural language description portion and a database-know-who content portion. A processor processes the natural language description portion using natural language processing techniques. A processor evaluates the database-know-who content portion. A processor combines a result of processing the natural language description portion and evaluating the database-know-who content portion. A processor identifies a solution based on the problem description and the combined result. A processor solves a problem using the identified solution.
-
公开(公告)号:US11275622B2
公开(公告)日:2022-03-15
申请号:US16204653
申请日:2018-11-29
Applicant: INTERNATIONAL BUSINESS MACHINES CORPORATION
Inventor: Min Li , John A. Bivens , Ruchi Mahindru , HariGovind V. Ramasamy , Yaoping Ruan , Valentina Salapura , Eugen Schenfeld
Abstract: Server resources in a data center are disaggregated into shared server resource pools, including an accelerator (e.g., FPGA) pool. Servers are constructed dynamically, on-demand and based on workload requirements, by allocating from these resource pools. According to this disclosure, accelerator utilization in the data center is managed proactively by assigning accelerators to workloads in a fine granularity and agile way, and de-provisioning them when no longer needed. In this manner, the approach is especially advantageous to automatically provision accelerators for data analytic workloads. The approach thus provides for a “micro-service” enabling data analytic workloads to automatically and transparently use FPGA resources without providing (e.g., to the data center customer) the underlying provisioning details. Preferably, the approach dynamically determines the number and the type of FPGAs to use, and then during runtime auto-scales the FPGAs based on workload.
-
公开(公告)号:US11243846B2
公开(公告)日:2022-02-08
申请号:US15994573
申请日:2018-05-31
Applicant: INTERNATIONAL BUSINESS MACHINES CORPORATION
Inventor: Valentina Salapura , John A. Bivens , Min Li , Ruchi Mahindru , Eugen Schenfeld
Abstract: Embodiments for replicating data in a disaggregated computing system. A memory pool is allocated, where the memory pool includes allocated memory elements at a first site and allocated memory elements at a second site. The allocated memory elements are mapped at the first site to the allocated memory elements at the second site. A replication operation is initiated to mirror data stored within the allocated memory elements at the first site to the allocated memory elements at the second site. The allocated memory elements at the first site are directly connected through an independent networking connection to the allocated memory elements at the second site such that the replication operation is processed exclusively through compute resources at the first site.
-
公开(公告)号:US11050637B2
公开(公告)日:2021-06-29
申请号:US16143427
申请日:2018-09-26
Applicant: INTERNATIONAL BUSINESS MACHINES CORPORATION
Inventor: Ruchi Mahindru , John A. Bivens , Min Li , Valentina Salapura , Eugen Schenfeld
Abstract: Embodiments for component lifecycle optimization in a disaggregated computing environment. A monitoring and machine learning process is performed to learn a respective lifecycle of different resource types as the different resource types are assigned to respective workloads. The monitoring and machine learning process is used to develop a set of learned failure patterns for determining a mitigation action to perform as new faults are encountered within each of the different resource types while executing the respective workloads. The mitigation action is performed to optimize a remaining lifecycle of respective ones of the different resource types according to the set of learned failure patterns.
-
87.
公开(公告)号:US11017103B2
公开(公告)日:2021-05-25
申请号:US16204463
申请日:2018-11-29
Applicant: INTERNATIONAL BUSINESS MACHINES CORPORATION
Inventor: HariGovind V. Ramasamy , John A. Bivens , Ruchi Mahindru , Valentina Salapura , Min Li , Yaoping Ruan , Eugen Schenfeld
Abstract: A group of processors in a processor pool comprise a secure “enclave” in which user code is executable and user data is readable solely with the enclave. This is facilitated through the key management scheme described that includes two sets of key-pairs, namely: a processor group key-pair, and a separate user key-pair (typically one per-user, although a user may have multiple such key-pairs). The processor group key-pair is associated with all (or some define subset of) the processors in the group. This key-pair is used to securely communicate a user private key among the processors. The user private key, however, is not transmitted to non-members of the group. Further, preferably the user private key is refreshed periodically or upon any membership change (in the group) to ensure that non-members or ex-members cannot decipher the encrypted user key.
-
88.
公开(公告)号:US10936374B2
公开(公告)日:2021-03-02
申请号:US15982579
申请日:2018-05-17
Applicant: INTERNATIONAL BUSINESS MACHINES CORPORATION
Inventor: John A. Bivens , Eugen Schenfeld , Ruchi Mahindru , Min Li , Valentina Salapura
IPC: G06F9/50
Abstract: Embodiments for optimizing memory placement in a disaggregated computing environment. A new workload is assigned to a subset of a plurality of processors, the subset of processors assigned a subset of a plurality of memory devices. In some embodiments, a determination is made as to whether the new workload is categorized as a memory-dependent workload having a memory need which can be met primarily by the subset of the memory devices. If the new workload is categorized as a memory-dependent workload, a determination is then made as to whether the subset of the memory devices is meeting the memory need of the new workload. When the subset of the memory devices is not meeting the memory need of the new workload, a memory related action is taken.
-
公开(公告)号:US10915373B2
公开(公告)日:2021-02-09
申请号:US16204425
申请日:2018-11-29
Applicant: INTERNATIONAL BUSINESS MACHINES CORPORATION
Inventor: Min Li , John A. Bivens , Ruchi Mahindru , HariGovind V. Ramasamy , Yaoping Ruan , Valentina Salapura , Eugen Schenfeld
Abstract: MapReduce processing is carried out in a disaggregated compute environment comprising a set of resource pools that comprise a processor pool, and a memory pool. Upon receipt of a MapReduce job, a task scheduler allocates resources from the set of resource pools, the resources including one or more processors drawn from the processor pool, and one or more memory modules drawn from the memory pool. The task scheduler then schedules a set of tasks required by the MapReduce job. At least one particular task in the set is scheduled irrespective of a location of data required for the particular task. In association with a shuffle phase of the MapReduce job, and in connection with the particular task, at least one connection between a processor and at least one memory module is dynamically rewired based on the location of the data required for the particular task, thereby obviating network transfer of that data.
-
公开(公告)号:US10901918B2
公开(公告)日:2021-01-26
申请号:US16204661
申请日:2018-11-29
Applicant: INTERNATIONAL BUSINESS MACHINES CORPORATION
Inventor: HariGovind V. Ramasamy , Eugen Schenfeld , Valentina Salapura , John A. Bivens , Yaoping Ruan , Min Li , Ashish Kundu , Ruchi Mahindru , Richard H. Boivie
Abstract: Server resources in a data center are disaggregated into shared server resource pools, which include a pool of secure processors. Advantageously, servers are constructed dynamically, on-demand and based on a tenant's workload requirements, by allocating from these resource pools. According to this disclosure, secure processor modules for new servers are allocated to provide security for data-in-use (and data-at-rest) in a dynamic fashion so that virtual and non-virtual capacity can be adjusted in the disaggregate compute system without any downtime, e.g., based on workload security requirements and data sensitivity characteristics. The approach herein optimizes an overall utilization of an available secure processors resource pool in the disaggregated environment. The resulting disaggregate compute system that is configured according to the approach cryptographically-protects workload data whenever it is outside the CPU chip.
-
-
-
-
-
-
-
-
-