Data Processing Method and Apparatus, and System

    公开(公告)号:US20180293108A1

    公开(公告)日:2018-10-11

    申请号:US16006503

    申请日:2018-06-12

    Abstract: A data processing method and apparatus, and a system, where the method applies to a system including a central processing unit (CPU) pool and a storage pool, a mapper node and a reducer node respectively run on different CPUs in the CPU pool, and a remote shared partition shared by the mapper node and the reducer node is delimited in the storage pool. In this method, the mapper node executes a map task, and stores a data segment obtained by executing the map task, into the remote shared partition, and the reducer node directly obtains a data segment to be processed by the reducer node from the remote shared partition, and executes a reduce task on the data segment. Hence, the method shortens a time for executing the Map/Reduce task.

Patent Agency Ranking