公开(公告)号：US08892529B2

公开(公告)日：2014-11-18

申请号：US14140403

申请日：2013-12-24

Applicant: Huawei Technologies Co., Ltd.

Inventor： Qiang Liu ,  Quancheng Sun ,  Xiaobo Liu ,  Jun You ,  Huadi Yang ,  Dan Zhou ,  Yan Huang

IPC: G06F17/30

CPC classification number: G06F17/30156 ,  G06F3/0608 ,  G06F3/0641 ,  G06F3/067 ,  G06F17/30

Abstract: In embodiments of the present invention, when a duplicate data query is performed on a received data stream, a first physical node which corresponds to each first sketch value and is in a cluster system is identified according to a first sketch value representing the data stream, and then the first sketch value representing the data stream is sent to the identified physical node for the duplicate data query, and a procedure of the duplicate data query does not change with an increase of the number of nodes in the cluster system; therefore, a calculation amount of each node does not increase with an increase of the number of nodes in the cluster system.

Abstract translation: 在本发明的实施例中，当对所接收的数据流执行重复数据查询时，根据表示数据流的第一草图值来识别对应于每个第一草图值并且在集群系统中的第一物理节点， 然后将表示数据流的第一个草图值发送到识别的物理节点进行重复数据查询，并且重复数据查询的过程不会随着集群系统中节点数量的增加而改变; 因此，随着群集系统中的节点数量的增加，每个节点的计算量不会增加。