公开(公告)号：US10484468B2

公开(公告)日：2019-11-19

申请号：US14624647

申请日：2015-02-18

Applicant: Tencent Technology (Shenzhen) Company Limited

Inventor： Haijun Wu ,  Rui Li ,  Yongqiang Zou ,  Shengyu Dong ,  Taifu Que ,  Lei Wang ,  Shaopeng Yang ,  Shuxin Zhang ,  Dayong Zhao ,  Chang Liu ,  Xiaodong Chen ,  Yinfeng Zhang

IPC: H04L29/08

Abstract: Disclosed are a method and a device for downloading a file. The method includes: when receiving a download request of a file, acquiring attribute information about the file, and learning information about the length of the file according to the attribute information about the file; when the length of the file goes beyond a preset value, segmenting the download request into at least two fragments of download requests; sending the at least two fragments of download requests to at least two data nodes to request to download corresponding fragments, and obtaining the at least two fragments; and according to the at least two fragments obtained by download, obtaining the file. By segmenting the download request into a plurality of fragments of requests, the present invention achieves the parallel download of the plurality of fragments of requests, thereby greatly improving the download efficiency of a file.

公开(公告)号：US09607355B2

公开(公告)日：2017-03-28

申请号：US15107475

申请日：2015-06-19

Applicant: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED

Inventor： Yongqiang Zou ,  Yi Li ,  Xing Jin ,  Bin Xiao ,  Zhimao Guo ,  Wei Xue ,  Bo Chen ,  Yong Li ,  Lei Xiao

IPC: G06F15/80 ,  G06T1/20

CPC classification number: G06T1/20 ,  G06F9/48 ,  G06T2200/28

Abstract: A parallel data processing method based on multiple graphic processing units (GPUs) is provided, including: creating, in a central processing unit (CPU), a plurality of worker threads for controlling a plurality of worker groups respectively, the worker groups including a plurality of GPUs; binding each worker thread to a corresponding GPU; loading one batch of training data from a nonvolatile memory to a GPU video memory corresponding to one worker group; transmitting, between a plurality of GPUs corresponding to one worker group, data required by data processing performed by the GPUs through peer to peer; and controlling the plurality of GPUs to perform data processing in parallel through the worker threads.

公开(公告)号：US20160321777A1

公开(公告)日：2016-11-03

申请号：US15210278

申请日：2016-07-14

Applicant: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED

Inventor： Xing Jin ,  Yi Li ,  Yongqiang Zou ,  Zhimao Guo ,  Eryu Wang ,  Wei Xue ,  Bo Chen ,  Yong Li ,  Chunjian Bao ,  Lei Xiao

IPC: G06T1/20 ,  G06F9/52 ,  G06F9/50 ,  G06T1/60

CPC classification number: G06T1/20 ,  G06F9/5016 ,  G06F9/5027 ,  G06F9/52 ,  G06N3/0454 ,  G06N3/063 ,  G06N3/084 ,  G06T1/60

Abstract: A parallel data processing method based on multiple graphic processing units (GPUs) is provided, including: creating, in a central processing unit (CPU), a plurality of worker threads for controlling a plurality of worker groups respectively, the worker groups including one or more GPUs; binding each worker thread to a corresponding GPU; loading a plurality of batches of training data from a nonvolatile memory to GPU video memories in the plurality of worker groups; and controlling the plurality of GPUs to perform data processing in parallel through the worker threads. The method can enhance efficiency of multi-GPU parallel data processing. In addition, a parallel data processing apparatus is further provided.

Abstract translation: 提供了一种基于多个图形处理单元（GPU）的并行数据处理方法，包括：在中央处理单元（CPU）中分别创建多个用于控制多个工作者组的工作线程，所述工作人员组包括一个或 更多GPU 将每个工作线程绑定到相应的GPU; 将多批培训数据从非易失性存储器加载到多个工作者组中的GPU视频存储器; 并且通过工作线程并行地控制多个GPU来执行数据处理。 该方法可以提高多GPU并行数据处理的效率。 另外，还提供并行数据处理装置。

公开(公告)号：US20150163280A1

公开(公告)日：2015-06-11

申请号：US14624647

申请日：2015-02-18

Applicant: Tencent Technology (Shenzhen) Company Limited

Inventor： Haijun WU ,  Rui Li ,  Yongqiang Zou ,  Shengyu Dong ,  Taifu Que ,  Lei Wang ,  Shaopeng Yang ,  Shuxin Zhang ,  Dayong Zhao ,  Chang Liu ,  Xiaodong Chen ,  Yinfeng Zhang

IPC: H04L29/08

CPC classification number: H04L67/108 ,  H04L67/02 ,  H04L67/06

Abstract: Disclosed are a method and a device for downloading a file. The method includes: when receiving a download request of a file, acquiring attribute information about the file, and learning information about the length of the file according to the attribute information about the file; when the length of the file goes beyond a preset value, segmenting the download request into at least two fragments of download requests; sending the at least two fragments of download requests to at least two data nodes to request to download corresponding fragments, and obtaining the at least two fragments; and according to the at least two fragments obtained by download, obtaining the file. By segmenting the download request into a plurality of fragments of requests, the present invention achieves the parallel download of the plurality of fragments of requests, thereby greatly improving the download efficiency of a file.

Abstract translation: 公开了一种用于下载文件的方法和装置。 该方法包括：当接收到文件的下载请求时，获取关于该文件的属性信息，以及根据关于该文件的属性信息来学习关于文件长度的信息; 当文件的长度超过预设值时，将下载请求分割成至少两个下载请求片段; 向所述至少两个数据节点发送所述至少两个下载请求片段以请求下载对应的片段，并获得所述至少两个片段; 并根据通过下载获得的至少两个片段获得该文件。 通过将下载请求分割为多个请求片段，本发明实现了多个片段的并行下载，大大提高了文件的下载效率。

公开(公告)号：US20140372457A1

公开(公告)日：2014-12-18

申请号：US14294989

申请日：2014-06-03

Applicant: Tencent Technology (Shenzhen) Company Limited

Inventor： Yi Yang ,  Yongqiang Zou ,  Ke Lu ,  Zheng Chen ,  Haijun Wu ,  Tao Yu ,  Luxin Li ,  Jiaxu Wu ,  Jingbing Cui ,  Diaoqin Xin ,  Zan Zou

IPC: G06F7/24

CPC classification number: G06Q10/04 ,  G06F16/2462

Abstract: A method and device for processing data in the field of data process are disclosed. The method includes: sorting samples according to primary keys, wherein the primary key includes a feature serial number and a sample serial number, and wherein a column value corresponding to the primary key is used as a feature value for the sample; acquiring a statistic of each feature in each category by taking the primary key and the feature value as an input key-value pair and calculating with a first algorithm model, and outputting the feature serial number and the statistic as an output key-value pair; and acquiring a contribution value of each feature to the category by performing calculation on the output key-value pair with a second algorithm model, and selecting a feature based on the contribution value. The device includes a sorting module, a first processing module and a second processing module.

Abstract translation: 公开了一种用于处理数据处理领域的数据的方法和装置。 该方法包括：根据主键分类样本，其中主键包括特征序列号和样本序列号，并且其中将与主键对应的列值用作样本的特征值; 通过将主键和特征值作为输入键值对并通过第一算法模型进行计算，并输出特征序列号和统计量作为输出键值对，获取每个类别中每个特征的统计量; 以及通过使用第二算法模型对所述输出键值对执行计算，并且基于所述贡献值来选择特征来获取每个特征的贡献值到所述类别。 该装置包括排序模块，第一处理模块和第二处理模块。

公开(公告)号：US10282809B2

公开(公告)日：2019-05-07

申请号：US15210278

申请日：2016-07-14

Applicant: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED

Inventor： Xing Jin ,  Yi Li ,  Yongqiang Zou ,  Zhimao Guo ,  Eryu Wang ,  Wei Xue ,  Bo Chen ,  Yong Li ,  Chunjian Bao ,  Lei Xiao

IPC: G06T1/20 ,  G06F9/50 ,  G06F9/52 ,  G06T1/60 ,  G06N3/04 ,  G06N3/063 ,  G06N3/08

Abstract: A parallel data processing method based on multiple graphic processing units (GPUs) is provided, including: creating, in a central processing unit (CPU), a plurality of worker threads for controlling a plurality of worker groups respectively, the worker groups including one or more GPUs; binding each worker thread to a corresponding GPU; loading a plurality of batches of training data from a nonvolatile memory to GPU video memories in the plurality of worker groups; and controlling the plurality of GPUs to perform data processing in parallel through the worker threads. The method can enhance efficiency of multi-GPU parallel data processing. In addition, a parallel data processing apparatus is further provided.

公开(公告)号：US20160321776A1

公开(公告)日：2016-11-03

申请号：US15107475

申请日：2015-06-19

Applicant: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED

Inventor： Yongqiang Zou ,  Yi Li ,  Xing Jin ,  Bin Xiao ,  Zhimao Guo ,  Wei Xue ,  Bo Chen ,  Yong Li ,  Lei Xiao

IPC: G06T1/20

CPC classification number: G06T1/20 ,  G06F9/48 ,  G06T2200/28

Abstract: A parallel data processing method based on multiple graphic processing units (GPUs) is provided, including: creating, in a central processing unit (CPU), a plurality of worker threads for controlling a plurality of worker groups respectively, the worker groups including a plurality of GPUs; binding each worker thread to a corresponding GPU; loading one batch of training data from a nonvolatile memory to a GPU video memory corresponding to one worker group; transmitting, between a plurality of GPUs corresponding to one worker group, data required by data processing performed by the GPUs through peer to peer; and controlling the plurality of GPUs to perform data processing in parallel through the worker threads.

Abstract translation: 提供了一种基于多个图形处理单元（GPU）的并行数据处理方法，包括：在中央处理单元（CPU）中分别创建多个用于控制多个工作者组的工作线程，所述工作人员组包括多个 的GPU; 将每个工作线程绑定到相应的GPU; 将一批训练数据从非易失性存储器加载到对应于一个工作者组的GPU视频存储器; 在对应于一个工作组的多个GPU之间传输由GPU通过对等设备执行的数据处理所需的数据; 并且通过工作线程并行地控制多个GPU来执行数据处理。