The embodiment of the invention provides a method and device of moving target tracking in video. The method comprises: obtaining a video frame comprising a moving target to be tracked, and extracting the image sequence of the video frame; initializing the image sequence, wherein the moving target to be tracked is divided into a plurality of subblocks; extracting the moving features of each subblock, and fusing the moving features and static features extracted in advance to form fusion features; obtaining the position information of each subblock according to the fusion features and a subblock position calculation model; according to the position information of each subblock, determining the effectiveness of each subblock, and obtaining the integral displacement of the moving target to be tracked according to effective subblocks; according to the effective subblocks, obtaining contraction coefficients; and according to the contraction coefficients and preset parameters, obtaining the image dimension of the moving target to be tracked, and performing tracking of the moving target to be tracked. The device executes the method mentioned above. The method and device provided by the embodiment of the invention improve the accuracy of the moving target tracking in the video.