The invention discloses a video abstraction generating method based on sketches, which belongs to the field of man-machine interaction. The method comprises the following steps: (1) carrying out shot segmentation and shot screening on the video to obtain key frames; (2) carrying out character analysis and scene analysis on the key frames, and determining the semantic significance level of each key frame according to the character analysis; (3) calculating an image quality assessed value of each key frame, and screening the key frames according to the image quality assessed value and the semantic significance level value; (4) generating a video object sketch according to each key frame determined in step (3); (5) resetting the size of the corresponding sketch according to the size of the abstraction drawing region and the significance level of the video contents corresponding to each sketch; and (6) determining the position of each sketch by utilizing the scene analysis result, generating connecting lines among the sketches, and connecting the sketches to generate a video abstraction. By using the method of the invention, the sketch abstraction can intensively and effectively reflect the main plot semanteme of the video, thereby conforming to the perception habits of users.