The invention provides a fractal-based video compression and decompression method. The compression method is a novel video compression and encoding method which adopts the fractal iteration principle for video encoding, and comprises the following steps: encoding a starting frame through block-DCT transformation, and carrying out block motion estimation/compensation coding for a non-I frame, which comprises calculating the sum and sum of squares of the pixels of sub-blocks related to sub-block domains and parent-block domains, finding a most similar matching block in a previous frame search window through full search, and finally compressing factors of an iteration function system through Huffman encoding. The corresponding decompression method comprises the following steps: decoding the I frame through anti-DCT transformation, carrying out Huffman decoding for the non-I frame to obtain the factors of the iteration function system, and carrying out macroblock-based decoding, which comprises calculating the sum and sum of squares of the pixels of the sub-blocks relevant to the parent-block domains, and decoding the macroblocks in the current frame in sequence. The method improves the traditional method of fractal video compression, and not only greatly increases the compression ratio and the peak signal-noise ratio, but also improves the encoding speed, thereby enhancing the fractal video compression and decoding performance, and making the fractal video compression and decoding more practical.