An apparatus comprising a first circuit, a second circuit and a third circuit. The first circuit may be configured to generate a plurality of first motion vectors and first error scores in response to a search of a macroblock of an image. The search generally comprises a range of motion vectors of the macroblock. The second circuit may be configured to generate a plurality of second motion vectors and second error scores for a plurality of sub-blocks of the macroblock in response to a set of discrete candidate motion vectors selected from the plurality of first motion vectors. The third circuit may be configured to segment the macroblock in response to (i) the plurality of first motion vectors and first error scores and (ii) the plurality of second motion vectors and second error scores.