A video data processor and a video data processing method which uses hardware resources efficiently even when a plurality of processing occurs to video data are performed in parallel is provided. The video data read from a recording medium in playback processing, editing processing, duplication processing, and other process that may be accomplished upon video is outputted to either of two lines by distributor 171. The video data that is outputted is temporarily held by input buffer 172-1, and decoded by decoder 173-1, and outputted to frame buffer 174-1. Frame buffer 174-1 and 174-2 are divided into a plurality of holding areas and these holding areas are dynamically assigned according to processing of playback processing, editing processing, or other processing that may be accomplished upon video. Thereby, the holding areas in frame buffer 174-1 and 174-2 can be used efficiently.