The present invention relates to an image processing apparatus and an image processing method that allow for setting information allowing for easily controlling the granularity in trick play. A setting unit sets Supplemental Enhancement Information (SEI) about the encoded data of pictures. The SEI includes the reference layer information indicating layers of the reference relationship among the pictures. The present invention can be used, for example, for a recording device that encodes the video data in a High Efficiency Video Coding (HEVC) scheme in the Blu-ray (registered trademark) Disc (BD) standard.