Provided are a method and device for encoding or decoding an image by using a block map. In the method and device for decoding an image according to an embodiment set forth herein, a bitstream of an encoded image is received, one or more blocks in a picture are determined using split information obtained from the bitstream, a block map indicating whether or not a previous block among the one or more blocks has been decoded is determined, a neighboring region which is available to be referred to for a current block to be decoded in a certain scan order among the one or more blocks is determined, and the current block is decoded based on the neighboring region which is available to be referred to.