Devices, systems, and methods for digital video coding, which includes constructing separate motion candidate lists for video coding, are described. An exemplary method for video processing includes constructing, for a first video block of a video, a first motion candidate list using a first motion candidate list construction method that excludes motion candidates that are only derived from temporal blocks, and performing a conversion between the first video block and a bitstream representation of the video including the first video block according to the constructed first motion candidate list.