A processor-implemented method with video processing includes: determining a first image feature of a first image of video data and a second image feature of a second image that is previous to the first image; determining a time-domain information fusion processing result by performing time-domain information fusion processing on the first image feature and the second image feature; and determining a panoptic segmentation result of the first image based on the time-domain information fusion processing result.