System and techniques are provided for three-dimension (3D) semantic segmentation. A device for 3D semantic segmentation includes: an interface, to obtain a point cloud data set for a time-ordered sequence of 3D frames, the 3D frames including a current 3D frame and one or more historical 3D frames previous to the current 3D frame; and processing circuitry, to: invoke a first artificial neural network (ANN) to estimate a 3D scene flow field for each of the one or more historical 3D frames by taking the current 3D frame as a reference frame; and invoke a second ANN to: produce an aggregated feature map, based on the reference frame and the estimated 3D scene flow field for each of the one or more historical 3D frames; and perform the 3D semantic segmentation based on the aggregated feature map.