A portable three-dimensional scanner includes at least two image sensing units and a depth map generation unit. When the portable three-dimensional scanner is moved around an object, a first image sensing unit and a second image sensing unit of the at least two image sensing units capture a plurality of first images comprising the object and a plurality of second images comprising the object, respectively. When the first image sensing unit captures each first image of the plurality of first images, a corresponding distance exists between the portable three-dimensional scanner and the object. The depth map generation unit generates a corresponding depth map according to the each first image and a corresponding second image. A plurality of depth maps generated by the depth map generation unit, the plurality of first images, and the plurality of second images are used for generating a color three-dimensional scan result corresponding to the object.