The invention discloses a method for monitoring the coal storage yard of a port by a remote sensing image. The method comprises the following steps of: (1) adopting a lee sigma edge extraction algorithm for one wave band of the remote sensing image, wherein the algorithm uses a specific edge filter, and two independent edge images, including a bright edge image and a dark edge image, are created from an original image; (2) enabling the bright edge image, the dark edge image and the remote sensing image to participate in multi-scale segmentation to obtain an image object; (3) in the obtained image object, classifying objects which have a certain distance with a berth which does not contain handling equipment into a category A; (4) using brightness mean value characteristics to remove the objects with high brightness in the category A; and (5) using a NDVI (Normalized difference vegetation index) characteristic to remove the object with a large threshold value out of the category A. The invention also discloses a system.