The present invention is to provide a human image recognition system capable of determining a next movement of a person approaching a particular region of space, which comprises a determination unit adapted to determine whether there is a moving object in continuous images taken by an image taking unit by analyzing the continuous images. If yes, the determination unit compares the moving object with contours of at least one admissible person stored in a memory so as to determine whether the moving object is a person recognized or is about to enter into or leave from a door of a building by analyzing the person's moving direction and changed positions, and then determine whether the person should be allowed to pass through or not.