The invention is suitable for the technical field of computer vision and image processing, and provides a wide baseline matching method and device. The method comprises the following steps of: carrying out line segment detection on an input image to obtain a straight line set of the input image; carrying out parallel straight line clustering on the straight line set to obtain parallel straight line groups of the input image; randomly selecting two straight lines from different parallel straight line groups; obtaining non-parallel straight line intersection points of the input image according to the selected straight lines; determining a feature scale of each non-parallel straight line intersection point; determining a feature point main direction according to each feature scale; and carrying out wide baseline matching according to the feature point main direction and a preset matching method. The method has relatively strong robustness for image noise and illumination variation, and has invariance for image rotation and certain viewpoint and scale variation, so that a feasible method is provided for the matching of structured scenes under complicated illumination conditions.