A method of forming a semiconductor device may include forming an interlayer insulating layer on a semiconductor substrate, and the interlayer insulating layer may have a contact hole therein exposing a portion of the semiconductor substrate. A single crystal semiconductor plug may be formed in the contact hole and on portions of the interlayer insulating layer adjacent the contact hole opposite the semiconductor substrate, and portions of the interlayer insulating layer opposite the semiconductor substrate may be free of the single crystal semiconductor plug. Portions of the single crystal semiconductor plug in the contact hole may be removed while maintaining portions of the single crystal semiconductor plug on portions of the interlayer insulating layer adjacent the contact hole as a single crystal semiconductor contact pattern. After removing portions of the single crystal semiconductor plug, a single crystal semiconductor layer may be formed on the interlayer insulating layer and on the single crystal semiconductor contact pattern. A second interlayer insulating layer may be formed on the single crystal semiconductor layer, and a common contact hole may be formed through the second interlayer insulating layer, through the single crystal semiconductor layer, and through the first interlayer insulating layer to expose a portion of semiconductor substrate. In addition, a conductive contact plug may be formed in the common contact hole in contact with the semiconductor substrate. Related devices are also discussed.