A method and apparatus for three-dimensional video coding and multi-view video coding are disclosed. Embodiments according to the present invention derive a unified disparity vector (DV) based on neighboring blocks of the current block or depth information associated with the current block and locate a single corresponding block in a reference view according to the unified DV. An inter-view motion vector prediction (MVP) candidate is then derived for both list0 and list1 from the single corresponding block. List0 and list1 MVs of the inter-view MVP candidate are derived from the single corresponding block located according to the unified DV.