One example includes an optical isolator system. An optical isolator element transmits a first optical beam provided at a first port to be output from a second port and blocks a second optical beam provided at the second port from being output from the first port. The optical beams each include a first component and a second component that are orthogonally linearly polarized. The optical isolator element can provide optical isolation based on transverse shifting the first and second components of the optical beams relative to each other to provide propagation of the first optical beam from the first port to the second port and to prevent propagation of the second optical beam from the second port to the first port. At least one phase adjuster adjusts a relative phase of the first and second components of the first optical beam to align the components of the first optical beam.