A rotary sensor unit comprises a magnetic encoder for detecting rotation of a shaft supported by a rolling bearing. In the sensor unit, a magnetic detection element is shielded from external magnetic field without relying upon a dedicated magnetic shield cover or a housing while preventing relative positional shift of the magnetic detection element and the encoder. Rigidity of a rolling bearing (1) is enhanced by pushing one side face of the outer ring (11) of the rolling bearing (1) with a lid (6) for pressed against a housing (2), thereby applying a preload. Consequently, an encoder (42) issupported by a shaft (3) axial run out of which is prevented, relative positional shift of a magnetic detection element (41) and the encoder (42) is prevented by supporting the magnetic detection element (41) by the lid (6) for pressing against the housing (2), a magnetic shield space is formed between one side end face of the shaft (3) and the lid (6) by forming the shaft (3), the lid (6) and the outer ring (11) of a soft magnetic material, and then the magnetic detection element (41) is arranged in that magnetic shield space.