A semiconductor device has contact plugs each for electrically connecting a capacitor element to the source/drain region of a transistor, conductive layers formed on the contact plugs and made of titanium nitride which is a nitride only of a refractory metal, and polycrystalline conductive oxygen barrier layers each composed of a multilayer structure consisting of a titanium aluminum nitride film, an iridium film, and an iridium oxide film to prevent the diffusion of oxygen. Since the conductive layers made of titanium nitride which is low in crystal orientation is provided under the oxygen barrier films, the titanium aluminum nitride films formed as the oxygen barrier films directly on the conductive layers have a compact film structure so that the penetration of oxygen is prevented effectively.