A method includes forming a plurality of trenches extending from a top surface of a semiconductor substrate into the semiconductor substrate, with semiconductor strips formed between the plurality of trenches. The plurality of trenches includes a first trench and second trench wider than the first trench. A first dielectric material is filled in the plurality of trenches, wherein the first trench is substantially fully filled, and the second trench is filled partially. A second dielectric material is formed over the first dielectric material. The second dielectric material fills an upper portion of the second trench, and has a shrinkage rate different from the first shrinkage rate of the first dielectric material. A planarization is performed to remove excess second dielectric material. The remaining portions of the first dielectric material and the second dielectric material form a first and a second STI region in the first and the second trenches, respectively.