A method includes bonding a first device die to a second device die, encapsulating the first device die in a first encapsulant, performing a backside grinding process on the second device die to reveal through-vias in the second device die, and forming first electrical connectors on the second device die to form a package. The package includes the first device die and the second device die. The method further includes encapsulating the first package in a second encapsulant, and forming an interconnect structure overlapping the first package and the second encapsulant. The interconnect structure comprises second electrical connectors.