A semiconductor memory device comprises a semiconductor substrate; a cell array block formed on the semiconductor substrate and including plural stacked cell array layers each comprising a plurality of first lines, a plurality of second lines crossing the plurality of first lines, and memory cells connected at intersections of the first and second lines between both lines; and a plurality of contact plugs extending in the stack direction of the cell array layers to connect between the first lines, between the second lines, between the first or second line and the semiconductor substrate, or between the first or second line and another metal line, in the cell array layers. The first or second line in a certain one of the cell array layers has a contact connector making contact with both sides of the contact plug.