A quickly installed headrest adjustment device includes a mount case connected to the chair back and includes a recessed area, an engaging hole, a latch and a first contact face on the rear side of the chair back. The engaging hole is located between the two side slots and defined through the inner bottom of the recessed area. The latch is connected to the inner top side of the engaging hole and includes a bottom face which protrudes beyond the engaging hole and faces the inner side of the recessed area. An adjustment device is mounted to the rail and includes a second contact face facing the first contact face, two ridges, and a block. The block is located between the two ridges. The users can connect the adjustment device to the mount case of the chair back to quickly install the headrest unit to the chair back.