Methods and apparatuses are provided for controlling retransmission by a User Equipment (UE) in a communication system. A negative acknowledgement (NACK) is received, from a Node B, for at least one transport block among a plurality of transport blocks transmitted by the UE. A precoding matrix and a number of layers for retransmission are determined. The at least one transport block is retransmitted using the determined precoding matrix and the determined number of layers. The precoding matrix is predefined and the number of layers is equal to a number of layers corresponding to the at least one transport block, if the UE does not receive a physical downlink control channel (PDCCH) intended for the UE and a number of the at least one transport block is not equal to a number of the plurality of transport blocks.