A method for optimizing a constellation for M-ary amplitude phase shift keying (M-APSK) that provides signal modulation for transmitting a series of digital bits in an analog signal by a transmitter to be received by a receiver over a communications channel. The method includes identifying symbol-wise mutual information of a transmitted constellation, where the symbol-wise mutual information includes a conditional probability density function. The method identifies an unconstrained capacity that defines an ultimate performance limit of the constellation, and minimizes a difference between a rate of the symbol-wise mutual information and a rate of the unconstrained capacity to determine the number of rings, the number of symbols in each ring and the ring ratio in the constellation.