In this paper, we propose a noncoherently non-catastrophic trellis-coded modulation scheme, in which the transmitter includes a differential encoder, a rotator, an inverse signal mapper, a convolutional encoder and a signal mapper. We present examples of the proposed scheme including MPSK (M-ary phase shift keying), QAM (quadrature-amplitude modulation), and TAPSK (twisted amplitude and phase shift keying). For trellis-coded QAM, a differential encoder with which the complexity of the proposed scheme can be reduced is proposed. Simulation results demonstrate that for noncoherent decoding, the proposed trellis-coded QAM has much better error performance than conventional trellis-coded QAM, and the proposed trellis-coded 16APSK outperforms trellis-coded 16QAM for short observation length.
