We consider space time block coded-continuous phase modulation (STBC-CPM), which has the advantages of both STBC and CPM at the same time. A weak point of STBC-CPM is that the normalized spectral efficiency (NSE) is limited by the orthogonality of the STBC and CPM parameters. The purpose of this study is to improve the NSE of STBC-CPM. The NSE depends on the transmission rate (TR), the bit error rate (BER) and the occupied bandwidth (OBW). First, to improve the TR, we adapt quasi orthogonal-STBC (QO-STBC) for four transmit antennas and quasi-group orthogonal Toeplitz code (Q-GOTC) for eight transmit antennas, at the expense of the orthogonality. Second, to evaluate the BER, we derive a BER approximation of STBC-CPM with non-orthogonal STBC (NO-STBC). The theoretical analysis and simulation results show that the NSE can be improved by using QO-STBC and Q-GOTC. Third, the OBW depends on CPM parameters, therefore, the tradeoff between the NSE and the CPM parameters is considered. A computer simulation provides a candidate set of CPM parameters which have better NSE. Finally, the adaptation of non-orthogonal STBC to STBC-CPM can be viewed as a generalization of the study by Silvester et al., because orthogonal STBC can be thought of as a special case of non-orthogonal STBC. Also, the adaptation of Q-GOTC to CPM can be viewed as a generalization of our previous letter, because linear modulation scheme can be thought of as a special case of non-linear modulation.

In this letter, the tradeoff between symbol rate and diversity gain of Space-Time Block Codes (STBCs) with linear receivers is considered. It is known that Group Orthogonal-Toeplitz Codes (GOTCs) can achieve a good tradeoff with linear receivers. However, the symbol rate of GOTCs is limited to that of the base Orthogonal Space-Time Block Codes (OSTBCs). We propose to simply change the GOTC base codes from OSTBCs to Quasi-Orthogonal Space-Time Block Codes (Q-OSTBCs). Q-OSTBCs can improve the symbol rate of GOTCs at the expense of diversity gain. Simulation results show that Q-OSTBC based GOTCs can improve the tradeoff between symbol rate and diversity gain over that of the original GOTCs.