In the fifth-generation mobile communications system (5G), it is critical to ensure wireless security as well as large-capacity and high-speed communication. To achieve this, a chaos modulation method as an encrypted and channel-coded modulation method in the physical layer is proposed. However, in the conventional chaos modulation method, the decoding complexity increases exponentially with respect to the modulation order. To solve this problem, in this study, a hybrid modulation method that applies quadrature amplitude modulation (QAM) and chaos to reduce the amount of decoding complexity, in which some transmission bits are allocated to QAM while maintaining the encryption for all bits is proposed. In the proposed method, a low-complexity decoding method is constructed by ordering chaos and QAM symbols based on the theory of index modulation. Numerical results show that the proposed method maintains good error-rate performance with reduced decoding complexity and ensures wireless security.