In this paper, a non-linear precoding algorithm with low out-of-band (OOB) radiation is proposed for massive multiple-input multiple-output (MIMO) systems. Massive MIMO sets more than one hundred antennas at each base station to achieve higher spectral efficiency and throughput. Full digital massive MIMO may constrain the resolution of digital-to-analog converters (DACs) since each DAC consumes a large amount of power. In massive MIMO systems with low resolution DACs, designing methods of DAC output signals by nonlinear processing are being investigated. The conventional scheme focuses only on a sum rate or errors in the received signals and so triggers large OOB radiation. This paper proposes an optimization criterion that takes OOB radiation power into account. Gibbs sampling is used as an algorithm to find sub-optimal solutions given this criterion. Numerical results obtained through computer simulation show that the proposed criterion reduces mean OOB radiation power by a factor of 10 as compared with the conventional criterion. The proposed criterion also reduces OOB radiation while increasing the average sum rate by optimizing the weight factor for the OOB radiation. As a result, the proposed criterion achieves approximately 1.3 times higher average sum rates than an error-based criterion. On the other hand, as compared with a sum rate based criterion, the throughput on each subcarrier shows less variation which reduces the number of link adaptation options needed although the average sum rate of the proposed criterion is smaller.

The sum rate performance of nonlinier quantized precoding using Gibbs sampling are evaluated in a massive multiuser multiple-input multiple-output (MU-MIMO) system in this paper. Massive MU-MIMO is a key technology to handle the growth of data traffic. In a full digital massive MU-MIMO system, however, the resolution of digital-to-analogue converters (DACs) in transmit antenna branches have to be low to yield acceptable power consumption. Thus, a combinational optimization problem is solved for the nonlinier quantized precoding to determine transmit signals from finite alphabets output from low resolution DACs. A conventional optimization criterion minimizes errors between desired signals and received signals at user equipments (UEs). However, the system sum rate may decrease as it increases the transmit power. This paper proposes two optimization criteria that take the transmit power into account in order to maximize the sum rate. Mixed Gibbs sampling is applied to obtain the suboptimal solution of the nonlinear optimization problem. Numerical results obtained through computer simulations show that the two proposed criteria achieve higher sum rates than the conventional criterion. On the other hand, the sum rate criterion achieves the largest sum rate while it leads to less throughputs than the MMSE criterion on approximately 60% of subcarriers.