We compare the demodulation performance of an analog OTDM demultiplexing scheme and digitized OTDM demultiplexing with an ultrahigh-speed digital signal processor in a single-channel OTDM coherent Nyquist pulse transmission. We evaluated the demodulation performance for 40, 80, and 160Gbaud OTDM signals with a baseline rate of 10Gbaud. As a result, we clarified that the analog scheme performs significantly better since the bandwidth for handling the demultiplexed signal is as narrow as 10GHz regardless of the symbol rate. This enables us to use a low-speed A/D converter (ADC) with a large effective number of bits (ENOB). On the other hand, in the digital scheme, the higher the symbol rate becomes, the more bandwidth the receiver requires. Therefore, it is necessary to use an ultrahigh-speed ADC with a low ENOB for a 160Gbaud signal. We measured the ENOB of the ultrahigh-speed ADC used in the digital scheme and showed that the measured ENOB was approximately 1.5 bits lower than that of the low-speed ADC used in the analog scheme. This 1.5-bit decrease causes a large degradation in the demodulation performance obtained with the digital demultiplexing scheme.

In this paper, we report an experimental and numerical analysis of ultrahigh-speed coherent Nyquist pulse transmission. First, we describe a low-nonlinearity dispersion compensator for ultrahigh-speed coherent Nyquist pulse transmission; it is composed of a chirped fiber Bragg grating (CFBG) and a liquid crystal on silicon (LCoS) device. By adopting CFBG instead of inverse dispersion fiber, the nonlinearity in a 160km transmission line was more than halved. Furthermore, by eliminating the group delay fluctuation of the CFBG with an LCoS device, the residual group delay was reduced to as low as 1.42ps over an 11nm bandwidth. Then, by using the transmission line with the newly constructed low-nonlinearity dispersion compensator, we succeeded in improving the BER performance of single-channel 15.3Tbit/s-160km transmission by one-third compared with that of a conventional dispersion-managed transmission line and obtained a spectral efficiency of 8.7bit/s/Hz. Furthermore, we numerically analyzed the BER performance of its Nyquist pulse transmission. The numerical results showed that the nonlinear impairment in the transmission line is the main factor limiting the transmission performance in a coherent Nyquist pulse transmission, which becomes more significant at higher baud rates.