In this paper, we report an experimental and numerical analysis of ultrahigh-speed coherent Nyquist pulse transmission. First, we describe a low-nonlinearity dispersion compensator for ultrahigh-speed coherent Nyquist pulse transmission; it is composed of a chirped fiber Bragg grating (CFBG) and a liquid crystal on silicon (LCoS) device. By adopting CFBG instead of inverse dispersion fiber, the nonlinearity in a 160km transmission line was more than halved. Furthermore, by eliminating the group delay fluctuation of the CFBG with an LCoS device, the residual group delay was reduced to as low as 1.42ps over an 11nm bandwidth. Then, by using the transmission line with the newly constructed low-nonlinearity dispersion compensator, we succeeded in improving the BER performance of single-channel 15.3Tbit/s-160km transmission by one-third compared with that of a conventional dispersion-managed transmission line and obtained a spectral efficiency of 8.7bit/s/Hz. Furthermore, we numerically analyzed the BER performance of its Nyquist pulse transmission. The numerical results showed that the nonlinear impairment in the transmission line is the main factor limiting the transmission performance in a coherent Nyquist pulse transmission, which becomes more significant at higher baud rates.

This paper investigates approaches that can cancel nonlinear phase noise effectively for the phase-conjugate pair diversity transmission of 16-QAM WDM signals through multi-core fiber. The geometric mean is introduced for the combination of the phase-conjugate pair. A numerical simulation suggests that span-by-span chromatic dispersion compensation is more effective at cancelling phase noise in long distance transmission than lumped compensation at the receiver. Simulations suggest the span-wise compensation described herein yields Q-value enhancement of 7.8 and 6.8dB for CD values of 10 and 20.6ps/nm/km, respectively, whereas the lumped compensation equivalent attains only 3.5dB. A 1050km recirculating loop experiment confirmed a Q-value enhancement of 4.1dB for 20.6ps/nm/km, span-wise compensation transmission.

Four-wave mixing (FWM) compensation using digital coherent detection is experimentally demonstrated. Two lights and the induced FWM components are combined with phase-locked local oscillator lights and received individually. The received signals are combined electrically and the signal-to-FWM crosstalk ratio is improved to more than 30 dB by backward propagation.

A 1.6-Tb/s dense WDM signal was successfully transmitted over 480 km using the carrier-suppressed return-to-zero (CS-RZ) modulation format. The CS-RZ format was chosen because it exhibited better transmission performance over a wide fiber-input power window than the NRZ and RZ formats in a 40-Gb/s-based WDM transmission experiment with 100-GHz channel spacing, confirming its nonlinearity-insensitive nature in dense WDM systems. With the wide power window of CS-RZ, we achieved stable transmission of 4040-Gb/s WDM signals over a 480-km (680 km) standard SMF line with only the C-band, in which a spectral ripple remained during transmission. Distributed Raman amplification and forward error correction were not used, providing a margin for already installed transmission lines.