We provide on overview of fundamental technologies and recent challenges on extremely higher-order quadrature amplitude modulation (QAM) such as 256–1024 levels, toward the realization of an ultrahigh spectral efficiency approaching the Shannon limit. Key components required for such a higher-order QAM transmission are described in detail, including a coherent light source, an optical phase-locked loop, an IQ modulator, and a digital demodulator. We also present recent demonstrations of single-carrier 1024 QAM, 256 QAM-OFDM, and OTDM-RZ/32 QAM transmissions realized with these fundamental technologies.