In this paper, we address the on-demand end-to-end optical network construction problem for grid applications in new generation large-scale multi-domain wavelength switched optical networks (WSON). According to users' requests for high-performance distributed computing, groups of dedicated end-to-end lightpaths among geographically distributed grid resources can be established dynamically forming multiple-lightpath optical networks for grid applications, namely, optical grid network (OGN). To facilitate the automated OGN construction, we introduce an optical grid network infrastructure providing an integrated and self-contained OGN service to grid users with totally distributed control. In this infrastructure, for easy construction, especially in a large-scale multi-domain WSON environment, we propose an overlay approach to construct OGNs in a peer-to-peer fashion, which conceals the communication architecture of the underlying heterogeneous optical networks. In particular, we propose an adaptive construction mechanism that can develop the OGN flexibly by adapting to the dynamically changed optical network circumstance. To enable users to take the advantage of the end-to-end lightpaths of WSON directly, a wavelength-oriented end-host configuration scheme is proposed. Experimental results on a developed prototype and an optical-fibre test-bed network successfully validate the proposal.