HARMONI is a visible and near-IR integral field spectrograph, providing the E-ELT's spectroscopic capability at first light. It obtains simultaneous spectra of 32000 spaxels, at a range of resolving powers from R~4000 to R~20000, covering the wavelength range from 0.47 to 2.45 μm. The 256 × 128 spaxel field of view has four different plate scales, with the coarsest scale (40 mas) providing a 5″ × 10″ FoV, while the finest scale is a factor of 10 finer (4mas). We describe the opto-mechanical design of HARMONI, prior to the start of preliminary design, including the main subsystems - namely the image de-rotator, the scale-changing optics, the splitting and slicing optics, and the spectrographs. We also present the secondary guiding system, the pupil imaging optics, the field and pupil stops, the natural guide star wavefront sensor, and the calibration unit.