The combination of high-contrast imaging and high-dispersion spectroscopy, which has successfully been use to detect the atmosphere of a giant planet, is one of the most promising potential probes of the atmosphere of Earth-size worlds. The forthcoming generation of extremely large telescopes (ELTs) may obtain sufficient contrast with this technique to detect O2 in the atmosphere of those worlds that orbit low-mass M dwarfs. This is strong motivation to carry out a census of planets around cool stars for which habitable zones can be resolved by ELTs, i.e. for M dwarfs within 5 parsec. Our HARPS survey has been a major contributor to that sample of nearby planets. Here we report on our radial velocity observations of Ross 128 (Proxima Virginis, GJ447, HIP 57548), an M4 dwarf just 3.4 parsec away from our Sun. This source hosts an exo-Earth with a projected mass m sini = 1.35 M⊕ and an orbital period of 9.9 days. Ross 128 b receives less than 1.5 times as much flux as Earth from the Sun and its equilibrium ranges in temperature between 269 K for an Earth-like albedo and 213 K for a Venus-like albedo. Recent studies place it close to the inner edge of the conventional habitable zone. An 80-day long light curve from K2 campaign C01 demonstrates that Ross 128 b does not transit. Together with the All Sky Automated Survey (ASAS) photometry and spectroscopic activity indices, the K2 photometry shows that Ross 128 rotates slowly and has weak magnetic activity. In a habitability context, this makes survival of its atmosphere against erosion more likely. Ross 128 b is the second closest known exo-Earth, after Proxima Centauri b (1.3 parsec), and the closest temperate planet known around a quiet star. The 15 mas planet-star angular separation at maximum elongation will be resolved by ELTs (>3λ/D) in the optical bands of O2. Based on observations made with the HARPS instrument on the ESO 3.6 m telescope under the programme IDs 072.C-0488(A), 183.C-0437(A), and 191.C-0873(A) at Cerro La Silla (Chile).Radial velocity data (Table 5) are only available at the CDS via anonymous ftp to cdsarc.u-strasbg.fr (130.79.128.5) or via http://cdsarc.u-strasbg.fr/viz-bin/qcat?J/A+A/613/A25