Context: Individual stars in dwarf spheroidal galaxies around the Milky Way Galaxy have been studied both photometrically and spectroscopically. Extremely metal-poor stars among them are very valuable because they should record the early enrichment in the Local Group. However, our understanding of these stars is very limited because detailed chemical abundance measurements are needed from high resolution spectroscopy. Aims: To constrain the formation and chemical evolution of dwarf galaxies, metallicity and chemical composition of extremely metal-poor stars are investigated. Methods: Chemical abundances of six extremely metal-poor ([Fe/H] < -2.5) stars in the Sextans dwarf spheroidal galaxy are determined based on high resolution spectroscopy (R=40 000) with the Subaru Telescope High Dispersion Spectrograph. Results: (1) The Fe abundances derived from the high resolution spectra are in good agreement with the metallicity estimated from the Ca triplet lines in low resolution spectra. The lack of stars with [Fe/H] ⪉ -3 in Sextans, found by previous estimates from the Ca triplet, is confirmed by our measurements, although we note that high resolution spectroscopy for a larger sample of stars will be necessary to estimate the true fraction of stars with such low metallicity. (2) While one object shows an overabundance of Mg (similar to Galactic halo stars), the Mg/Fe ratios of the remaining five stars are similar to the solar value. This is the first time that low Mg/Fe ratios at such low metallicities have been found in a dwarf spheroidal galaxy. No evidence for over-abundances of Ca and Ti are found in these five stars, though the measurements for these elements are less certain. Possible mechanisms to produce low Mg/Fe ratios, with respect to that of Galactic halo stars, are discussed. (3) Ba is under-abundant in four objects, while the remaining two stars exhibit large and moderate excesses of this element. The abundance distribution of Ba in this galaxy is similar to that in the Galactic halo, indicating that the enrichment of heavy elements, probably by the r-process, started at metallicities [Fe/H] ≤ -2.5, as found in the Galactic halo. Based on data collected at Subaru Telescope, which is operated by the National Astronomical Observatory of Japan. Table [see full text] is only available in electronic form at http://www.aanda.org