We report an anticorrelation between continuum luminosity and the equivalent width (EW) of the Hα emission line in X-ray binary systems. The effect is evident both in a universal monotonic increase in Hα EW with time following outbursts, as systems fade, and in a comparison between measured EWs and contemporaneous X-ray measurements. The effect is most clear for black hole binaries in the low/hard X-ray state, which is prevalent at X-ray luminosities below ~1 per cent of the Eddington luminosity. We do not find strong evidence for significant changes in line profiles across accretion state changes, but this is hampered by a lack of good data at such times. The observed anticorrelation, highly significant for black hole binaries, is only marginally so for neutron star systems, for which there are far less data. Comparison with previously established correlations between optical and X-ray luminosity suggests that the line luminosity is falling as the X-ray and optical luminosities drop, but not as fast, approximately, as LHα ~ L~0.4X ~ L~0.7opt. We briefly discuss possible origins for such an effect, including the optical depth, form of the irradiating spectrum and geometry of the accretion flow. Further refinement of the relation in the future may allow measurements of Hα EW to be used to estimate the luminosity of, and hence the distance to, X-ray binary systems. Beyond this, further progress will require a better sample of spectrophotometric data.