The main goal of this work is to calculate the contributions of bound-bound transitions of helium to the cosmological recombination spectrum. We show that helium in the early Universe causes unique features to appear in the total cosmological recombination spectrum. These may provide a unique observational possibility to determine the relative abundance of primordial helium, well before the formation of first stars. We include the effect of the tiny fraction of neutral hydrogen atoms on the dynamics of He II → He I recombination at redshifts z ~ 2500. As discussed recently, this process significantly accelerates He II → He I recombination, resulting in rather narrow and distinct features in the associated recombination spectrum. In addition this process induces some emission within the hydrogen Lyman-α line, before the actual epoch of hydrogen recombination around z ~ 1100-1500. We also show that some of the fine-structure transitions of neutral helium appear in absorption, again leaving unique traces in the cosmic microwave background blackbody spectrum, which may allow confirmation of our understanding of the early Universe and of detailed atomic physics.