In this paper we discuss the results of a programme of spectral synthesis modelling of a sample of starburst radio galaxies in the context of scenarios for the triggering of the activity and the evolution of the host galaxies. New optical spectra are also presented for a subset of the objects discussed. The starburst radio galaxies - comprising ˜15-25 per cent of all powerful extragalactic radio sources - frequently show disturbed morphologies at optical wavelengths, and unusual radio structures, although their stellar masses are typical of radio galaxies as a class. In terms of the characteristic ages of their young stellar populations (YSPs), the objects can be divided into two groups: those with YSP ages tYSP≤ 0.1 Gyr, in which the radio source has been triggered quasi-simultaneously with the main starburst episode, and those with older YSP in which the radio source has been triggered or re-triggered a significant period after the starburst episode. Most of the former group are associated with a large mid- to far-IR (MFIR) continuum and [O III] emission-line luminosities (LIR > 1011 L&sun;, ? W), while most of the latter have lower luminosities. Combining the information on the YSP with that on the optical morphologies of the host galaxies, we deduce that the majority of the starburst radio galaxies have been triggered in galaxy mergers in which at least one of the galaxies is gas rich. However, the triggering (or re-triggering) of the radio jets can occur immediately before, around or a significant period after the final coalescence of the merging nuclei, reflecting the complex gas infall histories of the merger events. Although ˜25 per cent of starburst radio galaxies are sufficiently bright at MFIR wavelengths to be classified as ultraluminous infrared galaxies (ULIRGs), we show that only the most massive ULIRGs are capable of evolving into radio galaxies. Finally, for a small subset of starburst radio galaxies in rich clusters of galaxies, cooling flows associated with the hot X-ray haloes offer a viable alternative to mergers as a trigger for the radio jet activity. Overall, our results provide further evidence that a powerful radio jet activity can be triggered via a variety of mechanisms, including different evolutionary stages of major galaxy mergers; clearly, radio-loud AGN activity is not solely associated with a particular stage of a unique type of gas accretion event.