The life cycle of baryonic matter in a galaxy is driven by the exchange of material between the interstellar medium (ISM) and stars, which are the agents of galaxy evolution. Dust is present at these key transition phases of matter: in the ISM, in the circumstellar environments of newly forming stars and in stellar ejecta of dying stars. The Spitzer and Herschel wavelengths provide a sensitive probe of circumstellar and interstellar dust and hence, allows us to study the physical processes of the ISM, the formation of new stars and the injection of mass by evolved stars and their relationships on a galaxy-wide scale. Due to their proximity, well constrained viewing angle, multi-wavelength information, and measured tidal interactions with the Milky Way (MW), the Large Magellanic Cloud (LMC) and Small Magellanic Cloud (SMC) are uniquely suited for surveying the agents of galaxy evolution (SAGE), the ISM and stars. In this talk, I will present some key results from the Spitzer SAGE and Herschel HERITAGE surveys including measurements of ISM mass estimates from dust emission, discoveries of thousands of young stellar object candidates, and precise measurements of dust mass loss rates from entire populations of evolved stars and w the mass budgets of these galaxies. I will end with a brief forward look to the future prospects with the James Webb Space Telescope Mission.