Total spectral irradiance is typically modeled by assinging an atmospheric model to each pixel of a full disk image and geometricllay combining the predicted wavelength dependent intensity for each of these models into a disk integrated spectrum. This works reasonably well, as the hydrostatic models that are used in this procedure generally reproduce observed spectra very well. However, for numerical expedience this scheme neglects some important physical aspects of the the solar atmosphere, in particular its three-dimensional and strongly dynamic nature. In this talk I will discuss the importance of some of these effects on the spectral irradiance signal, using forward radiative transfer modeling in realistic three-dimenional simulations. Obviously, modeling the three-dimensional dynamic structure over the whole disk is computaionally prohibitive, but if some of the effects discused above are important, strategies will have to be implemented to incorporate them approximately. Characterizing these cotributions to the spectral irradiance will also help us to better understand the physical nature of the forces that drive variability, and hopefully improve our predictive capabilities.