Numerous solar and space science and operations projects rely heavily on accurate, consistent magnetic field measurements from the solar surface. These projects are hindered by well-known but poorly understood discrepancies between magnetograms obtained with different solar telescopes and instrumentation. Existing efforts to characterize these discrepancies have mostly been limited to direct comparisons between final data products and have been inconclusive regarding the correct measurement. To attack this problem, we model every step of the line-of-sight (LOS) photospheric field measurement all the way to the final magnetogram. Beginning with known MHD simulation data for the magnetic field, the "ground truth," we compute for different viewing angles the radiative transfer for the Stokes spectra using the Rybicky and Hummer (RH) radiative transfer code. We then use the Solar Orbiter Polarimetric and Helioseismic Imager Software Simulator, adapted for the Solar Dynamics Observatory Helioseismic and Magnetic Imager (HMI) instrument, to simulate the instrument response to emergent polarized spectra. We model every significant process undergone by the solar signal during an observation: degradation by instrumental limitations including finite spatial and spectral resolution, Doppler shift variations due to the radial spacecraft orbital velocity, and the effects of Stokes inversion for the LOS magnetic field. Finally, we compare the simulated magnetograms with the MHD field data using the computed line formation information along each LOS and construct a detailed end-to-end magnetogram calibration. Effects of the calibration on real HMI magnetograms are discussed, including open magnetic flux estimates, and are compared with high-resolution data.