Wave propagation through sunspots involves conversion between waves of acoustic and magnetic character. In addition, the thermal structure of sunspots is very different than that of the quiet Sun. As a consequence, the interpretation of local helioseismic measurements of sunspots has long been a challenge. With the aim of understanding these measurements, we carry out numerical simulations of wave propagation through sunspots. Helioseismic holography measurements made from the resulting simulated wavefields show qualitative agreement with observations of real sunspots. We use additional numerical experiments to determine, separately, the influence of the thermal structure of the sunspot and the direct effect of the sunspot magnetic field. We use the ray approximation to show that the travel-time shifts in the thermal (non-magnetic) sunspot model are primarily produced by changes in the wave path due to the Wilson depression rather than variations in the wave speed. This shows that inversions for the subsurface structure of sunspots must account for local changes in the density. In some ranges of horizontal phase speed and frequency there is agreement (within the noise level in the simulations) between the travel times measured in the full magnetic sunspot model and the thermal model. If this conclusion proves to be robust for a wide range of models, it would suggest a path toward inversions for sunspot structure.