It is essential to properly calibrate the polarimetric properties of telescopes, if one wants to take advantage of the capabilities of high precision spectro-polarimeters. We have constructed a model for the German Vacuum Tower Telescope (VTT) that describes its time-dependent polarization properties. Since the coelostat of the telescope changes the polarization state of the light by introducing cross talk among different polarization states, such a model is necessary to correct the measurements, in order to retrieve the true polarization as emitted from the Sun. The telescope model is quantified by a time-dependent Mueller matrix that depends on the geometry of the light beam through the telescope, and on material properties: the refractive indices of the coelostat mirrors, and the birefringence of the entrance window to the vacuum tube. These material properties were determined experimentally in-situ by feeding the telescope with known states of polarization (including unpolarized light) and by measuring its response, and from measurements of an aluminum-coated sample in the laboratory. Accuracy can in our case be determined only for the combination of telescope and spectro-polarimeter used; for the instrument POLIS at the VTT, we estimate an accuracy of ±4-5× 10-3 for the cross talk correction coefficients.