The chromosphere of the quiet Sun is a very important stellar atmospheric region whose thermal and magnetic structure we need to decipher in order to unlock new discoveries in solar and stellar physics. To this end, we need to identify and exploit observables sensitive to weak magnetic fields (B <~ 100 G) and to the presence of cool and hot gas in the bulk of the solar chromosphere. Here, we report on an investigation of the Hanle effect in two semi-empirical models of the quiet solar atmosphere with different chromospheric thermal structures. Our study reveals that the linear polarization profiles produced by scattering in the Ca II IR triplet have thermal and magnetic sensitivities potentially of great diagnostic value. The linear polarization in the 8498 Å line shows a strong sensitivity to inclined magnetic fields with strengths between 0.001 and 10 G, while the emergent linear polarization in the 8542 Å and 8662 Å lines is mainly sensitive to magnetic fields with strengths between 0.001 and 0.1 G. The reason for this is that the scattering polarization of the 8542 Å and 8662 Å lines, unlike the 8498 Å line, is controlled mainly by the Hanle effect in their (metastable) lower levels. Therefore, in regions with magnetic strengths noticeably larger than 1 G, their Stokes Q and U profiles are sensitive only to the orientation of the magnetic field vector. We also find that for given magnetic field configurations the sign of the Q/I and U/I profiles of the 8542 Å and 8662 Å lines is the same in both atmospheric models, while the sign of the linear polarization profile of the 8498 Å line turns out to be very sensitive to the thermal structure of the lower chromosphere. We suggest that spectropolarimetric observations providing information on the relative scattering polarization amplitudes of the Ca II IR triplet will be very useful to improve our empirical understanding of the thermal and magnetic structure of the quiet chromosphere.