It is shown that the two-part minimum description length principle can be used to discriminate among different models that can explain a given observed data set. The description length is chosen to be the sum of the lengths of the message needed to encode the model plus the message needed to encode the data when the model is applied to the data set. It is verified that the proposed principle can efficiently distinguish the model that correctly fits the observations while avoiding overfitting. The capabilities of this criterion are shown in two simple problems for the analysis of observed spectropolarimetric signals. The first is the denoising of observations with the aid of the PCA technique. The second is the selection of the optimal number of parameters in LTE inversions. We propose this criterion as a quantitative approach for distinguishing the most plausible model among a set of proposed models. This quantity is very easy to implement as an additional output on the existing inversion codes.