We have studied the radial structure of the stellar mass surface density (μ∗) and stellar population age as a function of the total stellar mass and morphology for a sample of 107 galaxies from the CALIFA survey. We applied the fossil record method based on spectral synthesis techniques to recover the star formation history (SFH), resolved in space and time, in spheroidal and disk dominated galaxies with masses from 109 to 1012 M⊙. We derived the half-mass radius, and we found that galaxies are on average 15% more compact in mass than in light. The ratio of half-mass radius to half-light radius (HLR) shows a dual dependence with galaxy stellar mass; it decreases with increasing mass for disk galaxies, but is almost constant in spheroidal galaxies. In terms of integrated versus spatially resolved properties, we find that the galaxy-averaged stellar population age, stellar extinction, and μ∗ are well represented by their values at 1 HLR. Negative radial gradients of the stellar population ages are present in most of the galaxies, supporting an inside-out formation. The larger inner (≤1 HLR) age gradients occur in the most massive (1011 M⊙) disk galaxies that have the most prominent bulges; shallower age gradients are obtained in spheroids of similar mass. Disk and spheroidal galaxies show negative μ∗ gradients that steepen with stellar mass. In spheroidal galaxies, μ∗ saturates at a critical value (~7 × 102 M⊙/pc2 at 1 HLR) that is independent of the galaxy mass. Thus, all the massive spheroidal galaxies have similar local μ∗ at the same distance (in HLR units) from the nucleus. The SFH of the regions beyond 1 HLR are well correlated with their local μ∗, and follow the same relation as the galaxy-averaged age and μ∗; this suggests that local stellar mass surface density preserves the SFH of disks. The SFH of bulges are, however, more fundamentally related to the total stellar mass, since the radial structure of the stellar age changes with galaxy mass even though all the spheroid dominated galaxies have similar radial structure in μ∗. Thus, galaxy mass is a more fundamental property in spheroidal systems, while the local stellar mass surface density is more important in disks. Table 1 and appendices are available in electronic form at http://www.aanda.org