We investigate the radial variation of the magnetic field structure across sunspots, pores and azimuth centers (ACs). We define ACs as magnetic structures of about the same size as pores (all structures studied here are larger than 3 Mm diameter), but without a clear (at least 5%) continuum decrease associated with them. We start from the full 3D vector fields as observed with the Advanced Stokes Polarimeter (ASP), and perform a statistical study of the azimuthally averaged field components in the local cylindrical reference frame centered on the structures. Our statistical study comprises a sample of 16 sunspot observations, a sample of 51 pores, and a sample of 22 ACs. For all structures, we derive mean radial profiles and their standard deviations. Due to the relatively large sample of pores, we are able to investigate variations of this mean radial field structure with the size of the pores. On the basis of our statistics, we identify systematic changes in the magnetic field structure over a considerable size range. We suggest how this may be the natural consequence of a formation scenario for the largest pores, by a lateral clustering of magnetic elements. Indeed, in this process, an AC may develop into a dark pore and gradually grow in size through the incremental addition of magnetic flux. Several observations where ACs turn into pores provide an estimate of about 4-5x10^19^Mx for the critical magnetic flux at which such transitions occur. We confirm the existence of a magnetic canopy for pores of all sizes, as their magnetic extent is virtually always larger than the associated continuum darkening. We observe a relatively rapid change in the continuum appearance of a large pore in the sample. We identify the associated changes in the field structure, and confront it with the determined mean field variation across sunspots. It appears that we have witnessed the formation of a partial penumbra.