Context. Luminous and ultraluminous infrared galaxies (LIRGs and ULIRGs) are much more numerous at higher redshifts than locally, dominating the star-formation rate density at redshifts ~1-2. Therefore, they are important objects in order to understand how galaxies form and evolve through cosmic time. Local samples provide a unique opportunity to study these objects in detail. Aims: We aim to characterize the morphologies of the stellar continuum and the ionized gas (Hα) emissions from local sources, and investigate how they relate with the dynamical status and IR-luminosity of the sources. Methods: We use optical (5250-7450 Å) integral field spectroscopic (IFS) data for a representative sample of 38 sources (31 LIRGs and 7 ULIRGs), taken with the VIMOS instrument on the VLT. Results: We present an atlas of IFS images of continuum emission, Hα emission, and Hα equivalent widths for the sample. The morphologies of the Hα emission are substantially different from those of the stellar continuum. The Hα images frequently reveal extended structures that are not visible in the continuum, such as HII regions in spiral arms, tidal tails, rings, bridges, of up to few kpc from the nuclear regions. The morphologies of the continuum and Hα images are studied on the basis of the C2 kpc parameter, which measures the concentration of the emission within the central 2 kpc. The C2 kpc values found for the Hα images are higher than those of the continuum for the majority (85%) of the objects in our sample. On the other hand, most of the objects in our sample (~62%) have more than half of their Hα emission outside the central 2 kpc. No clear trends are found between the values of C2 kpc and the IR-luminosity of the sources. On the other hand, our results suggest that the star formation in advance mergers and early-stage interactions is more concentrated than in isolated objects. Finally, we compared the Hα and infrared emissions as tracers of the star-formation activity. We find that the star-formation rates derived using the Hα luminosities generally underpredict those derived using the IR luminosities, even after accounting for reddening effects.