Context. Characterizing the masses, radii, and compositions of small planets orbiting M dwarfs is key to understanding their formation and identifying the best targets for atmospheric follow-up with facilities such as JWST. Aims. We present the characterization of two planetary systems orbiting the M dwarfs TOI-4336 A (M3.5V) and TOI-4342 (M0V), each hosting two transiting planets previously validated with TESS and ground-based observations. Methods. We refined the photometry of the TOI-4342 system using TESS and LCOGT data, and characterized the host stars with NIRPS and ESPRESSO spectroscopy. High-precision ESPRESSO radial velocities (RVs) allowed us to constrain the planetary masses and investigate their potential compositions. Results. The TOI-4336 A system is composed of a sub-Neptune with a period of 16.34 days, a radius of 2.14 ± 0.08 R⊕, and a mass of 3.33 ± 0.36 M⊕, along with an inner super-Earth on a 7.59-day orbit with a radius of 1.25 ± 0.07 R⊕ and a mass of 1.55 ± 0.13 M⊕. The TOI-4342 system hosts two sub-Neptunes of similar sizes (2.33 ± 0.09 R⊕ and 2.35 ± 0.09 R⊕), with periods of 5.54 and 10.69 days. Their masses are measured to be 7.3 ± 1.3 M⊕ and 4.8 ± 1.4 M⊕, respectively. The RVs also reveal a planet candidate around TOI-4342, most likely non-transiting, with a period of 47.5 days and a minimum mass of 17.8 ± 3.0 M⊕. Conclusions. With precise radii and masses, we derived bulk densities and explored possible compositions. The TOI-4336 A subNeptune and super-Earth have densities of 1.87 ± 0.30 and 4.35 ± 0.79 g cm−3, while the two similar-sized sub-Neptunes in TOI-4342 show distinct densities of 3.18 ± 0.67 and 2.01 ± 0.63 g cm−3. Using an inference model, we find that TOI-4336 A b, TOI-4342 b, and TOI-4342 c have an atmosphere mass fraction (AMF) of ∼3.7%, ∼1.8%, and ∼2.9%, respectively, while the super-Earth TOI-4336 A c could contain ∼2% of water or have a core-to-mass fraction (CMF) of ∼31%. All four planets are excellent targets for future atmospheric characterization with JWST, and their multi-planet nature makes them especially interesting for comparative planetology. Notably, TOI-4336 A b stands out as one of the best known targets in its size and temperature regime, with a transmission spectroscopy metric (TSM) of 138, comparable to benchmark planets such as K2-18 b and LHS 1140 b. Its inner sibling, TOI-4336 A c, may also be of interest for emission spectroscopy and exploring the "cosmic shoreline", similarly to the Rocky Worlds DDT JWST program.