Context. We still do not understand which physical mechanisms are responsible for the transport of angular momentum inside stars. The recent detection of mixed modes that contain the clear signature of rotation in the spectra of Kepler subgiants and red giants gives us the opportunity to make progress on this question. Aims: Our aim is to probe the radial dependence of the rotation profiles for a sample of Kepler targets. For this purpose, subgiants and early red giants are particularly interesting targets because their rotational splittings are more sensitive to the rotation outside the deeper core than is the case for their more evolved counterparts. Methods: We first extracted the rotational splittings and frequencies of the modes for six young Kepler red giants. We then performed a seismic modeling of these stars using the evolutionary codes Cesam2k and astec. By using the observed splittings and the rotational kernels of the optimal models, we inverted the internal rotation profiles of the six stars. Results: We obtain estimates of the core rotation rates for these stars, and upper limits to the rotation in their convective envelope. We show that the rotation contrast between the core and the envelope increases during the subgiant branch. Our results also suggest that the core of subgiants spins up with time, while their envelope spins down. For two of the stars, we show that a discontinuous rotation profile with a deep discontinuity reproduces the observed splittings significantly better than a smooth rotation profile. Interestingly, the depths that are found to be most probable for the discontinuities roughly coincide with the location of the H-burning shell, which separates the layers that contract from those that expand. Conclusions: We characterized the differential rotation pattern of six young giants with a range of metallicities, and with both radiative and convective cores on the main sequence. This will bring observational constraints to the scenarios of angular momentum transport in stars. Moreover, if the existence of sharp gradients in the rotation profiles of young red giants is confirmed, it is expected to help in distinguishing between the physical processes that could transport angular momentum in the subgiant and red giant branches. Appendices and Tables 3-9 are available in electronic form at http://www.aanda.org