Directly imaged substellar companions around stars are key objects for better understanding the formation and evolution of brown dwarfs and massive planets, as we can infer physical properties like the age, distance or metallicity from their host stars. Moreover, we can characterize them spectroscopically, seeking for features that may provide information on the properties of gas giant exoplanets, in particular, on the composition of their complex atmospheres. Here we outline our program aiming to identify and characterize substellar companions to stars using the ongoing VISTA Hemisphere Survey (VHS). Focusing on the young L and T dwarfs and objects from the frontier of the Y-type dwarfs we enter the domain of planetary masses and below 1000 K temperatures.