Context: L-type ultra-cool dwarfs and brown dwarfs have cloudy atmospheres that could host weather-like phenomena. The detection of photometric or spectral variability would provide insight into unresolved atmospheric heterogeneities, such as holes in a global cloud deck. Indeed, a number of ultra-cool dwarfs have been reported to vary. Additional time-resolved spectral observations of brown dwarfs offer the opportunity for further constraining and characterising atmospheric variability. Aims: It has been proposed that growth of heterogeneities in the global cloud deck may account for the L- to T-type transition when brown dwarf photospheres evolve from cloudy to clear conditions. Such a mechanism is compatible with variability. We searched for variability in the spectra of five L6 to T6 brown dwarfs to test this hypothesis. Methods: We obtained spectroscopic time series using the near-infrared spectrographs ISAAC on VLT-ANTU, over 0.99-1.13 μm, and SpeX on the Infrared Telescope Facility for two of our targets in the J, H, and K bands. We searched for statistically variable lines and for a correlation between those. Results: High spectral-frequency variations are seen in some objects, but these detections are marginal and need to be confirmed. We find no evidence of large-amplitude variations in spectral morphology and we place firm upper limits of 2 to 3% on broad-band variability, depending on the targets and wavelengths, on the time scale of a few hours. In contrast to the rest of the sample, the T2 transition brown dwarf SDSS J1254-0122 shows numerous variable features, but a secure variability diagnosis would require further observations. Conclusions: Assuming that any variability arises from the rotation of patterns of large-scale clear and cloudy regions across the surface, we find that the typical physical scale of cloud-cover disruption should be smaller than 5-8% of the disk area for four of our targets, using simplistic heterogeneous atmospheric models. The possible variations seen in SDSS J1254-0122 are not strong enough to allow us to confirm the cloud-breaking hypothesis. Based on observations obtained at the European Observatory, Paranal, Chile, under programme 71.C-0559.