Recent evidence based independently on spectral line strengths and dynamical modelling point towards a non-universal stellar initial mass function (IMF), probably implying an excess of low-mass stars in elliptical galaxies with a high velocity dispersion. Here, we show that a time-independent bottom-heavy IMF is compatible neither with the observed metal-rich populations found in giant ellipticals nor with the number of stellar remnants observed within these systems. We suggest a two-stage formation scenario involving a time-dependent IMF to reconcile these observational constraints. In this model, an early strong starbursting stage with a top-heavy IMF is followed by a more prolonged stage with a bottom-heavy IMF. Such model is physically motivated by the fact that a sustained high star formation will bring the interstellar medium to a state of pressure, temperature and turbulence that can drastically alter the fragmentation of the gaseous component into small clumps, promoting the formation of low-mass stars. This toy model is in good agreement with the different observational constrains on massive elliptical galaxies, such as age, metallicity, α-enhancement, mass-to-light ratio or the mass fraction of the stellar component in low-mass stars.