IGR J17091-3624 is a low-mass X-ray binary (LMXB), which received wide attention from the community thanks to its similarities with the bright black hole system GRS 1915+105. Both systems exhibit a wide range of highly structured X-ray variability during outburst, with time-scales from few seconds to tens of minutes, which make them unique in the study of mass accretion in LMXBs. In this work, we present a general overview into the long-term evolution of IGR J17091-3624, using Swift/XRT observations from the onset of the 2011-2013 outburst in 2011 February till the end of the last bright outburst in 2016 November. We found four re-flares during the decay of the 2011 outburst, but no similar re-flares appear to be present in the latter one. We studied, in detail, the period with the lowest flux observed in the last 10 yr, just at the tail end of the 2011-2013 outburst, using Chandra and XMM-Newton observations. We observed changes in flux as high as a factor of 10 during this period of relative quiescence, without strong evidence of softening in the spectra. This result suggests that the source has not been observed at its true quiescence so far. By comparing the spectral properties at low luminosities of IGR J17091-3624 and those observed for a well-studied population of LMXBs, we concluded that IGR J17091-3624 is most likely to host a black hole as a compact companion rather than a neutron star.