Galactic cirrus emission at far-infrared wavelengths affects many extragalactic observations. Separating this emission from that associated with extragalactic objects is both important and difficult. In this paper we discuss a particular case, the M81 group, and the identification of diffuse structures prominent in the infrared, but also detected at optical wavelengths. The origin of these structures has previously been controversial, ranging from them being the result of a past interaction between M81 and M82 or due to more local Galactic emission. We show that over an order of a few arcmin scales, the far-infrared (Herschel 250μm) emission correlates spatially very well with a particular narrow-velocity (2-3 kms-1) component of the Galactic HI. We find no evidence that any of the far-infrared emission associated with these features actually originates in the M81 group. Thus we infer that the associated diffuse optical emission must be due to galactic light-back scattered off dust in our galaxy. Ultraviolet observations pick out young stellar associations around M81, but no detectable far-infrared emission. We consider in detail one of the Galactic cirrus features, finding that the far-infrared HI relation breaks down below arcmin scales and that at smaller scales there can be quite large dust-temperature variations.