We study the spectral energy distributions, SEDs (from FUV to MIR bands), of the first sizeable sample of 34 low-luminosity radio galaxies at high redshifts, selected in the COSMOS field. To model the SEDs, we use two different template-fitting techniques: (1) the Hyperz code that only considers single stellar templates and (2) our own developed technique 2SPD that also includes the contribution from a young stellar population and dust emission. The resulting photometric redshifts range from z ~ 0.7 to 3 and are in substantial agreement with measurements from earlier work, but significantly more accurate. The SED of most objects is consistent with a dominant contribution from an old stellar population with an age ~1-3 × 109 years. The inferred total stellar mass range is ~1010-1012 M ☉. Dust emission is needed to account for the 24 μm emission in 15 objects. Estimates of the dust luminosity yield values in the range L dust ~ 1043.5-1045.5 erg s–1. The global dust temperature, crudely estimated for the sources with an MIR excess, is ~300-850 K. A UV excess is often observed with a luminosity in the range ~1042-1044 erg s–1 at 2000 Å rest frame. Our results show that the hosts of these high-z low-luminosity radio sources are old massive galaxies, similar to the local FR Is. However, the UV and MIR excesses indicate the possible significant contribution from star formation and/or nuclear activity in such bands, not seen in low-z FR Is. Our sources display a wide variety of properties: from possible quasars at the highest luminosities to low-luminosity old galaxies.