We study the evolution of galaxy structure since z ~ 1 to the present. From a Great Observatories Origins Deep Survey South (GOODS-S) multi-band catalog, we define (blue) luminosity- and mass-weighted samples, limited by MB <= -20 and M sstarf >= 1010 M sun, comprising 1122 and 987 galaxies, respectively. We extract early-type (ET; E/S0/Sa) and late-type (LT; Sb-Irr) subsamples by their position in the concentration-asymmetry plane, in which galaxies exhibit a clear bimodality. We find that the ET fraction, f ET, rises with cosmic time, with a corresponding decrease in the LT fraction, f LT, in both luminosity- and mass-selected samples. However, the evolution of the comoving number density is very different: the decrease in the total number density of MB <= -20 galaxies since z = 1 is due to the decrease in the LT population, which accounts for ~75% of the total star formation rate in the range under study, while the increase in the total number density of M sstarf >= 1010 M sun galaxies in the same redshift range is due to the evolution of ETs. This suggests that we need a structural transformation between LT galaxies that form stars actively and ET galaxies in which the stellar mass is located. Comparing the observed evolution with the gas-rich major merger rate in GOODS-S, we infer that only ~20% of the new ET galaxies with M sstarf >= 1010 M sun appeared since z ~ 1 can be explained by this kind of mergers, suggesting that minor mergers and secular processes may be the driving mechanisms of the structural evolution of intermediate-mass (M sstarf ~ 4 × 1010 M sun) galaxies since z ~ 1.