We examine the latest results concerning the evolution in the structures of galaxies from the local universe up to z ~ 6. We present results from the COSMOS, EGS and UDF surveys and characterise the structure of galaxies in terms of stellar masses. We find in general that galaxy structure becomes more asymmetric with time, and we use this information to determine the merger history of galaxies and the role of mergers in galaxy formation, placing the first firm constraints on the importance of this formation mode.