Progressive functional decline in the epilepsies is largely unexplained. We formed the ENIGMA-Epilepsy consortium to understand factors that influence brain measures in epilepsy, pooling data from 24 research centres in 14 countries across Europe, North and South America, Asia, and Australia. Structural brain measures were extracted from MRI brain scans across 2149 individuals with epilepsy, divided into four epilepsy subgroups including idiopathic generalized epilepsies (n =367), mesial temporal lobe epilepsies with hippocampal sclerosis (MTLE; left, n = 415; right, n = 339), and all other epilepsies in aggregate (n = 1026), and compared to 1727 matched healthy controls. We ranked brain structures in order of greatest differences between patients and controls, by meta-analysing effect sizes across 16 subcortical and 68 cortical brain regions. We also tested effects of duration of disease, age at onset, and age-by-diagnosis interactions on structural measures. We observed widespread patterns of altered subcortical volume and reduced cortical grey matter thickness. Compared to controls, all epilepsy groups showed lower volume in the right thalamus (Cohen's d = -0.24 to -0.73; P < 1.49 × 10-4), and lower thickness in the precentral gyri bilaterally (d = -0.34 to -0.52; P < 4.31 × 10-6). Both MTLE subgroups showed profound volume reduction in the ipsilateral hippocampus (d = -1.73 to -1.91, P < 1.4 × 10-19), and lower thickness in extrahippocampal cortical regions, including the precentral and paracentral gyri, compared to controls (d = -0.36 to -0.52; P < 1.49 × 10-4). Thickness differences of the ipsilateral temporopolar, parahippocampal, entorhinal, and fusiform gyri, contralateral pars triangularis, and bilateral precuneus, superior frontal and caudal middle frontal gyri were observed in left, but not right, MTLE (d = -0.29 to -0.54; P < 1.49 × 10-4). Contrastingly, thickness differences of the ipsilateral pars opercularis, and contralateral transverse temporal gyrus, were observed in right, but not left, MTLE (d = -0.27 to -0.51; P < 1.49 × 10-4). Lower subcortical volume and cortical thickness associated with a longer duration of epilepsy in the all-epilepsies, all-other-epilepsies, and right MTLE groups (beta, b < -0.0018; P < 1.49 × 10-4). In the largest neuroimaging study of epilepsy to date, we provide information on the common epilepsies that could not be realistically acquired in any other way. Our study provides a robust ranking of brain measures that can be further targeted for study in genetic and neuropathological studies. This worldwide initiative identifies patterns of shared grey matter reduction across epilepsy syndromes, and distinctive abnormalities between epilepsy syndromes, which inform our understanding of epilepsy as a network disorder, and indicate that certain epilepsy syndromes involve more widespread structural compromise than previously assumed.