Abstract Aim Island taxa often attain forms outside the range achieved by mainland relatives. Body size evolution of vertebrates on islands has therefore received much attention, with two seemingly conflicting patterns thought to prevail: (1) islands harbour animals of extreme size, and (2) islands promote evolution towards medium body size (‘the island rule’). We test both hypotheses using body size distributions of mammal, lizard and bird species. Location World-wide. Methods We assembled body size and insularity datasets for the world’s lizards, birds and mammals. We compared the frequencies with which the largest or smallest member of a group is insular with the frequencies expected if insularity is randomly assigned within groups. We tested whether size extremes on islands considered across mammalian phylogeny depart from a null expectation under a Brownian motion model. We tested the island rule by comparing insular and mainland members of (1) a taxonomic level and (2) mammalian sister species, to determine if large insular animals tend to evolve smaller body sizes while small ones evolve larger sizes. Results The smallest species in a taxon (order, family or genus) are insular no more often than would be expected by chance in all groups. The largest species within lizard families and bird genera (but no other taxonomic levels) are insular more often than expected. The incidence of extreme sizes in insular mammals never departs from the null, except among extant genera, where gigantism is marginally less common than expected under a Brownian motion null. Mammals follow the island rule at the genus level and when comparing sister species and clades. This appears to be driven mainly by insular dwarfing in large-bodied lineages. A similar pattern in birds is apparent for species within orders. However, lizards follow the converse pattern. Main conclusions The popular misconception that islands have more than their fair share of size extremes may stem from a greater tendency to notice gigantism and dwarfism when they occur on islands. There is compelling evidence for insular dwarfing in large mammals, but not in other taxa, and little evidence for the second component of the island rule – gigantism in small-bodied taxa.