The empirical literature of the influence of unions on productivity is extended by considering the effect in an industry with heterogeneous firms. Recent theoretical papers suggest that, in such an industry, unions will tend to organize the exogenously 'more productive' firms. Thus, a spurious correlation between unions and productivity may emerge. We test this hypothesis by estimating production functions for coal with data from Eastern Kentucky underground coal mines. The aspect of mine heterogeneity that we focus on is the width of the mine's seam of coal. Wider seams increase productivity. Empirically, we find that unions disproportionately organize mines with wider seams and this accounts for the positive relationship between unions and productivity observed in our data. In fact, once seam thickness is accounted for, the estimated effect of unions on productivity is negative.