It has often been claimed that there is a natural or universal language of gesture, one that communicates directly without words. The present essay critically scrutinizes some older efforts to recover such a natural language of gesture, including John Bulwer’s seventeenth-century Chirologia and Chironomia, and the classical South Asian systems of Natyashastra and Tantra. Although these older systems lay claim to communicate and even to represent nature directly, often by virtue of a posited iconic or mimetic relationship to nature, closer examination reveals that they signify only in terms of a conventional code. Moreover, the very effort to replace gesture with a verbal gloss held to constitute its “meaning” reveals a gap between signifier and signified that is analogous to the gap between culture and nature, as well as present and past, that these systems of gesture attempt to bridge rhetorically. These systems demonstrate, not the existence of a natural language of gesture, but rather the function of the idea of such a language in an ideology in which gesture is to nature as speech is to culture. Although the failure of such earlier systems does not vitiate modern, scientific studies of gesture, it does suggest greater circumspection regarding what may be hoped to be achieved by such studies.