One group of signs in French Sign Language (LSF) is described in the Dictionnaire des sourds-muets at the end of the 18th century as having in common the form of a cross, placed in front of the face. All of these signs have negative connotations. We identify the etymon of the signs as an emblematic gesture of hostility used by hearing people since the 15th century. Inherited from the hearing milieu, the gesture evolved into an important lexical family in use by the deaf in both LSF and its sister language, American Sign Language (ASL). At each step in the gesture's evolution, two conceptual mechanisms explain changes in both form and meaning: economy of articulation and metaphorical abstraction. We show that latent meanings have been invested in the signs' handshapes, placements, and movements, all of which were inherited from gestures of the hearing world.