Constrained connectionism and the limits of human semantics: a review essay of Terry Regier's The human semantic potential
Taking to heart Massaro's [(1988) Some criticisms of connectionist models of human performance, Journal of Memory and Language, 27, 213-234] criticism that multi-layer perceptrons are not appropriate for modeling human cognition because they are too powerful (i.e. they can simulate just about anything, which gives them little explanatory power), Regier develops the notion of constrained connectionism. The model that he discusses is a distributed network but with numerous constraints added that are (more or less) motivated by real psychophysical and neurophysical constraints. His model learns static prepositions of spatial location such as in, above, to the left of, to the right of, under, etc., as well as dynamic prepositions such as through and the Russian iz-pod, meaning out from under. The network learns these prepositions by viewing a number of examples of them. Very importantly, this book tackles-and goes a long way towards resolving-the problem of the lack of negative exemplars (i.e. we are only very rarely told when something is not above something else), which should lead to overgeneralization, but does not. This book is a significant contribution to connectionist literature.