Speaker: Janet Pierrehumbert (University of Oxford)
Title: LLMs can pass the Turing Test — are they intelligent?
Time: Thursday, May 14, 12:30pm – 2pm
Location: 32-D461
Abstract: In 1950, Turing proposed that if a person can not tell whether they are in conversation with another person or a computer algorithm, we can consider the algorithm to be intelligent. The Turing Test effectively launched the field of AI, and advanced a strong connection between natural language and intelligence. The newest Large Language Models (LLMs) engage in conversations and produce amazingly human-like output. Often, people cannot reliably tell whether they are talking to a chatbot or another person. LLMs seem to pass the Turing Test. Does this mean they are intelligent?
In this talk, I will discuss the nature of the Turing Test and the current level of evidence that LLMs can pass it. I will argue that the Turing Test in its original formulation had a limited conception of intelligence, failing to capture aspects of intelligence that come to the fore in theories of embodied cognition. These aspects of intelligence play a crucial role for humans in acquiring a mental lexicon of meaningful units, and mastering semantic operators (such as markers of temporal, numerical and logical relationships). When probed, LLMs exhibit persistent shortcomings in these areas of language, shortcomings which are systematic consequences of their architecture and training.