Loading...
April 17, 2024 • 3hr 0min
#426 – Edward Gibson: Human Language, Psycholinguistics, Syntax, Grammar & LLMs
Lex Fridman Podcast
Edward Gibson is a professor of psycholinguistics at MIT and head of the MIT Language Lab. His work investigates why human languages have the structures they do, the relationship between language and cognition, and how people process and learn language. Some key areas he has studied include dependency structures in language, center embedding, language universals, and language in remote cultures.
In this wide-ranging conversation, Gibson discusses his views on the nature of language, how it is processed in the brain, what we can learn from studying diverse languages, and the implications for artificial intelligence and machine translation. He also touches on his fieldwork studying language in remote Amazonian tribes and his thoughts on the evolution and future of language.
#426 – Edward Gibson: Human Language, Psycholinguistics, Syntax, Grammar & LLMs
0:00
-0:00
Key Takeaways
- Dependency grammar is a framework for analyzing language that focuses on connections between words. It reveals that shorter dependencies between words are easier to process and produce across all languages.
- Center embedding (nesting clauses within sentences) creates long dependencies and is difficult to process in all languages. This is likely due to memory/processing constraints.
- Language and thought appear to be separate systems in the brain. fMRI studies show language activates specific brain regions, while other cognitive tasks activate different areas.
- Large language models are very good at mimicking the form of language, but likely do not have true understanding of meaning.
- Remote cultures like the Pirahã in the Amazon reveal insights about language, such as not having words for exact numbers beyond "few" and "many".
- Legalese is exceptionally difficult to understand due to extensive use of center embedding. Simplifying legal language could make it much more comprehensible.
- Language death is common, often driven by economic factors as people shift to more widely spoken languages for opportunity.
Introduction
Edward Gibson is a professor of psycholinguistics at MIT and head of the MIT Language Lab. His work investigates why human languages have the structures they do, the relationship between language and cognition, and how people process and learn language. Some key areas he has studied include dependency structures in language, center embedding, language universals, and language in remote cultures.
In this wide-ranging conversation, Gibson discusses his views on the nature of language, how it is processed in the brain, what we can learn from studying diverse languages, and the implications for artificial intelligence and machine translation. He also touches on his fieldwork studying language in remote Amazonian tribes and his thoughts on the evolution and future of language.
Topics Discussed
Dependency Grammar and Language Processing (10:53)
Gibson explains his preference for dependency grammar as a framework for analyzing language structure. Key points:
- Dependency grammar focuses on connections between individual words in a sentence
- It reveals that shorter dependencies between connected words are easier to process and produce
- This pattern holds across all studied languages
- Longer dependencies, especially in center-embedded structures, are more difficult
Gibson states: "What I find beautiful about human language is some of the generalizations that happen across the human languages, within and across a language."
Center Embedding and Processing Difficulty (1:53:14)
A key finding is that center embedding (nesting clauses within sentences) creates processing difficulty:
- Center embedding leads to long-distance dependencies between words
- This is difficult to process and produce in all languages
- Likely due to memory/processing constraints in the brain
- Large language models struggle with center embedding in similar ways to humans
Gibson notes: "That slows people down. That makes them very poor understanding. That makes them. They can't recall what was said as well. Nearly as well."
Language and Thought in the Brain (1:26:46)
Gibson discusses evidence that language and other cognitive processes are separate in the brain:
- fMRI studies show language activates specific left-lateralized brain regions
- Other cognitive tasks like math, music, spatial reasoning activate different areas
- This suggests language is not necessary for thought, contrary to some theories
- People with language impairments can still perform non-linguistic cognitive tasks
He states: "There are this group of people who've had a massive stroke on the left side and wiped out their language network...they can do any task just fine, but not language."
Large Language Models and Understanding (1:40:16)
Gibson shares his views on large language models like GPT-3:
- They are very good at mimicking the form of language
- However, they likely do not have true understanding of meaning
- Their success is based on learning patterns in form, not concepts
- They can be easily tricked in ways humans would not be
He notes: "I would argue they're doing the form, they're doing the form, doing it really, really well. And are they doing the meaning? No, probably not."
Studying Language in Remote Cultures (2:30:10)
Gibson discusses insights from studying language in remote Amazonian tribes:
- The Pirahã language lacks words for exact numbers beyond "few" and "many"
- This reveals number concepts are not innate but develop based on cultural needs
- Color terms also vary across cultures based on usefulness
- Studying diverse languages provides insight into language universals
He explains: "We invent labels for colors, not that I can see, but the things I need to tell you about so that I can get objects from you or get you to give me the right objects."
Legalese and Comprehension Difficulty (1:59:12)
Gibson's research on legal language reveals:
- Legalese is exceptionally difficult to understand due to extensive center embedding
- About 70% of sentences in contracts have center-embedded clauses
- This is much higher than other types of text
- Simplifying by reducing center embedding greatly improves comprehension
He notes: "The center embedding makes big differences in [comprehension]...Lawyers don't like this. They don't want to write like this."
Language Evolution and Death (39:20)
Gibson discusses factors in language evolution and death:
- Languages evolve to optimize communication while maintaining learnability
- Language death is common, often driven by economic factors
- People shift to more widely spoken languages for economic opportunity
- Efforts to preserve languages compete with economic pressures
He explains: "The reason we learn languages is to communicate and we use it to make money and to do whatever it is to feed our families. And if that's not happening, then it won't take off."
Conclusion
This wide-ranging conversation with Edward Gibson provides fascinating insights into the nature of human language, how it is processed in the brain, and what we can learn from studying diverse languages. Key takeaways include the importance of dependency structures in language processing, the separation of language and other cognitive systems in the brain, and how studying remote cultures can reveal fundamental properties of language.
Gibson's work challenges some common assumptions about language, such as its necessity for thought or the innateness of certain concepts. It also has implications for artificial intelligence, suggesting that mimicking language form is not sufficient for true understanding. Overall, the discussion highlights the complexity of human language and the ongoing scientific efforts to unravel its mysteries.