Researchers have achieved a significant milestone in artificial intelligence (AI) with the creation of a neural network capable of human-like language generalization. This remarkable AI system performs on par with humans in the crucial cognitive skill of systematically generalizing newly learned words into various contexts.
The researchers conducted tests comparing this new neural network to the AI model underlying the chatbot ChatGPT, renowned for its human-like conversational abilities. Surprisingly, the new neural network outperformed ChatGPT on the language generalization test, highlighting its potential to enhance human-AI interactions significantly.
Published on October 25 in Nature, this breakthrough could pave the way for AI systems that interact with people in a more natural and intuitive manner than currently available. While large language models like ChatGPT excel in many conversational contexts, they often display notable gaps and inconsistencies in language comprehension.
This neural network’s human-like performance signifies a substantial advancement in training networks to exhibit systematic behavior, as noted by Paul Smolensky, a cognitive scientist specializing in language at Johns Hopkins University.
The concept of systematic generalization is the ability to apply newly acquired words across different scenarios effortlessly. For example, understanding the term ‘photobomb’ allows one to use it in various contexts, such as ‘photobomb twice’ or ‘photobomb during a Zoom call.’ Likewise, grasping the sentence ‘the cat chases the dog’ also equates to understanding ‘the dog chases the cat.’
However, neural networks, commonly used in AI research to mimic human cognition, typically struggle with systematic generalization. They often require extensive training on multiple sample texts to integrate new words, unlike humans who can quickly apply learned words in novel contexts.
To address this limitation, the researchers took a novel approach. They trained a neural network to learn as it completed tasks and allowed it to make mistakes, much like human learning. This methodology led to the neural network emulating human error patterns. When tested on new challenges, the neural network’s performance closely mirrored that of human participants and, in some cases, even exceeded it.
To confirm their findings, the researchers first evaluated 25 people’s ability to deploy newly learned words in various situations. They designed a pseudo-language with basic and abstract words, along with specific rules for combining them. Participants linked primitive words with colored circles, associating a red circle with ‘dax’ and a blue circle with ‘lug.’ When given combinations of primitive and function words, participants had to choose the correct colored circles and order them.
As expected, humans excelled in this task, achieving an 80% success rate on average. In contrast, GPT-4, similar to ChatGPT, faced difficulties, with success rates ranging from 42% to 86%, depending on the presentation. This contrast emphasized that the neural network’s progress was due to practice and learning.
The study’s success as a proof of principle opens the door to further research aiming to scale up this training method to handle more extensive datasets and even other domains such as image recognition. The researchers intend to investigate how humans develop systematic generalization from a young age to create more robust neural networks.
This research has the potential to make neural networks more efficient learners, reducing the amount of data required to train AI systems and minimizing issues like hallucinations. Infusing systematicity into neural networks represents a substantial leap forward, addressing critical challenges in AI development.
By Impact Lab
