In kitchens worldwide, robots dutifully prepare meals with precision, mimicking the assembly line model familiar for the past 50 years. However, Ishika Singh, a Ph.D. student at the University of Southern California, envisions a robot that can do more than follow programmed steps. She aims to create a robot capable of entering a kitchen, navigating the fridge and cabinets, selecting ingredients, and crafting a delectable meal – a task that requires knowledge, common sense, flexibility, and resourcefulness beyond the capabilities of current robotic programming.

Singh points out the limitation of the classical planning pipeline used by roboticists, emphasizing the need for a robot to understand not only the specific kitchen it’s in but also the culture, individuals it’s feeding, and any unexpected variables. Traditional robotic programming, with its predefined actions and preconditions, struggles when confronted with unanticipated situations.

To achieve this ambitious goal, Singh’s research advisor, Jesse Thomason, notes that the robot’s policy formulation must encompass a broad range of factors, from cultural nuances to specific kitchen details and individual preferences. The challenge lies in developing a robot that is not only knowledgeable but also flexible enough to adapt to surprises and changes, such as dropped ingredients or unexpected dietary restrictions.

While videos showcase impressive robots in various fields, none possess the adaptability and coping abilities of humans. Classical robotics, according to Naganand Murty, CEO of Electric Sheep, is “brittle” due to the need to teach robots a static map of the world in a constantly changing environment.

A potential solution emerges with the advent of ChatGPT, a user-friendly interface for large language models (LLMs) like GPT-3. These models, trained with vast amounts of information about dinners, kitchens, and recipes, possess the knowledge to answer practically any question a robot might have about cooking. The synergy between LLMs and robots could bridge the gap, enabling robots to leverage the semantic knowledge of LLMs while providing a tangible presence in the physical world.

However, the excitement about this potential leap forward is met with skepticism. Critics point to occasional errors, biased language, and privacy concerns associated with LLMs. Despite the potential challenges, some technologists see the connection between LLMs and robots as a transformative path forward, marking a departure from the preprogrammed limitations of traditional robotics. As the industry races to teach LLMs how to manipulate tools, the prospect of robots with culinary creativity looms on the horizon.

By Impact Lab