Robots Learn to Cook Watching YouTube Videos

If you thought that you were the only one who uses YouTube videos to educate yourself, think again.  A team of scientists from the University of Maryland and the Australian research center NICTA successfully taught a robot how to use tools by showing it cooking videos on YouTube.


Human beings are very good at manipulating objects because the human hand is amazingly versatile.  The important task is to find a way to get a robot to imitate what a human can do when chopping a onion or whisking an egg.


Now, not just people can learn from YouTube videos, robot can too and this represents an important step towards the development of futuristic, self-learning helper robots.  Training robots how to manipulate objects has been tricky, but cooking requires a huge range of manipulation actions so that is why the team chose this particular skill for their study.  To teach their robot, the researchers used a method of artificial intelligence training known as ‘deep learning.’  This involves converting information from a variety of inputs, such as audio and image data, into commands.  Key to this technique was a series of artificial neurons that were hooked up to form a network, called a convolutional neural network (CNN), which not only served as a sophisticated image recognitions system, but also allowed the robot to break down the actions presented.


The researchers used a pair of CNNs in their system that performed different roles.  One observed the cook in the YouTube video and identified various actions, such as a particular grasp used on an object, while the other broke down that action in order to work out how the object was being manipulated.  This was also used to predict the next action that was most likely to be performed with the object.


After using data from 88 different YouTube cooking videos, the robot was able to identify which type of grasp was used and the object being grasped.  It then selected the most appropriate manipulator from a small repertoire to replicate the grasp.  “We believe this preliminary integrated system raises hope towards a fully intelligent robot for manipulation tasks that can automatically enrich its own knowledge resource by “watching” recordings from the World Wide Web,” the researchers said.

Now this is impressive, not only could it analyze and break down the videos, but it used the information to learn to handle different tools too.  Eventually the robot determines the cooking scenarios such as cutting, pouring, spreading, chopping, or peeling.  The researchers says that they will be working on refining the robot’s grasping skills even further.   The team’s work will be presented at the 29th annual conference of the Association for the Advancement of Artificial Intelligence.

Leave a Reply

Your email address will not be published. Required fields are marked *