MIT develops a motion and task planning system for home robots

Why aren’t there more robots in homes? This a surprising complex question — and our homes are surprisingly complex places. A big part of the reason autonomous systems are thriving on warehouse and factory floors first is the relative ease of navigating a structured environment. Sure, most systems still require a space be mapped prior to getting to work, but once that’s in place there tends to be little in the way of variation.

Homes, on the other hand, are kind of a nightmare. Not only do they vary dramatically from unit to unit, they’re full of unfriendly obstacles and tend to be fairly dynamic, as furniture is moved around or things are left on the floor. Vacuums are the most prevalent robots in the home, and they’re still being refined after decades on the market.

This week, researchers at MIT CSAIL are showcasing PIGINet (Plans, Images, Goal, and Initial facts), which is designed to bring task and motion planning to home robotic systems. The neural network is designed to help streamline their ability to create plans of action in different environments.

MIT explains PIGINet thusly:

[I]t employs a transformer encoder, a versatile and state-of-the-art model designed to operate on data sequences. The input sequence, in this case, is information about which task plan it is considering, images of the environment, and symbolic encodings of the initial state and the desired goal. The encoder combines the task plans, image, and text to generate a prediction regarding the feasibility of the selected task plan.

The system is largely focused on kitchen-based activities at present. It draws on simulated home environments to build plans that require interactions with various different elements of the environment, like counters, cabinets, the fridge, sinks, etc. The researchers say that in simpler scenarios, PIGINet was able to reduce planning time by 80%. For more complex situations, that number was generally around 20-50%.

The team suggests that houses are just the start.

“The practical applications of PIGINet are not confined to households,” says PhD student, Zhutian Yang. “Our future aim is to further refine PIGINet to suggest alternate task plans after identifying infeasible actions, which will further speed up the generation of feasible task plans without the need of big datasets for training a general-purpose planner from scratch. We believe that this could revolutionize the way robots are trained during development and then applied to everyone’s homes.”