Earlier this weekThe Toyota Research Institute opened the doors of its Bay Area offices to members of the media for the first time. It was a day full of demonstrations, from driving simulators and driving instructors to talks around machine learning and sustainability.
Also on display was robotics, which has long been the focus of Toyota’s research division. SVP Max Bajracharya showcased several projects. The first was what could be expected from Toyota. an industrial arm with a modified handle designed for the surprisingly complex task of moving boxes from the back of a truck to nearby conveyors, something many factories hope to automate. the future
The other is a little more surprising, at least for those who have not followed the work of the department so closely. A shopping robot picks up different products on the shelf based on barcodes and general location. The system can extend to the top shelf to find items before determining the best method to grab a wide range of different objects and drop them into its basket.
The system is the direct result of a 50-person robotics team at a center focused on geriatric care that addresses the issue of Japan’s aging population. It does, however, represent a pivot away from their original work of creating robots designed to perform household tasks such as washing dishes and cooking.
You can read a longer post on that thread in an article published on TechCrunch earlier this week. This comes from a conversation with Bajracharya, which we print in full below. Note that the text has been edited for clarity and length.
Image credits: Brian Hayter
TechCrunch. I was hoping to get a demonstration of the home robot.
Max Bajracharya. We are still doing some home robots[…] What we’ve done has changed. The house was one of our initial challenges.
Elder care was the first pillar.
Absolutely. One of the things we learned in that process is that we didn’t measure our progress very well. Home is so hard. We choose challenging tasks because they are difficult. The problem with the house is not that it was too difficult. It was that it was extremely difficult to measure our progress. We tried many things. We tried to make a procedural mess. We put flour and rice on the tables and tried to wipe them. We put things all over the house to keep the robot tidy. We were staying at Airbnbs to see how well we were doing, but the problem was that we couldn’t get to the same house every time. But if we did, we would be overfitting in that house.
Isn’t it ideal that you don’t get the same house every time?
True, but the problem is that we haven’t been able to measure how well we’re doing. Let’s say we were a little better at getting this one house in order, we don’t know if it was because our chances got better or if that house was a little easier. We used to do the standard, “play a demo, play a cool video. We’re not good enough yet, here’s a great video.” We didn’t know if we were doing well or not. The food challenge task where we said, we need an environment where it’s as challenging as home or has the same representative problems as home, but where we can measure how much progress we’re making.
You’re not talking about specific goals for the home or the supermarket, but about solving problems that can span both of those places.
Or even just measure whether we are advancing the state of the art in robotics. Can we carry out perception, movement planning, behavior, which essentially pursue a common goal? To be completely honest, the challenge issue is kind of irrelevant. The DARPA Robotics Challenges, they were just fictional tasks that were difficult. This is also true of our challenge tasks. We like the house because it represents how we ultimately want to help people at home. But it doesn’t have to be a house. The grocery market is a very good representation because it has that huge variety.
Image credits: Brian Hayter
However, there is frustration. We know how hard these challenges are and how far things are, but some random person sees your video and suddenly it’s on the horizon, even though you can’t represent it.
Absolutely. That’s why Gil [Pratt] every time he says “Re-emphasize why this is a challenging task.”
How do you translate that to normal people? Normal people are not addicted to challenging tasks.
Right, but that’s why in the demo you saw today, we tried to show the challenge tasks, but also an example of how you take the capabilities that come out of that challenge and apply it to a real application like container handling. It is a real problem. We went to the factories and they said yes, this is a problem. Can you help us? And we said, yes, we have the technologies that deal with that. So now we’re trying to show that getting out of these challenges are these few breakthroughs that we think are important, and then apply them to real applications. And I think that helps people understand that because they see that second step.
How big is the robotics team?
The division is about 50 people, split equally between here and Cambridge, Massachusetts.
You have examples like Tesla and Figure trying to create all-purpose humanoid robots. You seem to be going in a different direction.
A little bit. One thing we’ve noticed is that the world is built for people. If you just got a blank slate, you say I want to build a robot that will work in human spaces. You tend to end up with human proportions and human-level capabilities. You end up with human legs and arms, not that it’s necessarily the optimal solution. That’s because the world was created around people.
Image credits: Toyota Research Institute
How do you measure milestones? What does success look like to your team?
Moving from home to the grocery store is a great example of this. We were making progress at home, but not as quickly or as clearly as when we moved to the grocery store. When we move to the grocery store, it becomes very apparent indeed how well you are doing and what the real problems are in your system. And then you can really focus on solving those problems. When we toured both Toyota’s logistics and manufacturing facilities, we saw all of these opportunities where they’re essentially the grocery shopping challenge, except for one small difference. Now, instead of parts being grocery items, parts are all parts in a distribution center.
You hear from 1,000 people that you know home robots are really hard, but then you feel like you have to try it for yourself, and then you’re like, really, you’re making the same mistakes that they did.
I think I’m probably as guilty as anyone. Looks like our GPUs are better now. Oh, we got machine learning and now you know we can do it. Oh well, maybe it was harder than we thought.
Something has to prompt that at some point.
Maybe. I think it will take a long time. Just like driving an automatic, I don’t think there is a silver bullet. There is no such thing as magic, it will be “ok, now we’ve solved it”. It will gradually decay, decay. That’s why it’s important to have that kind of roadmap with shorter timelines, you know, shorter or shorter milestones that give you small wins so you can keep working on it to really achieve that long-term vision.
What is the actual manufacturing process for any of these technologies?
That’s a very good question that we try to answer ourselves. I believe we kind of understand the landscape now. Maybe I was naive in the beginning thinking that, well, we just need to find this person that we’re going to transfer the technology to a third party or someone inside Toyota. But I think we’ve learned that whatever it is, whether it’s a business unit or a company or a startup or a unit at Toyota, they don’t seem to exist. So we’re trying to find a way to be creative, and I think that’s the story of TRI-AD, a little bit. It was created to take the automated driving research we were doing and translate it into something more real. We have the same problem in robotics and many advanced technologies that we work on.
Image credits: Brian Hayter
Are you thinking about potentially getting to a place where you can have spinoffs?
Potential. But that is not the primary mechanism by which we will commercialize the technology.
What is the basic mechanism?
We don’t know. The answer is that the variety of things we do is likely to be different for different groups.
How has TRI changed since its inception?
When I first started, I feel like we were very clearly just doing research in robotics. Part of this is because we were a long way from technology that was applicable to almost any challenging real-world application in the human environment. Over the last five years, I feel like we’ve made enough progress on that very difficult problem that we’re now starting to see it translate into real-world applications. We moved consciously. We’re still 80% driving the state of the art with research, but now we’re devoting 20% of our resources to finding out if that research is maybe as good as we think it is, and if it can actually be applied. – global applications. We can fail. We may realize that we thought we made some interesting progress, but it’s nowhere near reliable or fast enough. But we put 20% of our effort into trying.
How does aged care fit into this?
I would say, in some ways, it’s still our north star. Projects are still exploring how we ultimately empower people in their homes. But over time, as we choose these challenge tasks, if things come up that are applicable to these other areas, that’s where we use these short-term milestones to show the progress of the research we’re doing.
How realistic is the possibility of a completely light-off factor?
I guess if you could start from scratch in the future, it might be. If I look at production today, especially for Toyota, it seems very unlikely that you can come close to that. We [told factory workers], we’re building robotic technology, where do you think it could apply? They showed us lots and lots of processes where it was things like, you take this wire, you feed it through here, and then you pull it here, and then you clamp it here, and you clamp it here, and you you take it here and you take it here and then you drive it like this. And it takes a person five days to learn that skill. “Yes, it’s too difficult for robotic technology.”
But the hardest things for humans are the things you’d like to automate.
Yes, difficult or potentially injury prone. Of course, we’d like to make stepping stones to get there, but where I see robotic technology today, we’re pretty far from that.