There’s a mesmerizing video animation on YouTube of simulated, self-driving traffic streaming through a six-lane, four-way intersection. Dozens of cars flow through the streets, pausing, turning, slowing and speeding up to avoid colliding with their neighbors. And not a single car stopping. But what if even one of those vehicles was not autonomous? What if only one was?
In the coming decades, autonomous vehicles will play a growing role in society, whether keeping drivers safer, making deliveries, or increasing accessibility and mobility for elderly or disabled passengers.
But, Prof. Cathy Wu argues, autonomous vehicles are just part of a complex transport system that may involve individual self-driving cars, delivery fleets, human drivers, and a range of last-mile solutions to get passengers to their doorstep – not to mention road infrastructure like highways, roundabouts and, yes, intersections.
Transport today accounts for about a third of US energy consumption. The decisions we make today about autonomous vehicles could have a big impact on this number — ranging from a 40 percent decrease in energy use to a doubling of energy consumption.
So how can we better understand the problem of integrating autonomous vehicles into the transportation system? Equally important, how can we use this understanding to guide us towards better-functioning systems?
Cathy, who joined LIDS and MIT in 2019, is the Gilbert W. Winslow Assistant Professor of Civil and Environmental Engineering as well as a core faculty member of the MIT Institute for Data, Systems, and Society. Growing up in a Philadelphia-area family of electrical engineers, Cathy sought a field that would enable her to harness engineering skills to solve societal challenges.
During her undergraduate years at MIT, she reached out to Professor Seth Teller of CSAIL to discuss her interest in self-driving cars.
Teller, who passed away in 2014, met her questions with warm advice, said Cathy. “He told me, ‘If you have an idea of what your passion in life is, then you have to go after it as hard as you possibly can. Only then can you hope to find your true passion.’
“Anyone can tell you to go after your dreams, but his insight was that dreams and ambitions are not always clear from the start. It takes hard work to find and pursue your passion.”
Chasing that passion, Cathy would go on to work with Teller, as well as in Professor Daniela Rus’s Distributed Robotics Lab, and finally as a graduate student at University of California, Berkeley, where she won the IEEE Intelligent Transportation Systems Society’s best PhD award in 2019.
In graduate school, Cathy had an epiphany: she realized that for autonomous vehicles to fulfill their promise of fewer accidents, time saved, lower emissions, and greater socioeconomic and physical accessibility, these goals must be explicitly designed for, whether as physical infrastructure, algorithms used by vehicles and sensors, or deliberate policy decisions.
At LIDS, Cathy uses a type of machine learning called reinforcement learning to study how traffic systems behave, and how autonomous vehicles in those systems ought to behave to get the best possible outcomes.
Reinforcement learning, which was most famously used by AlphaGo, DeepMind’s human-beating Go program, is a powerful class of methods which capture the idea behind trial-and-error — given an objective, a learning agent repeatedly attempts to achieve the objective, failing and learning from its mistakes in the process.
In a traffic system, the objectives might be to maximize the overall average velocity of vehicles, to minimize travel time, to minimize energy consumption, and so on.
When studying common components of traffic networks such as grid roads, bottlenecks, and on- and off-ramps, Cathy and her colleagues have found that reinforcement learning can match, and in some cases exceed, the performance of current traffic control strategies. And more importantly, reinforcement learning can shed new light towards understanding complex networked systems -- which have long evaded classical control techniques. For instance, if just five to ten percent of vehicles on the road were autonomous and used reinforcement learning, that could eliminate congestion and boost vehicle speeds by 30 to 140 percent. And the learning from one scenario often translates well to others. These insights could one day soon help to inform public policy or business decisions.
In the course of this research, Cathy and her colleagues helped improve a class of reinforce- ment learning methods called policy gradient methods. Their advancements turned out to be a general improvement to most existing deep reinforcement learning methods.
But reinforcement learning techniques will need to be continually improved to keep up with the scale and shifts in infrastructure and changing behavior patterns. And research findings will need to be translated into action by urban plan- ners, auto makers and other organizations.
Today, Cathy is collaborating with public agencies in Taiwan, and Indonesia to use insights from her work to guide better dialogues and decisions. By changing traffic signals or using nudges to shift drivers’ behavior, are there other ways to achieve lower emissions or smoother traffic?
“I’m surprised by this work every day,” Cathy said. “We set out to answer a question about self-driving cars, and it turns out you can pull apart the insights, apply them in other ways, and then this leads to new exciting questions to answer.”
Cathy is happy to have found her intellectual home at LIDS. Her experience of it is as a “very deep, intellectual, friendly, and welcoming place.” And she counts among her research inspirations MIT course 6.003 Signals and Systems — a class she encourages everyone to take — taught in the tradition of Prof. Alan Oppenheim (RLE) and Prof. Alan Willsky (LIDS). “The course taught me that so much in this world could be fruitfully examined through the lens of signals and systems, be it electronics or institutions or society,” she said. “I am just realizing as I’m saying this, that I’ve been empowered by LIDS thinking all along!”
Research and teaching through a pandemic haven’t been easy, but Cathy is making the best of a challenging first year as faculty. (“I’ve been working from home in Cambridge – my short walking commute is irrelevant at this point,” she said, wryly.) To unwind, she enjoys running, listening to podcasts covering topics ranging from science to history, and reverse-engineering her favorite Trader Joe’s frozen foods.
She’s also been working on two COVID-related projects born at MIT: one explores how data from the environment, such as data collected by IoT connected thermometers, can help identify emerging community outbreaks. Another project asks if it’s possible to ascertain how contagious the virus is on public transport, and how different factors might decrease the transmission risk.
Both are in their early stages, Cathy said. “We hope to contribute a bit to the pool of knowledge that can help decision-makers somewhere. It’s been very enlightening and rewarding to do this and see all the other efforts going on around MIT.”