A self-driving car can make a mistake in seconds, but the reason it happened may stretch far back through a long chain of decisions. That is part of what makes autonomous vehicle crashes so hard to explain, and so hard to prevent.
A team at King’s College London says it has developed a new way to tackle that problem. Instead of only estimating how likely a failure is to happen again, the approach is designed to work backward through a crash and identify why a specific failure occurred.
That distinction matters as autonomous vehicles appear more often on public roads, including in cities such as London and San Francisco. Collisions and serious road safety breaches have sharpened pressure on manufacturers to explain what went wrong when these systems fail.
Current methods can offer only limited answers. They tend to rely on failure statistics, which are useful for measuring risk but weaker at explaining one concrete event.
“Traditional methods rely on compiling failure statistics, to tell us how likely another failure is to happen in the future, but they cannot definitively tell you why a self-driving car made the specific error it did. For that, you need to leverage what is known as ‘actual causality’, where an algorithm analyses past mistakes retrospectively,” said Dr Khen Elimelech, leader of the Autonomous Robots Lab at King’s and first author of the paper.
The research centers on a concept known as actual causality. In simple terms, that means examining events after a failure has happened and asking which of them truly caused the outcome.
The idea is especially important for self-driving cars because the causes of failure may be rare, complicated, and potentially catastrophic. A crash may not stem from one obvious mistake. It may emerge from a sequence of observations and decisions that build over time until the system can no longer recover safely.
According to the team, this is the first time actual causality has been applied to the more complicated setting of AI-driven cyber-physical systems. Those are systems in which software continuously interacts with the physical world.
It had previously only been tried in AI systems used to classify images.
The new work builds on earlier research from the same group. In that earlier effort, the team developed an algorithm to efficiently and proactively find rare scenarios that would lead to a crash. That problem is known as falsification.
This latest step goes further. Instead of stopping at identifying dangerous scenarios, the researchers analyze those crash cases to explain them.
That is not a simple task. An autonomous vehicle operating in the real world must constantly process what it sees around it, including cars, people, and other objects, then convert that stream of information into driving decisions.
When something goes wrong, the number of possible causes can become enormous.
The team notes that in some cases, an object the vehicle saw miles before the crash may have started the chain of events that eventually ended in a collision. That makes root-cause analysis both technically difficult and computationally expensive.
To deal with that complexity, the researchers developed what they call a responsibility-guided search algorithm. Its job is to move quickly through the many possible causes and narrow them down to the events that best explain the failure.
According to the team, that algorithm can return an explanation for an event with orders of magnitude less computational effort than a baseline algorithm.
That reduction in computing effort could make the method more practical for use in complex autonomous systems, where brute-force searches may be too slow or too costly to be useful.
For advocates of autonomous vehicles, one of the most stubborn problems has not just been safety itself, but explanation. If a system makes a dangerous choice and nobody can clearly say why, public trust becomes harder to build.
That challenge has also been a barrier to deployment.
“In a world where autonomous vehicles are taking up more space on London’s streets, being able to explain why something happened is vital if we’re going to build trust with this type of technology and integrate cyber-physical systems like this into our lives,” Dr Elimelech said.
Although the researchers focused on autonomous vehicles as a test case, they say the broader method could be used to explain failures across physical systems powered by AI.
That wider potential matters because many of the same concerns now surrounding self-driving cars are beginning to surface in other forms of automation. As AI moves beyond screens and into machines that act in the physical world, the ability to explain failure may become as important as the ability to avoid it.
The team says future work could extend the approach to even more complex settings, including autonomous assistive robots in care homes. The goal is to help design systems across a range of domains that are both reliable and explainable.
The most immediate value of this work is that it offers a way to move beyond broad risk estimates and toward explanations of individual failures. For companies building self-driving vehicles, that could help pinpoint what needs to be fixed after a crash instead of only showing that similar failures might happen again.
It could also shape how regulators, safety investigators, and the public assess autonomous systems. If developers can clearly trace the chain of events behind a collision, that may improve accountability and make safety reviews more useful.
Over time, the same approach could help other AI-powered physical systems, including care robots, earn trust by making their failures easier to understand and correct.
The original story “Why do self-driving cars crash? King’s College London researchers think they have the answer” is published in The Brighter Side of News.
Like these kind of feel good stories? Get The Brighter Side of News’ newsletter.
The post Why do self-driving cars crash? King’s College London researchers think they have the answer appeared first on The Brighter Side of News.
Leave a comment
You must be logged in to post a comment.