On Sunday Ethiopian Airlines Flight 302, a Boeing 737 MAX 8 aircraft, crashed a few minutes after taking off from Addis Ababa, killing all 157 people on board.
It was the second such tragedy in five months, following an October crash of another Boeing 737 MAX aircraft in Indonesia that killed 189 passengers and crew on Lion Air Flight 610.
According to a preliminary report released by the Indonesian air safety investigative agency, the crash was due to a software failure of the Boeing 737 MAX aircraft. Lion Air 610 crashed because a faulty sensor erroneously reported that the plane was stalling. The false report of a stall triggered an automated system that tried to point the aircraft’s nose down so that it could gain enough speed to fly safely.
Software failures impact our lives. The only difference is the severity of the outcome. Whereas when something goes wrong with the software in your cellphone or desktop computer the impact is minimal, other glitches can prove deadly.
These incidents should serve as a wake-up call for software-driven industries. Everything today is being automated as society and industry adopt the Internet of Things and software. What that means is that every object that we interact with will be driven by software and the internet. It also means that we will have less control of everyday objects.Future cars will have minds of their own. As they drive themselves they will make decisions based on data. For some advanced countries, even the police force will be automated. Decisions about who gets arrested will be made not by human beings but by machines.
In some places even your right to travel or get funding will be decided by machines. This will be fine as long as the systems are operating without glitches or bias. When something goes wrong, results will devastating for humans.
Imagine being wrongly accused of a crime because of a glitch in the system. Such a failure is not minor – it’s a major one for those affected.
As we automate life around us, including buildings, cars and factories, we need to bear in mind that software has a tendency to fail. Factoring the imperfection of these systems in design should lead to the creation of safeguards to avoid a disaster when software fails.
It should not be that there’s a reactive approach to software failures, especially when people’s lives are involved. To avoid loss of life, there’s a need for proactive approaches to detect disasters before they happen.
The airline industry is great at investigating what happened to avoid repeating the same mistake. It is probably one of the few industries that does well with ensuring there’s less harm caused by their industry. It has institutions and standards that govern how it operates to limit the harm.
What happens when Internet of Things products present a potential to harm? Is there a body that is watching to warn us about the imminent danger that may be caused by the connected and automated objects?
The automation of life around us should go hand in hand with oversight systems that will detect dangers before they occur to safeguard lives. To avoid future disasters that may be caused by technology, regulations will not be enough to stop disasters.
Active technology tools that are independent of commercial influence will have to be set up to proactively watch other technologies to detect errors before they have a wider impact.