What could be Artificial Intelligence’s Chernobyl?
At time of writing watching I am right in between the 10th anniversary of the Fukushima disaster (March 11, 2011) and the 35th anniversary of the Chernobyl disaster (April 26, 1986). These are the only two Level 7 International Nuclear Event Scale events in history. This fact and watching HBO’s fantastic Chernobyl has gotten me asking a few questions: Can we make a scale for what AI incidents look like? What would a level 7 failure equivalent look like in AI? Has an AI ever caused a disaster of large scale monetary or life loss already? What can software engineers in the AI sector learn from Nuclear and Chemical accidents?
An event is considered in terms of Events are considered in terms of:
- Impact on people and the environment
- Impact on radiological barriers and control
- Impact on defense in depth (also known as deep defense or elastic defense), meaning our ability to delay the spread of radioactive materials rather than the ability to stop it outright
Translating this to the field of AI would be something like this:
- Impact on people, the systems running the AI, and the environment
- Impacts on ability to control and contain the AI
- Impact on defense in depth, can we slow the AI down? How fast can we react?
Let’s look at some other disasters and compare them to what an AI would do..
Bhopal Gas Tragedy — How fast we react
The Bhopal gas tragedy is one of the worst industrial disasters in history. Bhopal is the 16th largest city in India, and 131st in the world. On December 2nd 1984 over 500,000 people were exposed to methyl isocyanate (MIC) gas. At least 3,787 deaths have been stated as a result of it, with over 16,000 claimed deaths, and at least 558,125 injuries. All this Mayhem was caused by gross negligence, cost cutting practices, and/or possible employee sabotage. My main interest in Bhopal is an evaluation of elastic defense of society. Bhopal almost 40 years later Bhopal has not been cleaned up. Bhopal people never received proper restitution from the perpetrator Union Carbide, or the parent company Dow Chemical Company.
The Defense in Depth during the Bhopal gas tragedy was abysmal. Many people received no warning other than a generic siren, and they were not trained about the meaning of the siren. We have absolutely no drills, structures, or defenses in place in place for an accident in AI. Much like Bhopal, if you look at society’s ability to react to a rogue AI you find we are not prepared for complete negligence. We do not have a plan B outside of the standard barriers we have in place for computer programs like firewalls, antivirus, and separate Local Networks. What stopped Bhopal from doing more damage was largely the range the gas was able to spread, what stopped Chernobyl was the range wind could carry the radioactive material and the physical barriers of the plant.
If we wait for the accident to put safety in place there is theoretically no range limit that an AI can reach on earth.
Is there any Precedent? — The Flash Crash
May 6, 2010, the NYSE, NASDAQ, and TSE stock markets open and the Dow is down. They trend this way for most of the day on worries due to the European Sovereign debt crisis. Caused by the countries Portugal, Ireland, Greece, and Spain. The PIGS (also Cyprus, but it ruins the acronym). If you’d like to know more about this crisis I recommend the book Boomerang: Travels In The New Third World by Michael Lewis. At 2:42 p.m., with the Dow down more than 300 points for the day, the equity market began to fall rapidly, dropping an additional 600 points in 5 minutes for a loss of nearly 1,000 points for the day by 2:47 p.m. Twenty minutes later, by 3:07 p.m., the market had regained most of the 600-point drop. Before most people even realized there was a problem, it had corrected itself.
There are a lot of theories about the cause for this such as technical glitches, individual manipulation, or massive directional bets on the market. One of the leading theories is it being directly from High Frequency Traders and Artificial Intelligence getting caught in a death spiral. The people involved watching the Market quickly began to panic, crashes to this severity and speed don’t happen. People actively trading thought Nuclear War broke out. This is the speed of an AI accident. On our AI-Nuclear Incident Scale, this is about a 0–1. Some amount of wealth was wiped out, panic was caused, no human life is being recorded as lost. The AI (which is no doubt a highly specific trading AI) did not have the ability to affect any system outside of the market in which it operated. However, from this event we can see just the rate that an unintentionally malicious program is faster than that which any government agency can reasonably react. Just what could an intentionally malicious AI do? What does a 7 on our scale look like?
Between Extinction and Reset
Obviously, I am not the first person to think about the total capability of AI to destroy humanity. I don’t think our current level of infrastructure automation is at the point where a super intelligent AI would have the ability to maintain power to itself. Assuming our power grid is completely connected to the internet, it seems that without intervention from humans to destroy or disrupt our own infrastructure, an AI could forcibly keep itself dominating the internet until it is depowered. Experts estimated the Internet uses 84 to 143 gigawatts of electricity every year, which amounts to between 3.6 and 6.2 percent of all electricity worldwide.(https://raghavan.usc.edu//papers/emergy-hotnets11.pdf) This means that if the AI can defend low intervention sources like solar and wind, it could keep itself up until these assets deteriorate. Considering solar panels last 25–30 years, and hydro plants need maintenance every three years. At current level of defense technology I do not think drones, missiles, and Wi-Fi enabled robots could win the short war to turn off the power over a 1–3 year span.
In the future this gap will surely close as our Robotics improve and our ability to quickly adapt production lines to create arbitrary objects improve. The ability for an AI to create its own defense in depth should buy it enough time to prevent humans and governments reacting in time, as well as keep it’s own power sources from deteriorating. We should build these automated systems and the facilities that power them as if they can be hijacked by an entity ten million times faster and more intelligent than us. If industrial accident history has taught us anything.. it is only a matter of time before someone is negligent, and we must build defense against an AI created by a negligent company before it happens. We do not have the safety net to have a level 7 AI event and adapt safety policy accordingly.
