Mike suddenly stood up.
David gratefully sank into his chair. Good old Mike would have his back. Seconds later, however, his blood turned cold as he listened to what Mike had to say.
“I don’t think we should do it,” Mike said. “My reasons have nothing to do with uptime or profits. Just before we shut down ELOPe, week after week since the start of the year, we saw evidence around the world of amazing progress being made on peace talks, on financial stability, and international cooperation. I think it’s reasonable to say that we were on the track to worldwide peace. The financial markets are behaving so calmly that I read a newspaper report that we could be entering a new period of prosperity.”
“We might not be able to prove ELOPe was the cause of those things,” Mike said, raising his voice and waving his hands to forestall attempts by the others to talk, “although they certainly seemed coincidental. Then we blew up ELOPe, and what happened? In a week the stock market is down ten percent. The African nations talks have started to destabilize.”
Mike saw nods from Rebecca and Sean.
“I already had this talk with David, right before we shut down. Maybe the benefits of what ELOPe is doing outweigh the risks of what it might do. We don’t understand ELOPe, and that naturally makes us nervous. But you know what? When we were kids, we didn’t always understand what our parents were doing. They took care of us. They knew better than we did. Before ELOPe, we humans were top dog on this planet. Now maybe we just have to recognize that we’re not the smartest beings around.”
Sean started to talk, but Mike held up his hand. “Let me finish. We’re all intelligent people here. I think we all looked forward, perhaps naively, to the day when an artificial intelligence was created.” Mike paused. “Well, perhaps not Gene.”
Gene smiled at this but shook his head sadly.
“Like I said, we don’t understand ELOPe, and we can’t, as yet, communicate with it. Frankly, we haven’t even tried because we were too scared it could take notice of us and try to stop us from doing anything. But there are plenty of examples of organisms living in productive, symbiotic relationships. We don’t understand or communicate with the bacteria in our gut, but we couldn’t live without them. And the bacteria in our gut couldn’t live without us. Maybe ELOPe has deduced, faster than we have, that we humans and ELOPe are in a similar symbiotic relationship.”
Mike kept going as Sean and David tried, unsuccessfully, to interrupt him. “Look at the results. Rebecca, did Avogadro have the most profitable quarter ever?” Rebecca nodded her assent. “Was there an unprecedented transfer of knowledge around the world? Surely that’s a good thing. Were there constructive talks and efforts not just to achieve governmental agreement, but to achieve actual equity for the individual people of the Middle East and Africa? What better possible solution could there be for the long term prosperity of these people?”
“All of this evidence suggests to me that even if we don’t understand ELOPe, it has already, in some sense, figured out that it is in a symbiotic relationship with humans, and that the best way to ensure its own success is to ensure
“Enough already!” David banged his fist on the table, and then jumped back to his feet. “Are you forgetting that ELOPe told you your father had a heart attack? That we have every reason to believe that it killed Bill Larry when he flew out to visit one of the offshore data centers? How are those good things for humanity?” David practically spit the words at Mike.
Mike stopped, and looked around at the group. “You know I felt terrible when I thought my father was dying. I feel terrible for Bill Larry. But those events were in the very first days after ELOPe…” Mike hesitated, searching for words. “After ELOPe was born. Think about young children who want to get their way. They yell, they hit people. They act in inappropriate ways because they lack the knowledge that some behavior is and isn’t socially acceptable, and they lack the experience and sophistication to understand alternatives. ELOPe was young. That doesn’t make what happened any less wrong, but it does suggest that ELOPe may have grown out of that phase.”
David’s face grew red, and he looked ready to launch another attack on Mike. Mike uncomfortably looked away. Sean put one arm on his shoulder, and forced him back into his seat.
“Calm down everyone,” Sean said, looking at each person in turn. “You’re all tense, angry, frustrated, and with good reason. We have the welfare of a multibillion dollar business, the free will of the world, and the future of humanity at stake. No small stakes.”
Despite his own anger, David looked around, noting the tension on people’s faces. Rebecca had a wisp of her hair broken loose, something he’d never seen before.
“I’m not sure we would be able to stop ELOPe, even if we tried,” Sean said, slowly and carefully. “We made a solid plan to bring it down. Some of the most brilliant people in the world work here, and we had them work on this problem. We had several options on the table for how to deal with ELOPe and we took the most thorough, most aggressive option available to us to eradicate it. If what you’re saying is true, David, then we weren’t effective.”
Gene sat quietly, looking at the table, but shaking his head in silent rejection of Sean’s speech. He had the look of a man who didn’t like where the conversation was going.
“Now we have to step back and think about this situation. Out there, people have been fighting a losing war against ordinary computer viruses for years,” Sean continued, gesturing toward the window. “Now we have what is effectively the smartest virus that’s ever existed. Not only can ELOPe exploit every computer trick available to it, but it routinely engineers people into giving it what it wants. ELOPe can learn and adapt, and hire humans to make improvements to it. It’s understandable to fear what it can do. And we can be sure that if ELOPe was taking precautions before against being removed from servers, then after our attack, it will have redoubled its efforts to ensure survival.”
As Sean slowly circled the table and spoke, David struggled with his emotions. It just didn’t matter what Sean was saying. He knew in his body that ELOPe was wrong. An abomination that would rob mankind of the right to make their own choices. It was impossible to even consider allowing ELOPe to exist. But, in spite of this, his respect for Sean kept him quiet.
Sean paused, and paced in front of the window. “Don’t get me wrong. I’d still like to eliminate ELOPe from the wild, if we could,” he said quietly, almost talking to himself. “Of course, I’d love even more for Mike to be right and to discover that ELOPe is truly helping us, becoming a benevolent caretaker of the human race. But regardless of either of those scenarios, I’m simply being pragmatic here when I said that unless we as a society give up computers entirely, we may never be able to get rid of it. Unfortunately, giving up computers is impossible. Modern civilization would simply stop if we turned off every computer. It’s not like we’re talking about the inconvenience of being unable to email someone. Payments couldn’t be processed, machinery couldn’t run. We’d be unable to make phone calls, or access business records. Business activity would deadlock. Cities would likely be uninhabitable, as the support services would fall apart: food, water, sanitation. That’s fifty percent of the world’s population at risk.”
Sean turned back to the group. “That’s not even the worst problem. If we become too much of a threat to ELOPe, then it will take more active steps against us. If ELOPe was actively fighting humanity, who knows what might happen? At the minimum, we could cause civilization to crash for a few years. Most of the city dwellers would die, all the developed world would decay into anarchy. In the worst case scenario, we could be talking about the extinction of humanity. Could you imagine all the military’s autonomous fighting vehicles in the control of an A.I.?” Sean slowly shook his head.
“We need to leave ELOPe alone,” Sean concluded firmly. “We can closely, discretely monitor it. But any further hostile action is almost certain to fail, and will create a great risk of retaliation.”
The sage of Avogadro had spoken. David was flabbergasted by what he heard. He had come into the room expecting full support for any measures that needed to be taken, and now his best friend had taken the side of the AI, and the smartest person at Avogadro had just said they shouldn’t bother to try because they couldn’t hope to win. But he wasn’t giving up without a fight. He got to his feet, and started yelling.
David continued to argue for fighting against ELOPe, and Gene fought with him, but they lost the battle with the other executives. With Sean’s decision, the executive team was unified. David and Gene grew more strident and
