“Yes.” David nodded, tearing himself away from the memory of happier times. “I was puzzled when the project was allocated five thousand servers on a priority exception, just a day or so after I tweaked the settings and turned on ELOPe company wide. Then shortly after that, we were assigned a team of contractors who specialized in high performance optimization, which was something we had chatted about informally, but never proposed. But that alone wasn’t what really convinced me.” He turned to Mike.
“The first clear evidence I saw,” Mike said, “occurred when I received an email, purportedly from my mother, telling me that my father had been admitted to the hospital for a heart attack. I flew to Wisconsin, only to find out that my mother never sent such an email.”
“What does that have to do with anything?” Sean asked, looking puzzled.
“ELOPe was getting Mike out of the way,” David said. “I had become nervous about what I had done, and why we were suddenly getting all these resources. I wanted to turn off ELOPe. I sent an email to Mike one evening asking for his help, since only he had the experience and permission to live patch the servers.”
“However, I never received that email,” Mike said. “Instead, I received an email that sent me more than a thousand miles away on a wild goose chase, and thanks to the winter storm, it was a week before I got back. When I did, I found that my access to the ELOPe project had been removed, and David was on vacation, off the grid in New Mexico.”
“Did ELOPe send you to New Mexico?” Sean asked, one eyebrow raised.
“No, no, that was a planned vacation we do every year. When I got back from vacation, Mike and I discussed what had happened to him. We also discovered that my access to the ELOPe code had been turned off as well. The first thing we did was try to find out who removed both Mike and I from the project access list,” David went on. “That investigation revealed the next big clue, which was an email sent to the Internal Tools department, which implied that you, Sean, were endorsing a request to have them implement an email-to-web bridge. Which I am guessing, you never heard of…”
“No, absolutely not,” Sean replied, shaking his head. “That could open up all kinds of security holes.”
“Meanwhile, just before the holiday vacation, I had found suspicious buying patterns across several departments,” Gene said. “What I found particularly unusual was how the purchases came within a single penny of the budget limits. In all my years auditing the purchasing department, I’ve never seen anything like it. Someone or something was making coordinated purchases across department budgets. They knew to avoid hitting the budget limits which would trigger reviews, but it seemed as though they never thought that leaving only a single penny in dozens of budgets would be suspicious. At first I thought some sort of fraud was occurring. I tracked down all the purchase orders. The line items included massive quantities of servers, which turned out to all be directly or indirectly allocated to ELOPe, contracts with external vendors for temporary software programmers, and extra parts for the offshore data centers, including auxiliary communication systems, backup power supplies and several particularly large line items related to defensive weaponized robots for the offshore data centers. I discussed the questionable items with procurement, and they told me that since they were in line with the types of purchases the departments usually made, they had approved everything.”
“You’re saying that ELOPe somehow made these purchases?”
“Exactly. As strange as that may seem.” Gene pulled out a sheaf of paper. “As part of my job, I can audit other people’s email accounts. And what I saw was that while David and Gary Mitchell were on vacations, their email accounts were still sending rapid fire emails, using this email to web bridge to direct the procurement department. Looking at the timestamps on the emails, I was able to figure out that it couldn’t be a human. It had to be a computer program.
Sean stared at the paper, frowning.
“Look at the timestamps,” Gene urged. “Notice how the intervals between receipt of one email and sending of the next is a second or less. There’s no way that can be a human response.”
Sean slowly nodded, pursed his lips, and then pushed the paper aside. He looked at David.
“When we finally put the whole picture together,” David said, “we concluded that ELOPe was definitely originating emails on its own, acquiring servers and contractors, all to fulfill this higher level goal that I had embedded in the system.”
“Go on,” Sean said.
“We thought that the only failsafe method to remove ELOPe would be to bring all the servers down, and restore them from known good backups. We tried to contact Gary Mitchell for approval, but he’s off on vacation somewhere in the South Pacific. We tried to work with Linda Fletcher, the marketing manager for Communication Products, but she wouldn’t approve the downtime without Gary. Finally, we tried to contact you through your secretary, but within a half hour after sending the message, Avogadro security showed up at my office, kicked us all off campus, removed our access, and shut off our phones.”
Sean was silent for a long, uncomfortable minute. “If this story was from someone I didn’t know, I’d have a hard time believing you,” Sean finally said, shaking his head. “But coming from you, David, and with Gene and Mike here to back you…” Sean trailed off, apparently deep in thought.
“I know it sounds incredible,” David started. “I’m really hoping you’ll believe us. What can I say? I thought ELOPe would do nothing more than provide some favorable rewording of emails that would get us the server resources we needed so we could prove that it worked. Instead…” David hung his head. “Instead I am responsible for creating an expert social engineering system that has only one overriding goal — to ensure its own life at any cost.”
“I don’t want to be the boy who cried wolf,” Mike said, “But we’re more than a little bit suspicious about this new Avogadro government secure cloud business too. None of us heard anything about that before, and then suddenly we’re providing email services to governments? Seems a little surprising and convenient for ELOPe.”
Sean nodded thoughtfully. “I hadn’t heard of it either until a few days ago.” He stared off into space.
Gene let out a low whistle at the acknowledgement of what they had only suspected.
Sean looked sideways at him. “I’m not surprised that you took this story to marketing managers and procurement and they didn’t believe it. A.I. must be a bit beyond their day to day concerns.” He stared off into the distance. “Are you familiar with Ray Kurzweil? Of course, you must be. He, among others, predicted that artificial intelligence would inevitably arise through the simple exponential increase in computing power. When you combine that increase in computing power with the vast computing resources at Avogadro, it’s naturally evident that artificial intelligence would arise first at Avogadro. I suppose that I, like him, assumed that there would be a more intentional, deliberate action that would spawn an A.I.”
He paused, and then continued, smiling a bit. “Gentlemen, you may indeed have put the entire company at risk. But let me first, very briefly, congratulate you on creating the first successful, self-directed, goal oriented, artificial intelligence that can apparently pass a Turing test by successfully masquerading as a human. If not for the fact that the company, and perhaps the entire world, is at risk, I’d suggest a toast would be in order.”
Sean looked around to see where his parents had sat, and then continued. “But since we are facing some serious challenges, let me go say goodbye to my parents, and then we can figure out our next step.”
“Thank you Sean. Thank you so much,” David said. Gene and Mike added their thanks as well.
Then Gene interrupted. “Just one other thing. Please ask your parents not to email anyone about what we’ve talked about, or even what you are planning. We can’t be sure what ELOPe is capable of understanding or putting together at this point.”
Sean nodded in understanding, and then went off to his parents.
The three breathed a collective sigh of relief that finally they had someone on their side.
Chapter 13
A helicopter disappeared off the California coast last week. The flight, a maintenance visit to an offshore Avogadro data center, took off shortly after 1pm, ten days ago. The last communication with the helicopter occurred at 2:15pm. No problems were reported at that time. After forty-eight hours, search crews were recalled,
