David had sent Mike a copy of his itinerary by email weeks earlier, so he had David’s flight information. Without even realizing what he was doing, he found himself standing in his living room, holding a printed copy of the itinerary, watching the clock. He had his jacket on, alternating between sitting on the couch tapping his toe and getting up to pace the living room. Finally, with more than an hour left before the flight would arrive, he drove to the airport.
Mike drove with furrowed brows through a light drizzle, with the streets threatening to ice over. His thoughts were bordering on obsessive. What had David done to ELOPe? What was ELOPe doing? Why was he locked out of the servers? He swore as his front tires spun crossing the light rail tracks, and the car fishtailed. He fought the wheel and recovered halfway through the turn onto Airport Way.
He sped into the parking garage, circling up the ramp, and parked near the skybridge. He walked across the covered bridge at a high speed, and stalked through the airport. He got to the security gate, and looked at his phone. Still nearly an hour until David’s flight would arrive.
He checked the flight monitors once, then forced himself to sit down and stay there, trying to squelch his nervous energy. He watched whole families disgorge through the security exit; suitcases, car-seats, and exhausted children in tow. He smiled as he watched a young woman welcome a man home with a single flower and a long embrace. He missed the bittersweet tang of young love.
Then he saw David and Christine coming through the gate. He walked over and waved. David and Christine were delighted to see him. Christine gave him a big hug, and a bigger smile. Standing in the middle of the stream of exiting passengers, Mike launched into an immediate explanation in a hushed voice. David and Christine leaned closer to listen, and their smiles vanished.
“My dad was absolutely fine. My mother never sent any emails at all. I’m telling you, I’m convinced that ELOPe originated those emails. What I don’t understand is why.”
“Did you get my email before you left?” David asked. “The one about the override I put in?”
Mike gave him a blank look. “No, but it sounds like you should start explaining.”
Before David could say anything, a TSA agent asked them to move out of the way of people.
“Let’s get to your car,” David said. “Then I’ll tell you.”
They rushed to the parking garage, made their way to Mike’s Jetta, and threw the bags in the trunk. Mike gripped the wheel tightly, while David sat next to him, and Christine sat in the middle of the rear seat, leaning forward.
“I put in an override,” David said once they got out of the garage. “I didn’t just use ELOPe to send an email to Gary. I have ELOPe filtering every company email.” David slammed the dashboard with his fist, startling Mike, who swerved sideways, skidding on the icy street, before regaining his lane.
“I sent you an email, the night you came over for dinner. I told you all this, and that I needed your help to do a live-patch on the server.”
“I never got it,” Mike said. ”I’ve been through all my emails from you.” He sighed. “This explains so much. Didn’t it occur to you that ELOPe would fabricate that email to get rid of me?”
“No, it didn’t.” David shook his head. He raised his voice defensively. “Look, I didn’t think that you would fly off to Wisconsin based only on an email.”
“Hey, I had just heard that my father had a heart attack. I was freaking out. And it’s your damn fault I had to go through that.” Mike was yelling now, and David leaned away in his seat.
“OK, chill out guys. If the two of you are fighting, you are not going to be thinking straight.” Christine said, also yelling to get their attention. “Look, what happened when you got back Mike? Did you look into ELOPe?” She spoke calmly, placatingly.
“I tried, but I couldn’t get access to either the source code or the system logs,” Mike said, resigned. “I assumed David had locked down access so that no one would find what he had done. And I didn’t want to say anything to any of the rest of the team, because I didn’t want to raise suspicions. I was still trying to cover for you, David.” Mike glared at him.
“I’m sorry, Mike,” David said gently. “I shouldn’t have said that about you and your dad. I’m really sorry you flew halfway across the country, but I’m glad your father was fine.”
Mike hesitated a minute, then nodded slightly, accepting the apology. “Well, did you do anything to lock down the system?” he asked.
“No. In retrospect, it sounds like a great idea to have locked down access to ELOPe, but I didn’t do it.”
“Shit, then somehow ELOPe has removed my access to the servers and the code.”
“Ugh, guys, that seems impossible.” Christine said. “Even if you are right, Mike, and ELOPe is somehow originating emails on its own, it seems preposterous to think that ELOPe could social engineer you to leave town. And how is ELOPe going to get your access rights removed? If all it can do is send emails, you can’t send an email to revoke someone’s access. I assume you guys have some kind of internal web application that handles access control. I think you’ve somehow become paranoid about David’s deceit being discovered, and now your imagination is running away with you.”
“No, Christine,” Mike said. “I’ve thought about this for days now, and it is possible. Let’s say ELOPe didn’t want to be turned off. It knows that I can turn it off. Now it has to figure out how to ensure I won’t do it. If it analyzes enough emails, it could figure out that people don’t do work when they leave town. If it can figure that out, then it can also determine that people leave town for family medical emergencies. If it had enough emails about medical emergencies, it could figure out that messages about family emergencies usually come from family members. My own email history would show who my parents are, and their email addresses, and that I’ve flown to visit them before. If it put all those things together in one long chain of deductions, it could figure out to fabricate an email from my mother saying that my father is sick. I know it sounds farfetched, but this is all within the design parameters of ELOPe.”
“Are you saying that this thing is reasoning and thinking like a human being?” Christine asked, shaking her head. “Because no matter how smart you guys are, I’m having a hard time believing that some code you wrote is suddenly developing a mind of its own.”
“It’s not thinking,” David said. “ELOPe is just analyzing emails, figuring out what language will optimize the success of the primary goal I entered, which was to maximize success of the ELOPe project. It’s a straightforward process; goal, analysis, language optimization, in response to inputs. It can chain goals together. It is not independent thought, but it can have the appearance of independent thought.”
Mike raised his hand up. “Look, here’s an analogy I thought of while I was waiting for your plane. Imagine that you’ve got all the pieces of all the jigsaw puzzles in the world. Now imagine you have a computer that is patient enough to try every possible combination of every possible puzzle. Given enough time, it could make any arbitrary picture it wanted out of those pieces. And that’s what emails are to ELOPe—puzzle pieces. It looks at the millions of emails in its library of emails, figures out all the components of them, and then figures out new ways to piece them together.”
“ELOPe, the computer system that ran away with itself.” David laughed nervously. “Well, we got the name right.”
“So is it an artificial intelligence? Is it thinking for itself, or isn’t it?” Christine spoke softly, half to herself.
“I don’t know, hon,” David said. “I don’t see how it could be capable of free-form thinking, which is what most people would think of as an AI. But it is pretty sophisticated when it comes to goal analysis and synthesis. We couldn’t hardwire goals into ELOPe and have it meet the design objectives. We had to let it discover people’s goals. So we gave it the ability to contextually determine goals by parsing emails.”
“Then it tries to make sense of those goals in terms of other goals it understands,” Mike added. “We implemented two approaches to learning about new goals. First, it can see whether goals might be similar based on language analysis. For example, a ‘break from work’ is semantically similar to a ‘vacation’, and a simple dictionary lookup can figure that out. Second, it can guess where one goal might be an extension of another. If it thinks one goal might be an extension of another, it will predict what people’s responses will be, and then test to see if the predictions match actual historical responses. For example, if I simply said I wanted to have fun today, then ELOPe might be able to extrapolate that activities such as playing a game, miniature golf, or going to see a band are fun.”
David nodded. “So when I added code to create an overriding goal to maximize the success of the ELOPe program, it’s hard to know what it might consider. The more emails it analyzes, the broader the definition of
