Short Story Contest Winner: "The Gerry Program"
It's an odd sensation knowing you're going to die, but it was a feeling Gerry Ferguson had become relatively acquainted with over the past two years. What most perplexed the terminally ill, he observed, was not the concept of death so much as the continuation of all other life.
Gerry's secret project had been in the works for two years now, ever since they found the growth.
Who will mourn me when I'm gone? What trait or idiosyncrasy will people most recall? Will I still be talked of, 100 years from now?
But Gerry didn't worry about these questions. He was comfortable that his legacy would live on, in one form or another. From his cozy flat in the west end of Glasgow, Gerry had managed to put his affairs in order and still find time for small joys.
Feeding the geese in summer at the park just down from his house, reading classics from the teeming bookcase in the living room, talking with his son Michael on Skype. It was Michael who had first suggested reading some of the new works of non-fiction that now littered the large oak desk in Gerry's study.
He was just finishing 'The Master Algorithm' when his shabby grandfather clock chimed six o'clock. Time to call Michael. Crammed into his tiny study, Gerry pulled his computer's webcam close and waved at Michael's smiling face.
"Hi Dad! How're you today?"
"I'm alright, son. How're things in sunny Australia?"
"Hot as always. How's things in Scotland?"
"I'd 'ave more chance gettin' a tan from this computer screen than I do goin' out there."
Michael chuckled. He's got that hearty Ferguson laugh, Gerry thought.
"How's the project coming along?" Michael asked. "Am I going to see it one of these days?"
"Of course," grinned Gerry, "I designed it for you."
Gerry's secret project had been in the works for two years now, ever since they found the growth. He had decided it was better not to tell Michael. He would only worry.
The two men chatted for hours. They discussed Michael's love life (or lack thereof), memories of days walking in the park, and their shared passion, the unending woes of Rangers Football Club. It wasn't until Michael said his goodbyes that Gerry noticed he'd been sitting in the dark for the best part of three hours, his mesh curtains casting a dim orange glow across the room from the street light outside. Time to get back to work.
*
Every night, Gerry sat at his computer, crawling forums, nourishing his project, feeding his knowledge and debating with other programmers. Even at age 82, Gerry knew more than most about algorithms. Never wanting to feel old, and with all the kids so adept at this digital stuff, Gerry figured he should give the Internet a try too. Besides, it kept his brain active and restored some of the sociability he'd lost in the previous decades as old friends passed away and the physical scope of his world contracted.
This night, like every night, Gerry worked away into the wee hours. His back would ache come morning, but this was the only time he truly felt alive these days. From his snug red brick home in Scotland, Gerry could share thoughts and information with strangers from all over the world. It truly was a miracle of modern science!
*
The next day, Gerry woke to the warm amber sun seeping in between a crack in the curtains. Like every morning, his thoughts took a little time to come into focus. Instinctively his hand went to the other side of the bed. Nobody there. Of course; she was gone. Rita, the sweetest woman he'd ever known. Four years this spring, God rest her soul.
Puttering around the cramped kitchen, Gerry heard a knock at the door. Who could that be? He could see two women standing in the hallway, their bodies contorted in the fisheye glass of the peephole. One looked familiar, but Gerry couldn't be sure. He fiddled with the locks and pulled the door open.
"Hi Gerry. How are you today?"
"Fine, thanks," he muttered, still searching his mind for where he'd seen her face before.
Noting the confusion in his eyes, the woman proffered a hand. "Alice, Alice Corgan. I pop round every now and again to check on you."
It clicked. "Ah aye! Come in, come in. Lemme get ya a cuppa." Gerry turned and shuffled into the flat.
As Gerry set about his tiny kitchen, Alice called from the living room, "This is Mandy. She's a care worker too. She's going to pay you occasional visits if that's alright with you."
Gerry poked his head around the doorway. "I'll always welcome a beautiful young lady in ma home. Though, I've tae warn you I'm a married man, so no funny business." He winked and ducked back into the kitchen.
Alice turned to Mandy with a grin. "He's a good man, our Gerry. You'll get along just fine." She lowered her voice. "As I said, with the Alzheimer's, he has to be reminded to take his medication, but he's still mostly self-sufficient. We installed a medi-bot to remind him every day and dispense the pills. If he doesn't respond, we'll get a message to send someone over."
Mandy nodded and scribbled notes in a pad.
"When I'm gone, Michael will have somethin' to remember me by."
"Also, and this is something we've been working on for a few months now, Gerry is convinced he has something…" her voice trailed off. "He thinks he has cancer. Now, while the Alzheimer's may affect his day-to-day life, it's not at a stage where he needs to be taken into care. The last time we went for a checkup, the doctor couldn't find any sign of cancer. I think it stems from--"
Gerry shouted from the other room: "Does the young lady take sugar?"
"No, I'm fine thanks," Mandy called back.
"Of course you don't," smiled Gerry. "Young lady like yersel' is sweet enough."
*
The following week, Mandy arrived early at Gerry's. He looked unsure at first, but he invited her in.
Sitting on the sofa nurturing a cup of tea, Alice tried to keep things light. "So what do you do in your spare time, Gerry?"
"I've got nothing but spare time these days, even if it's running a little low."
"Do you have any hobbies?"
"Yes actually." Gerry smiled. "I'm makin' a computer program."
Alice was taken aback. She knew very little about computers herself. "What's the program for?" she asked.
"Well, despite ma appearance, I'm no spring chicken. I know I don't have much time left. Ma son, he lives down in Australia now, he worked on a computer program that uses AI - that's artificial intelligence - to imitate a person."
Alice still looked confused, so Gerry pressed on.
"Well, I know I've not long left, so I've been usin' this open source code to make ma own for when I'm gone. I've already written all the code. Now I just have to add the things that make it seem like me. I can upload audio, text, even videos of masel'. That way, when I'm gone, Michael will have somethin' to remember me by."
Mandy sat there, stunned. She had no idea anybody could do this, much less an octogenarian from his small, ramshackle flat in Glasgow.
"That's amazing Gerry. I'd love to see the real thing when you're done."
"O' course. I mean, it'll take time. There's so much to add, but I'll be happy to give a demonstration."
Mandy sat there and cradled her mug. Imagine, she thought, being able to preserve yourself, or at least some basic caricature of yourself, forever.
*
As the weeks went on, Gerry slowly added new shades to his coded double. Mandy would leaf through the dusty photo albums on Gerry's bookcase, pointing to photos and asking for the story behind each one. Gerry couldn't always remember but, when he could, the accompanying stories were often hilarious, incredible, and usually a little of both. As he vividly recounted tales of bombing missions over Burma, trips to the beach with a young Michael and, in one particularly interesting story, giving the finger to Margaret Thatcher, Mandy would diligently record them through a Dictaphone to be uploaded to the program.
Gerry loved the company, particularly when he could regale the young woman with tales of his son Michael. One day, as they sat on the sofa flicking through a box of trinkets from his days as a travelling salesman, Mandy asked why he didn't have a smartphone.
He shrugged. "If I'm out 'n about then I want to see the world, not some 2D version of it. Besides, there's nothin' on there for me."
Alice explained that you could get Skype on a smartphone: "You'd be able to talk with Michael and feed the geese at the park at the same time," she offered.
Gerry seemed interested but didn't mention it again.
"Only thing I'm worried about with ma computer," he remarked, "is if there's another power cut and I can't call Michael. There's been a few this year from the snow 'n I hate not bein' able to reach him."
"Well, if you ever want to use the Skype app on my phone to call him you're welcome," said Mandy. "After all, you just need to add him to my contacts."
Gerry was flattered. "That's a relief, knowing I won't miss out on calling Michael if the computer goes bust."
*
Then, in early spring, just as the first green buds burst forth from the bare branches, Gerry asked Mandy to come by. "Bring that Alice girl if ya can - I know she's excited to see this too."
The next day, Mandy and Alice dutifully filed into the cramped study and sat down on rickety wooden chairs brought from the living room for this special occasion.
An image of Gerry, somewhat younger than the man himself, flashed up on the screen.
With a dramatic throat clearing, Gerry opened the program on his computer. An image of Gerry, somewhat younger than the man himself, flashed up on the screen.
The room was silent.
"Hiya Michael!" AI Gerry blurted. The real Gerry looked flustered and clicked around the screen. "I forgot to put the facial recognition on. Michael's just the go-to name when it doesn't recognize a face." His voice lilted with anxious excitement. "This is Alice," Gerry said proudly to the camera, pointing at Alice, "and this is Mandy."
AI Gerry didn't take his eyes from real Gerry, but grinned. "Hello, Alice. Hiya Mandy." The voice was definitely his, even if the flow of speech was slightly disjointed.
"Hi," Alice and Mandy stuttered.
Gerry beamed at both of them. His eyes flitted between the girls and the screen, perhaps nervous that his digital counterpart wasn't as polished as they'd been expecting.
"You can ask him almost anything. He's not as advanced as the ones they're making in the big studios, but I think Michael will like him."
Alice and Mandy gathered closer to the monitor. A mute Gerry grinned back from the screen. Sitting in his wooden chair, the real Gerry turned to his AI twin and began chattering away: "So, what do you think o' the place? Not bad eh?"
"Oh aye, like what you've done wi' it," said AI Gerry.
"Gerry," Alice cut in. "What did you say about Michael there?"
"Ah, I made this for him. After all, it's the kind o' thing his studio was doin'. I had to clear some space to upload it 'n show you guys, so I had to remove Skype for now, but Michael won't mind. Anyway, Mandy's gonna let me Skype him from her phone."
Mandy pulled her phone out and smiled. "Aye, he'll be able to chat with two Gerry's."
Alice grabbed Mandy by the arm: "What did you tell him?" she whispered, her eyes wide.
"I told him he can use my phone if he wants to Skype Michael. Is that okay?"
Alice turned to Gerry, who was chattering away with his computerized clone. "Gerry, we'll just be one second, I need to discuss something with Mandy."
"Righto," he nodded.
Outside the room, Alice paced up and down the narrow hallway.
Mandy could see how flustered she was. "What's wrong? Don't you like the chatbot? I think it's kinda c-"
"Michael's dead," Alice spluttered.
"What do you mean? He talks to him all the time."
Alice sighed. "He doesn't talk to Michael. See, a few years back, Michael found out he had cancer. He worked for this company that did AI chatbot stuff. When he knew he was dying he--" she groped in the air for the words-- "he built this chatbot thing for Gerry, some kind of super-advanced AI. Gerry had just been diagnosed with Alzheimer's and I guess Michael was worried Gerry would forget him. He designed the chatbot to say he was in Australia to explain why he couldn't visit."
"That's awful," Mandy granted, "but I don't get what the problem is. I mean, surely he can show the AI Michael his own chatbot?"
"No, because you can't get the AI Michael on Skype. Michael just designed the program to look like Skype."
"But then--" Mandy went silent.
"Michael uploaded the entire AI to Gerry's computer before his death. Gerry didn't delete Skype. He deleted the AI Michael."
"So… that's it? He-he's gone?" Mandy's voice cracked. "He can't just be gone, surely he can't?"
The women stood staring at each other. They looked to the door of the study. They could still hear Gerry, gabbing away with his cybercopy.
"I can't go back in there," muttered Mandy. Her voice wavered as she tried to stem the misery rising in her throat.
Alice shook her head and paced the floor. She stopped and stared at Mandy with grim resignation. "We don't have a choice."
When they returned, Gerry was still happily chatting away.
"Hiya girls. Ya wanna ask my handsome twin any other questions? If not, we could get Michael on the phone?"
Neither woman spoke. Gerry clapped his hands and turned gaily to the monitor again: "I cannae wait for ya t'meet him, Gerry. He's gonna be impressed wi' you."
Alice clasped her hands to her mouth. Tears welled in the women's eyes as they watched the old man converse with his digital copy. The heat of the room seemed to swell, becoming insufferable. Mandy couldn't take it anymore. She jumped up, bolted to the door and collapsed against a wall in the hallway. Alice perched on the edge of her seat in a dumb daze, praying for the floor to open and swallow the contents of the room whole.
Oblivious, Gerry and his echo babbled away, the blue glow of the screen illuminating his euphoric face. "Just wait until y'meet him Gerry, just wait."
Facial Recognition Can Reduce Racial Profiling and False Arrests
[Editor's Note: This essay is in response to our current Big Question, which we posed to experts with different perspectives: "Do you think the use of facial recognition technology by the police or government should be banned? If so, why? If not, what limits, if any, should be placed on its use?"]
Opposing facial recognition technology has become an article of faith for civil libertarians. Many who supported the bans in cities like San Francisco and Oakland have declared the technology to be inherently racist and abusive.
The greatest danger would be to categorically oppose this technology and pretend that it will simply go away.
I have spent my career as a criminal defense attorney and a civil libertarian -- and I do not fear it. Indeed, I see it as positive so long as it is appropriately regulated and controlled.
We are living in the beginning of a biometric age, where technology uses our physical or biological characteristics for a variety of products and services. It holds great promises as well as great risks. The greatest danger, however, would be to categorically oppose this technology and pretend that it will simply go away.
This is an age driven as much by consumer as it is government demand. Living in denial may be emotionally appealing, but it will only hasten the creation of post-privacy world. If we do not address this emerging technology, movements in public will increasingly result in instant recognition and even tracking. It is the type of fish-bowl society that strips away any expectation of privacy in our interactions and associations.
The biometrics field is expanding exponentially, largely due to the popularity of consumer products using facial recognition technology (FRT) -- from the iPhone program to shopping ones that recognize customers.
But the privacy community is losing this battle because it is using the privacy rationales and doctrines forged in the earlier electronic surveillance periods. Just as generals are often accused of planning to fight the last war, civil libertarians can sometimes cling to past models despite their decreasing relevance in the current world.
I see FRT as having positive implications that are worth pursuing. When properly used, biometrics can actually enhance privacy interests and even reduce racial profiling by reducing false arrests and the warrantless "patdowns" allowed by the Supreme Court. Bans not only deny police a technology widely used by businesses, but return police to the highly flawed default of "eye balling" suspects -- a system with a considerably higher error rate than top FRT programs.
Officers are often wrong and stop a great number of suspects in the hopes of finding a wanted felon.
A study in Australia showed that passport officers who had taken photographs of subjects in ideal conditions nonetheless experienced high error rates when identifying them shortly afterward, including 14 percent false acceptance rates. Currently, officers stop suspects based on their memory from seeing a photograph days or weeks earlier. They are often wrong and stop a great number of suspects in the hopes of finding a wanted felon. The best FRT programs achieve an astonishing accuracy rate, though real-world implementation has challenges that must be addressed.
One legitimate concern raised in early studies showed higher error rates in recognitions for certain groups, particularly African American women. An MIT study finding that error rate prompted major improvements in the algorithms as well as training changes to greatly reduce the frequency of errors. The issue remains a concern, but there is nothing inherently racist in algorithms. These are a set of computer instructions that isolate and process with the parameters and conditions set by creators.
To be sure, there is room for improvement in some algorithms. Tests performed by the American Civil Liberties Union (ACLU) reportedly showed only an 80 percent accuracy rate in comparing mug shots to pictures of members of Congress when using Amazon's "Rekognition" system. It recently showed the same 80 percent rate in doing the same comparison to members of the California legislators.
However, different algorithms are available with differing levels of performance. Moreover, these products can be set with a lower discrimination level. The fact is that the top algorithms tested by the National Institute of Standards and Technology showed that their accuracy rate is greater than 99 percent.
The greatest threat of biometric technologies is to democratic values.
Assuming a top-performing algorithm is used, the result could be highly beneficial for civil liberties as opposed to the alternative of "eye balling" suspects. Consider the Boston Bombing where police declared a "containment zone" and forced families into the street with their hands in the air.
The suspect, Dzhokhar Tsarnaev, moved around Boston and was ultimately found outside the "containment zone" once authorities abandoned near martial law. He was caught on some surveillance systems but not identified. FRT can help law enforcement avoid time-consuming area searches and the questionable practice of forcing people out of their homes to physically examine them.
If we are to avoid a post-privacy world, we will have to redefine what we are trying to protect and reconceive how we hope to protect it. In my view, the greatest threat of biometric technologies is to democratic values. Authoritarian nations like China have made huge investments into FRT precisely because they know that the threat of recognition in public deters citizens from associating or interacting with protesters or dissidents. Recognition changes conduct. That chilling effect is what we have the worry about the most.
Conventional privacy doctrines do not offer much protection. The very concept of "public privacy" is treated as something of an oxymoron by courts. Public acts and associations are treated as lacking any reasonable expectation of privacy. In the same vein, the right to anonymity is not a strong avenue for protection. We are not living in an anonymous world anymore.
Consumers want products like FaceFind, which link their images with others across social media. They like "frictionless" transactions and authentications using faceprints. Despite the hyperbole in places like San Francisco, civil libertarians will not succeed in getting that cat to walk backwards.
The basis for biometric privacy protection should not be focused on anonymity, but rather obscurity. You will be increasingly subject to transparency-forcing technology, but we can legislatively mandate ways of obscuring that information. That is the objective of the Biometric Privacy Act that I have proposed in recent research. However, no such comprehensive legislation has passed through Congress.
The ability to spot fraudulent entries at airports or recognizing a felon in flight has obvious benefits for all citizens.
We also need to recognize that FRT has many beneficial uses. Biometric guns can reduce accidents and criminals' conduct. New authentications using FRT and other biometric programs could reduce identity theft.
And, yes, FRT could help protect against unnecessary police stops or false arrests. Finally, and not insignificantly, this technology could stop serious crimes, from terrorist attacks to the capturing of dangerous felons. The ability to spot fraudulent entries at airports or recognizing a felon in flight has obvious benefits for all citizens.
We can live and thrive in a biometric era. However, we will need to bring together civil libertarians with business and government experts if we are going to control this technology rather than have it control us.
[Editor's Note: Read the opposite perspective here.]
The Case for an Outright Ban on Facial Recognition Technology
[Editor's Note: This essay is in response to our current Big Question, which we posed to experts with different perspectives: "Do you think the use of facial recognition technology by the police or government should be banned? If so, why? If not, what limits, if any, should be placed on its use?"]
In a surprise appearance at the tail end of Amazon's much-hyped annual product event last month, CEO Jeff Bezos casually told reporters that his company is writing its own facial recognition legislation.
The use of computer algorithms to analyze massive databases of footage and photographs could render human privacy extinct.
It seems that when you're the wealthiest human alive, there's nothing strange about your company––the largest in the world profiting from the spread of face surveillance technology––writing the rules that govern it.
But if lawmakers and advocates fall into Silicon Valley's trap of "regulating" facial recognition and other forms of invasive biometric surveillance, that's exactly what will happen.
Industry-friendly regulations won't fix the dangers inherent in widespread use of face scanning software, whether it's deployed by governments or for commercial purposes. The use of this technology in public places and for surveillance purposes should be banned outright, and its use by private companies and individuals should be severely restricted. As artificial intelligence expert Luke Stark wrote, it's dangerous enough that it should be outlawed for "almost all practical purposes."
Like biological or nuclear weapons, facial recognition poses such a profound threat to the future of humanity and our basic rights that any potential benefits are far outweighed by the inevitable harms.
We live in cities and towns with an exponentially growing number of always-on cameras, installed in everything from cars to children's toys to Amazon's police-friendly doorbells. The use of computer algorithms to analyze massive databases of footage and photographs could render human privacy extinct. It's a world where nearly everything we do, everywhere we go, everyone we associate with, and everything we buy — or look at and even think of buying — is recorded and can be tracked and analyzed at a mass scale for unimaginably awful purposes.
Biometric tracking enables the automated and pervasive monitoring of an entire population. There's ample evidence that this type of dragnet mass data collection and analysis is not useful for public safety, but it's perfect for oppression and social control.
Law enforcement defenders of facial recognition often state that the technology simply lets them do what they would be doing anyway: compare footage or photos against mug shots, drivers licenses, or other databases, but faster. And they're not wrong. But the speed and automation enabled by artificial intelligence-powered surveillance fundamentally changes the impact of that surveillance on our society. Being able to do something exponentially faster, and using significantly less human and financial resources, alters the nature of that thing. The Fourth Amendment becomes meaningless in a world where private companies record everything we do and provide governments with easy tools to request and analyze footage from a growing, privately owned, panopticon.
Tech giants like Microsoft and Amazon insist that facial recognition will be a lucrative boon for humanity, as long as there are proper safeguards in place. This disingenuous call for regulation is straight out of the same lobbying playbook that telecom companies have used to attack net neutrality and Silicon Valley has used to scuttle meaningful data privacy legislation. Companies are calling for regulation because they want their corporate lawyers and lobbyists to help write the rules of the road, to ensure those rules are friendly to their business models. They're trying to skip the debate about what role, if any, technology this uniquely dangerous should play in a free and open society. They want to rush ahead to the discussion about how we roll it out.
We need spaces that are free from government and societal intrusion in order to advance as a civilization.
Facial recognition is spreading very quickly. But backlash is growing too. Several cities have already banned government entities, including police and schools, from using biometric surveillance. Others have local ordinances in the works, and there's state legislation brewing in Michigan, Massachusetts, Utah, and California. Meanwhile, there is growing bipartisan agreement in U.S. Congress to rein in government use of facial recognition. We've also seen significant backlash to facial recognition growing in the U.K., within the European Parliament, and in Sweden, which recently banned its use in schools following a fine under the General Data Protection Regulation (GDPR).
At least two frontrunners in the 2020 presidential campaign have backed a ban on law enforcement use of facial recognition. Many of the largest music festivals in the world responded to Fight for the Future's campaign and committed to not use facial recognition technology on music fans.
There has been widespread reporting on the fact that existing facial recognition algorithms exhibit systemic racial and gender bias, and are more likely to misidentify people with darker skin, or who are not perceived by a computer to be a white man. Critics are right to highlight this algorithmic bias. Facial recognition is being used by law enforcement in cities like Detroit right now, and the racial bias baked into that software is doing harm. It's exacerbating existing forms of racial profiling and discrimination in everything from public housing to the criminal justice system.
But the companies that make facial recognition assure us this bias is a bug, not a feature, and that they can fix it. And they might be right. Face scanning algorithms for many purposes will improve over time. But facial recognition becoming more accurate doesn't make it less of a threat to human rights. This technology is dangerous when it's broken, but at a mass scale, it's even more dangerous when it works. And it will still disproportionately harm our society's most vulnerable members.
Persistent monitoring and policing of our behavior breeds conformity, benefits tyrants, and enriches elites.
We need spaces that are free from government and societal intrusion in order to advance as a civilization. If technology makes it so that laws can be enforced 100 percent of the time, there is no room to test whether those laws are just. If the U.S. government had ubiquitous facial recognition surveillance 50 years ago when homosexuality was still criminalized, would the LGBTQ rights movement ever have formed? In a world where private spaces don't exist, would people have felt safe enough to leave the closet and gather, build community, and form a movement? Freedom from surveillance is necessary for deviation from social norms as well as to dissent from authority, without which societal progress halts.
Persistent monitoring and policing of our behavior breeds conformity, benefits tyrants, and enriches elites. Drawing a line in the sand around tech-enhanced surveillance is the fundamental fight of this generation. Lining up to get our faces scanned to participate in society doesn't just threaten our privacy, it threatens our humanity, and our ability to be ourselves.
[Editor's Note: Read the opposite perspective here.]