This reminds me a little bit of a Philip K. Dick story called the Golden Man. A human is genetically engineered without sapience, only instincts. But it’s designed to be sexually appealing and beautiful, and turns out to have mild precognitive abilities. It’s traits make it more survivable than merely intelligent humans, and the genes are dominant. So humans breed with it and sapience becomes extinct. Cuteness wins.
We see this unfold in that Huxley, not Orwell, has proven the more prescient to date. No need to place a boot on our necks when they can merely seduce us with comforts and conveniences.
This is kind of the theme of Spike Jonez's "Her." I'd say that it's the theme of the early portion of Gerard Johnstone's "M3gan," though that veers into more traditional AI concerns. ... ... As Shakespeare wrote in Hamlet, "The devil hath power to assume a pleasing shape." The process you describe reminds me of the imprinting of ducks. In a psychologically critical moment, the first creature the baby duck sees becomes "mother," and they'll follow that creature relentlessly, be it human or dog or fowl. ... ... Your AI concern is more "Brave New World" than "1984."
As I often tell my students, the best minds in the world are working overtime to figure out how to get you to swipe and tap their app. AI will just make it easier.
On a different note, I used to tell my students that robotics is the future and you need to learn to work on them. Now I tell them AI-enhanced robotics is the future and you need to learn how to shut them down.
We are already seeing this with Alexa, Siri, ChatGPT, etc. It’s just going to get worse. Someone will soon use the OpenAI API (ChatGPT is based on OpenAI) to create a personal assistant. And we know that ChatGPT’s training has led it to be balanced in the direction of the OpenAI team’s own biases.
"Essentially, that it will further empower the tech/political class that wants more than anything else to control discussion, debate, and ultimately thought"
Marc Andreessen, on Dec 3-5 of last year:
"Seriously, though. The censorship pressure applied to social media over the last decade pales in comparison to the censorship pressure that will be applied to AI."
"“AI regulation” = “AI ethics” = “AI safety” = “AI censorship”. They’re the same thing."
"The level of censorship pressure that’s coming for AI and the resulting backlash will define the next century of civilization. Search and social media were the opening skirmishes. This is the big one. World War Orwell."
"But despite the perceived difference in intelligence, kids felt neither the Roomba nor the Alexa deserve to be yelled at or harmed." Yeah, nice kids, those. When I introduced my niece to Siri, she took great delight in telling it that it was stupid and a poopy-head.
I see this happening in the near term. Your subscription tier will determine the level of service. Celebrity impersonation will cost more. Amazon already has a virtual Samuel L. Jackson licensed, but you can't make it Alexa's default personality.
Great article, as always! We would argue, though, that a form of this already exists: social media stars are increasingly getting funding from lobbyists to promote certain ideas and discussions (see Wired's "The Lobbyist Next Door") while creating toxic parasocial relationships with their fans. The mental health of those fandoms is abysmal, and it remains to be seen whether AI will have the same kind of effect, despite its "superhuman cuteness."
In the early 1950s, there was much speculation as to what the then-new computers ('thinking machines') and industrial automation systems would mean for humanity, and a lot of SF stories were written on this theme. I discuss some of them here:
Particularly relevant to our current circumstances, I believe, are:
'Dumbwaiter'...maybe the threat is not super-intelligent systems, but super-rigid systems.
'Appointment in Tomorrow'...a supercomputer is viewed as an oracle, but its answers are *really* being simply written by the group of Experts that constructed it.
'Boomerang'...AI-based weapons systems might act in ways that their developers did not expect.
This reminds me a little bit of a Philip K. Dick story called the Golden Man. A human is genetically engineered without sapience, only instincts. But it’s designed to be sexually appealing and beautiful, and turns out to have mild precognitive abilities. It’s traits make it more survivable than merely intelligent humans, and the genes are dominant. So humans breed with it and sapience becomes extinct. Cuteness wins.
We see this unfold in that Huxley, not Orwell, has proven the more prescient to date. No need to place a boot on our necks when they can merely seduce us with comforts and conveniences.
I can't let you do that, Dave...
This is kind of the theme of Spike Jonez's "Her." I'd say that it's the theme of the early portion of Gerard Johnstone's "M3gan," though that veers into more traditional AI concerns. ... ... As Shakespeare wrote in Hamlet, "The devil hath power to assume a pleasing shape." The process you describe reminds me of the imprinting of ducks. In a psychologically critical moment, the first creature the baby duck sees becomes "mother," and they'll follow that creature relentlessly, be it human or dog or fowl. ... ... Your AI concern is more "Brave New World" than "1984."
Great. Just great. Now who’s going to rock me to sleep tonight?
As I often tell my students, the best minds in the world are working overtime to figure out how to get you to swipe and tap their app. AI will just make it easier.
On a different note, I used to tell my students that robotics is the future and you need to learn to work on them. Now I tell them AI-enhanced robotics is the future and you need to learn how to shut them down.
We are already seeing this with Alexa, Siri, ChatGPT, etc. It’s just going to get worse. Someone will soon use the OpenAI API (ChatGPT is based on OpenAI) to create a personal assistant. And we know that ChatGPT’s training has led it to be balanced in the direction of the OpenAI team’s own biases.
"Essentially, that it will further empower the tech/political class that wants more than anything else to control discussion, debate, and ultimately thought"
Marc Andreessen, on Dec 3-5 of last year:
"Seriously, though. The censorship pressure applied to social media over the last decade pales in comparison to the censorship pressure that will be applied to AI."
"“AI regulation” = “AI ethics” = “AI safety” = “AI censorship”. They’re the same thing."
"The level of censorship pressure that’s coming for AI and the resulting backlash will define the next century of civilization. Search and social media were the opening skirmishes. This is the big one. World War Orwell."
"But despite the perceived difference in intelligence, kids felt neither the Roomba nor the Alexa deserve to be yelled at or harmed." Yeah, nice kids, those. When I introduced my niece to Siri, she took great delight in telling it that it was stupid and a poopy-head.
I see this happening in the near term. Your subscription tier will determine the level of service. Celebrity impersonation will cost more. Amazon already has a virtual Samuel L. Jackson licensed, but you can't make it Alexa's default personality.
Plausible.
Hmm. More than plausible--I'd say likely.
You might have even inspired some early action, Professor. /:
Good points. What seems nice can hurt us more than that which is threatening.
Great article, as always! We would argue, though, that a form of this already exists: social media stars are increasingly getting funding from lobbyists to promote certain ideas and discussions (see Wired's "The Lobbyist Next Door") while creating toxic parasocial relationships with their fans. The mental health of those fandoms is abysmal, and it remains to be seen whether AI will have the same kind of effect, despite its "superhuman cuteness."
In the early 1950s, there was much speculation as to what the then-new computers ('thinking machines') and industrial automation systems would mean for humanity, and a lot of SF stories were written on this theme. I discuss some of them here:
https://chicagoboyz.net/archives/68808.html
Particularly relevant to our current circumstances, I believe, are:
'Dumbwaiter'...maybe the threat is not super-intelligent systems, but super-rigid systems.
'Appointment in Tomorrow'...a supercomputer is viewed as an oracle, but its answers are *really* being simply written by the group of Experts that constructed it.
'Boomerang'...AI-based weapons systems might act in ways that their developers did not expect.