An AI Expert Warning: 6 People Are (Quietly) Deciding Humanity’s Future!

Full Transcript

https://www.youtube.com/watch?v=P7Y-fynYsgE

[00:00] In October, over 850 experts, including yourself and other leaders like Richard Branson and Geoffrey Hinton, signed a statement to ban AI superintelligence as you guys raised concerns of potential human extinction.
[00:11] Because unless we figure out how do we guarantee that the AI systems are safe, we're toast.
[00:19] And you've been so influential on the subject of AI.
[00:20] You wrote the textbook that many of these CEOs who are building some of the AI companies now would have studied on the subject of AI.
[00:25] Yep.
[00:27] So, do you have any regrets?
[00:31] Um Professor Stuart Russell has been named one of Time magazine's most influential voices in AI.
[00:36] After spending over 50 years researching, teaching, and finding ways to design AI in such a way that humans maintain control.
[00:44] You talk about this gorilla problem as a way to understand AI in the context of humans.
[00:48] Yeah, so a few million years ago, the human line branched off from the gorilla line in evolution.
[00:53] And now the gorillas have no say in whether they continue to exist because we are much smarter than they are.
[00:57] So, intelligence is actually the single most important factor to
[01:00] the single most important factor to control planet Earth.
[01:02] But we're in the process of making something more intelligent than us.
[01:04] Exactly. Why don't people stop then?
[01:06] Well, one of the reasons is something called the Midas touch.
[01:08] So, King Midas is this legendary king who asked gods, "Can everything I touch turn to gold?"
[01:13] And we think of the Midas touch as being a good thing, but he goes to drink some water, the water has turned to gold.
[01:17] And he goes to comfort his daughter, his daughter turns to gold.
[01:20] And so he dies in misery and starvation.
[01:22] So, this applies to our current situation in two ways.
[01:24] One is that greed is driving these companies to pursue technology with the probabilities of extinction being worse than playing Russian roulette.
[01:32] And that's even according to the people developing the technology without our permission.
[01:35] And people are just fooling themselves if they think it's naturally going to be controllable.
[01:43] So, you know, after 50 years I could retire, but instead I'm working 80 or 100 hours a week trying to move things in the right direction.
[01:49] So, if you had a button in front of you which would stop all progress in artificial intelligence, would you press it?
[01:57] Not yet.
[02:00] Not yet.
[02:00] I think there's still a decent chance to guarantee safety.
[02:02] I can explain more of what that is.
[02:07] I see messages all the time in the comment section that some of you didn't realize you didn't subscribe.
[02:10] So, if you could do me a favor and double-check if you're a subscriber to this channel, that would be tremendously appreciated.
[02:16] It's the simple, it's the free thing that anybody that watches this show frequently can do to help us here to keep everything going in this show in the trajectory it's on.
[02:23] So, please do double-check if you subscribed and thank you so much because in a strange way you are you're part of our history and you're on this journey with us and I appreciate you for that.
[02:32] So, thank you.
[02:35] [Music]
[02:41] Professor Stuart Russell, OBE.
[02:44] A lot of people have been talking about AI for the last couple of years.
[02:48] It appears you've This really shocked me.
[02:50] It appears you've been talking about AI for most of your life.
[02:53] Well, I started doing AI in high school um back in England.
[02:56] But then I did my PhD starting in '82 at Stanford.
[03:02] PhD starting in '82 at Stanford.
[03:02] I joined the faculty of Berkeley in '86.
[03:06] joined the faculty of Berkeley in '86.
[03:06] So, I'm in my 40th year as a professor at Berkeley.
[03:09] The main thing that the AI community is familiar with in my work uh is a textbook that I wrote.
[03:17] Is this the textbook that most students who study AI are likely learning from?
[03:23] Yep.
[03:23] So, you wrote the textbook on artificial intelligence 30 one years ago.
[03:30] You actually start probably started writing it because it's so bloody big in the year that I was born.
[03:35] So, I was born in '92.
[03:35] Uh yeah, took me about 2 years.
[03:39] Me and your book are the same age, which is just a wonderful way for me to understand just how long you've been talking about this and how long you've been writing about this.
[03:50] And actually it's interesting that many of the CEOs who are building some of the AI companies now probably learned from your textbook.
[03:57] You had a conversation with somebody who said that in order for people to get the message that we're going to be talking
[04:04] message that we're going to be talking about today, there would have to be a about today, there would have to be a catastrophe for people to wake up.
[04:09] catastrophe for people to wake up.
[04:09] Can you give me context on that conversation and a gist of who you had this conversation with?
[04:15] Uh so, it was with one of the CEOs of uh leading AI company.
[04:20] company.
[04:20] He sees two possibilities as do I, which is um either we have a small or let's say small-scale disaster same scale as Chernobyl.
[04:33] The nuclear meltdown in Ukraine.
[04:34] Yeah, so this uh nuclear plant blew up in 1986.
[04:39] Killed uh a fair number of people directly and maybe tens of thousands of people indirectly through radiation.
[04:49] Recent cost estimates more than a trillion dollars.
[04:53] So, that would wake people up.
[04:58] That would get the governments to regulate.
[05:00] He's talked to the governments and they won't do it.
[05:03] So, he looked at this Chernobyl-scale
[05:06] he looked at this Chernobyl-scale disaster as the best-case scenario because then the governments would regulate and require AI systems to be built.
[05:15] AI systems to be built.
[05:15] And is this CEO building an AI company?
[05:21] He runs one of the leading AI companies.
[05:22] And even he thinks that the only way that people will wake up is if there's a Chernobyl-level nuclear disaster.
[05:28] Uh yeah, not wouldn't have to be a nuclear disaster.
[05:30] It would be either an AI system that's being misused by someone, for example, to engineer a pandemic or an AI system that does something itself such as crashing our financial system or our communication systems.
[05:46] The alternative is a much worse disaster where we just lose control altogether.
[05:50] You have had lots of conversations with lots of people in the world of AI, both people that are you know, have built the technology, have studied and researched the technology, all the CEOs and founders that are currently in the AI race.
[06:04] What are some of the the interesting sentiments that the
[06:06] the interesting sentiments that the general public wouldn't believe
[06:09] general public wouldn't believe that you hear privately about their
[06:12] that you hear privately about their perspectives?
[06:14] perspectives? Cuz I find that's so fascinating.
[06:15] I've had some private conversations with people
[06:18] very close to these tech companies.
[06:20] And the shocking sentiment that I was exposed to was that
[06:24] they are aware of the risks often, but
[06:26] they don't feel like there's anything that can be done.
[06:27] So, they're carrying on, which is feels like a bit of a paradox to me.
[06:31] Like it's >> Yes, it's it's it must be a very difficult position to be in in a sense, right?
[06:38] You're you're doing something that you know has a good chance of bringing an end
[06:43] to life on Earth including that of yourself and your own family.
[06:48] They feel that they can't escape this race.
[06:54] Right? If they you know, if a CEO of one of those companies was to say, "You know, we're we're not going to do this anymore."
[07:04] They would just be replaced. Because the investors are putting their money up because they want to
[07:08] money up because they want to create AGI.
[07:10] create AGI and reap the benefits of it.
[07:13] So, it's a strange situation where every.
[07:16] at least all the ones I've spoken to.
[07:18] I haven't spoken to Sam Altman about this, but you know, Sam Altman.
[07:23] even before.
[07:25] becoming CEO of OpenAI.
[07:27] said that creating super human.
[07:30] intelligence is the biggest risk.
[07:34] to human existence that there is.
[07:36] My worst fears are that we cause significant we, the field, the.
[07:40] technology, the industry cause.
[07:41] significant harm to the world.
[07:43] You know, Elon Musk is also on record saying this.
[07:45] So, uh Dario Amodei estimates up to a 25%.
[07:50] risk of extinction.
[07:52] Was there a particular moment when you realized that.
[07:56] these CEOs are well aware of the extinction-level risks?
[08:00] I mean, they all signed a statement in May of '23.
[08:03] Uh called it's called the extinction statement.
[08:05] It basically says AGI is an
[08:10] statement.
[08:10] It basically says AGI is an extinction risk.
[08:11] extinction risk at the same level as nuclear war and
[08:13] at the same level as nuclear war and pandemics.
[08:15] pandemics.
[08:15] But I don't think they feel it in their
[08:17] But I don't think they feel it in their gut.
[08:19] gut.
[08:19] You know, imagine that you are one of
[08:20] You know, imagine that you are one of the nuclear physicists.
[08:22] the nuclear physicists.
[08:23] You know,
[08:23] I guess you've seen Oppenheimer, right?
[08:25] So, you're there, you're watching that
[08:27] So, you're there, you're watching that first nuclear explosion.
[08:30] first nuclear explosion.
[08:31] How how would that make you
[08:33] how would that make you feel about the potential impact of
[08:36] feel about the potential impact of nuclear war on the human race?
[08:38] nuclear war on the human race?
[08:38] Right?
[08:38] I I think you would probably
[08:40] Right? I I think you would probably become a pacifist and say, "This weapon
[08:43] become a pacifist and say, "This weapon is so terrible.
[08:43] We have got to find a
[08:46] is so terrible. We have got to find a way to
[08:48] way to uh keep it under control.
[08:49] uh keep it under control.
[08:49] We are not there yet
[08:53] We are not there yet with the people making these decisions
[08:55] with the people making these decisions and certainly not with the governments.
[08:58] and certainly not with the governments.
[08:58] Right?
[08:58] You know,
[09:00] Right? You know, what policy makers they
[09:02] what policy makers they you know, they listen to experts.
[09:06] you know, they listen to experts.
[09:06] They keep their finger in the wind.
[09:08] keep their finger in the wind.
[09:08] You got some experts, you know,
[09:10] You got some experts, you know, dangling 50 billion-dollar checks and saying, "Oh, you know, all that doomer stuff, it's just fringe nonsense. Don't worry about it. Take my 50 billion-dollar check."
[09:21] You know, on the other side you've got very well-meaning, brilliant scientists like like Jeff Hinton saying, "Actually, no, this is the end of the human race."
[09:30] But Jeff doesn't have a 50 billion-dollar check.
[09:34] So, the view is the only way to stop the race is if governments intervene and say, "Okay, we don't we don't want this race to go ahead until we can be sure that it's going ahead in absolute safety."
[09:53] Closing off on your career journey, you got a you received an OBE from Queen Elizabeth.
[09:58] Uh yes. And what was the listed reason for that for the award?
[10:01] Uh contributions to artificial intelligence research. And you've been listed as a Time magazine most influential person in in AI several years in a row including
[10:12] in AI several years in a row including this year.
[10:13] this year.
[10:15] In 2025.
[10:17] Yep.
[10:17] Now there's two terms here that are central to the things we're going to discuss.
[10:20] One of them is AI and the other is AGI.
[10:22] In my muggle in interpretation of that, it's artificial general is when the system, the computer, whatever it might be, the technology, has generalized intelligence which means that it could theoretically see, understand um the world.
[10:40] It knows everything.
[10:40] It can understand everything in the the world as well as or better than a human being.
[10:45] Yep.
[10:45] can do it.
[10:47] And I think take action as well.
[10:49] I mean so some people say, "Oh, you know, AGI doesn't have to have a body."
[10:54] But a good chunk of our intelligence actually is about managing our body, about perceiving the real environment and acting on it, moving, grasping and so on.
[11:05] So I think that's part of intelligence and and AGI systems should be able to operate robots successfully.
[11:12] But there's often a misunderstanding,
[11:13] But there's often a misunderstanding, right?
[11:14] That people say, "Well, if it doesn't have a robot body, then it can't actually do anything."
[11:20] But then if you remember most of us don't do things with our bodies.
[11:28] Some people do.
[11:31] Bricklayers, painters, gardeners, chefs.
[11:35] Um but people who do podcasts you're doing it with your mind.
[11:39] Right? You're doing it with your ability to to produce language.
[11:44] Uh you know, Adolf Hitler didn't do it with his body.
[11:49] He did it by producing language.
[11:52] I hope you're not comparing us.
[11:56] But uh you know, so even an AGI that has no body uh it actually has more access to the human race than Adolf Hitler ever did because it can send emails and texts to what, 3/4 of the world's population directly.
[12:12] It can It also speaks all of
[12:15] directly. It can It also speaks all of their languages.
[12:17] their languages. And it can devote 24 hours a day to each
[12:21] And it can devote 24 hours a day to each individual person on Earth
[12:24] individual person on Earth to convince them of to do whatever it
[12:26] to convince them of to do whatever it wants them to do. And our whole society
[12:27] wants them to do. And our whole society runs down the internet. I mean, if
[12:29] runs down the internet. I mean, if there's an issue with the internet,
[12:30] there's an issue with the internet, everything breaks down in society.
[12:32] everything breaks down in society. Airplanes become grounded. And we'll
[12:34] Airplanes become grounded. And we'll have a electricity's running off as
[12:36] have a electricity's running off as internet systems.
[12:38] internet systems. So I mean, my entire life it seems to
[12:40] So I mean, my entire life it seems to run off the internet now.
[12:42] run off the internet now. Yeah. Water supplies. So so this is one
[12:44] Yeah. Water supplies. So so this is one of the roots by which AI systems could
[12:48] of the roots by which AI systems could bring about
[12:49] bring about a medium-size catastrophe
[12:52] a medium-size catastrophe is by basically shutting down
[12:55] is by basically shutting down our life support systems.
[12:58] our life support systems. Do you believe that at some point in the
[13:01] Do you believe that at some point in the coming decades
[13:02] coming decades we'll arrive at a point of AGI where
[13:05] we'll arrive at a point of AGI where these systems are generally intelligent?
[13:08] these systems are generally intelligent? Uh yes, I think it's virtually certain
[13:12] Uh yes, I think it's virtually certain unless something else intervenes like a
[13:15] unless something else intervenes like a nuclear war or
[13:17] nuclear war or or we may refrain from doing it.
[13:19] But I think it will be extraordinarily difficult uh for us to refrain.
[13:25] When I look down the list of predictions from the top 10 AI CEOs on when AGI will arrive, you've got Sam Altman who's the founder of OpenAI/ChatGPT um says before 2030.
[13:37] Demis at DeepMind says 2030 to 2035.
[13:43] Jensen from Nvidia says around 5 years.
[13:47] Dario at Anthropic says 2026, 2027, powerful AI close to AGI.
[13:50] Elon says in the 2020s.
[13:55] Um and I go down the list of all of them and they're all saying relatively within 5 years.
[14:02] I actually think it'll take longer.
[14:03] I don't think you can make a prediction based on engineering um in the sense that yes, we could make machines 10 times bigger and 10 times faster.
[14:17] bigger and 10 times faster.
[14:20] But that's probably not the reason why we don't have AGI.
[14:22] Right?
[14:27] In fact, I think we have far more computing power than we need for AGI.
[14:29] Maybe a thousand times more than we need.
[14:33] The reason we don't have AGI is cuz we don't understand how to make it properly.
[14:37] Um what we've seized upon is one particular technology called the language model and we observe that as you make language models bigger they produce text, language that's more coherent and sounds more intelligent.
[14:58] And so mostly what's been happening the last few years is just, "Okay, let's keep doing that."
[15:06] Because one thing companies are very good at unlike universities is spending money.
[15:11] They have spent gargantuan amounts of money.
[15:15] And they're going to spend even more gargantuan
[15:18] gargantuan amounts of money.
[15:20] I mean, you know, we amounts of money.
[15:22] I mean, you know, we mentioned nuclear weapons.
[15:24] So the Manhattan Project uh in World War II to develop nuclear
[15:27] weapons
[15:28] its budget in 2025 dollars was about 20 odd billion dollars.
[15:37] The budget for AGI is going to be a trillion dollars next year.
[15:42] So 50 times bigger than the Manhattan Project.
[15:46] Humans have a remarkable history of figuring things out when they galvanize towards a shared objective.
[15:53] You know, thinking about the the moon landings or whatever it else it might be through history.
[15:59] And the thing that makes this feel all quite inevitable to me is just the sheer volume of money being invested into it.
[16:05] I've never seen anything like it in my life.
[16:07] Well, there's never been anything like this in history.
[16:08] This is the biggest technology project in human history by orders of magnitude.
[16:13] And there doesn't seem to be anybody that is pausing
[16:19] that is pausing to ask the questions about safety.
[16:21] to ask the questions about safety.
[16:22] It doesn't even It doesn't even appear that there's room for that in such a race.
[16:25] race.
[16:27] I think that's right. To varying extents, each of these companies has a division that focuses on safety.
[16:33] has a division that focuses on safety.
[16:35] Does that division have any sway?
[16:37] Can they tell the other divisions, "No, you can't release that system"?
[16:39] Not really.
[16:41] Um
[16:42] I think some of the companies do take it more seriously.
[16:44] Anthropic uh does.
[16:46] I think Google DeepMind.
[16:49] Even there, I think the commercial imperative to be at the forefront is absolutely vital.
[16:59] If a company is perceived as you know, falling behind and not likely to be competitive not likely to be the one to reach AGI first, then people will move their money elsewhere very quickly.
[17:16] And we saw some quite high-profile departures from company like companies
[17:19] departures from company like companies like OpenAI.
[17:21] Um I don't like OpenAI.
[17:25] Um I don't a chap called Jan LeCun left who was a chap called Jan LeCun left who was working on AI safety at OpenAI.
[17:28] AI safety at OpenAI.
[17:30] And he said that the reason for his leaving was that safety culture and processes processes have taken a backseat to shiny products at OpenAI.
[17:35] And he gradually lost trust in leadership.
[17:41] But also Ilya Sutskever Sutskever.
[17:43] Uh Ilya Sutskever, yeah.
[17:45] Sutskever. So he was the co-founder uh co-founder and chief scientist for a while.
[17:49] And then yeah, so he and Jan LeCun were the main safety people.
[17:52] Um
[17:54] And so when they say OpenAI doesn't care about safety that's pretty concerning.
[18:02] I've heard you talk about this gorilla problem.
[18:08] What is the gorilla problem?
[18:10] As a way to understand AI in the context of humans.
[18:13] So so the gorilla problem is is the problem that gorillas face with respect to humans.
[18:19] So you can imagine that, you know, a few
[18:21] So you can imagine that, you know, a few million years ago the the human line branched off from the gorilla line in evolution.
[18:28] Uh and now the gorillas are looking at the human line and saying, "Yeah, well was that a good idea?"
[18:36] And they have no um they have no say in whether they continue to exist.
[18:41] Because we have a We are much smarter than they are.
[18:43] If we chose to, we could make them extinct in in a couple of weeks.
[18:48] And there's nothing they can do about it.
[18:49] So that's the gorilla problem, right?
[18:52] Just the the problem a species faces uh when there's another species that's much more capable.
[19:00] And so this says that intelligence is actually the single most important factor to control planet Earth.
[19:06] Yes, intelligence is the ability to bring about what you want in the world.
[19:12] And we're in the process of making something more intelligent than us.
[19:16] Which suggests that maybe we become the gorillas.
[19:18] Exactly. Yep. Is that Is there
[19:21] Gorillas. Exactly. Yep.
[19:21] Is that is there any fault in the reasoning there?
[19:23] Any fault in the reasoning there?
[19:25] Because it seems to make such perfect sense to me, but
[19:28] if it why doesn't why don't people stop then?
[19:30] Cuz it it seems like a crazy thing to want to
[19:34] Because they think that uh if they create this technology
[19:39] it will have enormous economic value.
[19:41] They'll be able to use it to replace all the human workers in the world uh to develop new uh products, drugs um forms of entertainment, any anything that has economic value, you could use AGI to to create it.
[20:01] And and maybe it's just an irresistible thing in itself, right?
[20:05] I think we as humans place so much store on our intelligence, you know, you know,
[20:12] how we think about, you know, what is the pinnacle of human achievement.
[20:19] If we had AGI, we could go
[20:22] If we had AGI, we could go way higher
[20:23] way higher than that. So, it it's very seductive
[20:27] than that. So, it it's very seductive for people to want to create this
[20:30] for people to want to create this technology.
[20:31] technology. And I think
[20:33] And I think people are just fooling themselves
[20:35] people are just fooling themselves if they think it's naturally going to be
[20:38] if they think it's naturally going to be controllable.
[20:40] controllable. I mean, the question is
[20:42] I mean, the question is how are you going to retain power
[20:44] how are you going to retain power forever
[20:46] forever over entities more powerful than
[20:48] over entities more powerful than yourself?
[20:50] yourself? And pull the plug out.
[20:51] And pull the plug out. People say that sometimes in the comment
[20:53] People say that sometimes in the comment section when we talk about AI, they say,
[20:54] section when we talk about AI, they say, "Well, I'll just pull the plug out."
[20:56] "Well, I'll just pull the plug out." Yeah, it's it's sort of funny. In fact,
[20:58] Yeah, it's it's sort of funny. In fact, you know, yeah, reading the comment
[20:59] you know, yeah, reading the comment sections
[21:00] sections in newspapers whenever there's an AI
[21:02] in newspapers whenever there's an AI article,
[21:04] article, there'll be people who say, "Oh, you can
[21:07] there'll be people who say, "Oh, you can just pull the plug out, right?" As if a
[21:08] just pull the plug out, right?" As if a superintelligent machine would never
[21:09] superintelligent machine would never have thought of that one.
[21:11] have thought of that one. Right?
[21:12] Right? Don't forget it's watched all those
[21:13] Don't forget it's watched all those films where they did try to pull the
[21:14] films where they did try to pull the plug out. Another thing they say, "Well,
[21:17] plug out. Another thing they say, "Well, you know, as long as it's not conscious,
[21:20] you know, as long as it's not conscious, then it doesn't matter. It won't ever do
[21:22] then it doesn't matter. It won't ever do anything."
[21:25] Uh, um,
[21:27] Uh, um, which is
[21:29] which is completely off the point. Because, you
[21:32] completely off the point. Because, you know, I I don't think the gorillas are
[21:34] know, I I don't think the gorillas are sitting there saying, "Oh, yeah, you
[21:35] sitting there saying, "Oh, yeah, you know,
[21:36] know, if only those humans hadn't been
[21:38] if only those humans hadn't been conscious,
[21:39] conscious, everything would have been fine."
[21:41] everything would have been fine." Right? No, of course not. What would
[21:43] Right? No, of course not. What would make gorillas go extinct is the things
[21:44] make gorillas go extinct is the things that humans do.
[21:46] that humans do. Right? How we behave, our ability to act
[21:50] Right? How we behave, our ability to act successfully
[21:51] successfully in the world. So, when I play chess
[21:54] in the world. So, when I play chess against my
[21:55] against my iPhone and I lose,
[21:58] iPhone and I lose, right? I don't I don't
[21:59] right? I don't I don't think, "Oh, well, I'm losing cuz it's
[22:01] think, "Oh, well, I'm losing cuz it's conscious." Right? No, I'm just losing
[22:03] conscious." Right? No, I'm just losing because it's better than I am at at in
[22:06] because it's better than I am at at in that little world,
[22:08] that little world, um, moving the bits around uh, to to get
[22:10] um, moving the bits around uh, to to get what it wants.
[22:12] what it wants. And and so, consciousness has nothing to
[22:15] And and so, consciousness has nothing to do with it, right? Competence is the
[22:16] do with it, right? Competence is the thing we're concerned about. So, I think
[22:19] thing we're concerned about. So, I think the only hope is can we
[22:22] the only hope is can we simultaneously
[22:25] simultaneously build machines that are more intelligent
[22:26] build machines that are more intelligent than us,
[22:28] than us, but guarantee
[22:31] but guarantee that they will always act in our best
[22:35] that they will always act in our best interest.
[22:36] interest. So, throwing that question to you, can
[22:38] So, throwing that question to you, can we build machines that are more
[22:39] we build machines that are more intelligent than us that will also
[22:41] intelligent than us that will also always act in our best interests?
[22:44] always act in our best interests? It sounds like a bit of
[22:46] It sounds like a bit of uh, contradiction to some degree.
[22:48] uh, contradiction to some degree. Because it's kind of like me saying,
[22:51] Because it's kind of like me saying, "I've got a French bulldog called Pablo
[22:53] "I've got a French bulldog called Pablo that's uh, 9 years old."
[22:55] that's uh, 9 years old." And it's like saying that he could be
[22:56] And it's like saying that he could be more intelligent than me, yet I still
[22:59] more intelligent than me, yet I still walk him and decide when he gets fed.
[23:01] walk him and decide when he gets fed. I think if he was more intelligent than
[23:03] I think if he was more intelligent than me, he would be walking me. I'd be on
[23:05] me, he would be walking me. I'd be on the leash. That's the that's the trick,
[23:07] the leash. That's the that's the trick, right? Can we make AI systems whose only
[23:10] right? Can we make AI systems whose only purpose
[23:11] purpose is to further human interest?
[23:14] is to further human interest? And I think the answer is
[23:16] And I think the answer is yes.
[23:18] yes. And this is actually what I've been
[23:19] And this is actually what I've been working on.
[23:21] working on. So, I I I think one part of my career
[23:22] So, I I I think one part of my career that I didn't mention is
[23:24] that I didn't mention is is sort of having this epiphany,
[23:27] is sort of having this epiphany, uh, while I was on sabbatical in Paris.
[23:30] uh, while I was on sabbatical in Paris. This was 2013 or so.
[23:32] This was 2013 or so. Just
[23:34] Just realizing that
[23:36] realizing that further progress
[23:37] further progress in the capabilities of AI,
[23:40] in the capabilities of AI, uh, you know, if if we succeeded in
[23:43] uh, you know, if if we succeeded in creating
[23:44] creating real superhuman intelligence, that it
[23:47] real superhuman intelligence, that it was potentially a catastrophe.
[23:49] was potentially a catastrophe. And so, I pretty much switched my focus
[23:53] And so, I pretty much switched my focus to work on how do we make it so that
[23:54] to work on how do we make it so that it's guaranteed to be safe.
[23:57] it's guaranteed to be safe. Are you somewhat troubled by
[24:01] Are you somewhat troubled by everything that's going on at the moment
[24:02] everything that's going on at the moment with
[24:04] with with AI and how it's progressing? Cuz
[24:06] with AI and how it's progressing? Cuz you strike me as someone that's
[24:08] you strike me as someone that's somewhat troubled under the surface by
[24:11] somewhat troubled under the surface by the way things are moving forward
[24:14] the way things are moving forward and the speed in which they're moving
[24:15] and the speed in which they're moving forward?
[24:16] forward? That's an understatement. I'm
[24:19] That's an understatement. I'm appalled
[24:20] appalled actually by the lack of attention
[24:23] actually by the lack of attention to safety. I mean, imagine if someone's
[24:26] to safety. I mean, imagine if someone's building a nuclear power station
[24:28] building a nuclear power station in your neighborhood.
[24:31] And you go along to the chief engineer
[24:33] And you go along to the chief engineer and you say, "Okay, this nuclear thing,
[24:35] and you say, "Okay, this nuclear thing, I've heard that they can actually
[24:37] I've heard that they can actually explode, right? There was this nuclear
[24:39] explode, right? There was this nuclear explosion
[24:41] explosion that happened in Hiroshima. And so, I'm
[24:43] that happened in Hiroshima. And so, I'm a bit worried about this. You know, what
[24:45] a bit worried about this. You know, what steps are you taking to make sure that
[24:47] steps are you taking to make sure that we don't have a nuclear explosion
[24:49] we don't have a nuclear explosion in our backyard?"
[24:52] in our backyard?" And the chief engineer says,
[24:54] And the chief engineer says, "Well, we thought about it. We don't
[24:55] "Well, we thought about it. We don't really have an answer."
[24:59] Yeah. You would What would you say? You
[25:02] Yeah. You would What would you say? You would
[25:03] would I think you would you would use some
[25:05] I think you would you would use some expletives.
[25:07] expletives. Well,
[25:10] >> and say,
[25:11] >> and say, you know, get these get these people
[25:13] you know, get these get these people out.
[25:14] out. I mean, what are they doing?
[25:17] I mean, what are they doing? You read out the list of, you know, the
[25:20] You read out the list of, you know, the projected dates for AGI.
[25:22] projected dates for AGI. But notice also that those people
[25:25] But notice also that those people I think I mentioned Dara Khosrowshahi
[25:27] I think I mentioned Dara Khosrowshahi says a 25% chance of extinction.
[25:30] says a 25% chance of extinction. Elon Musk has a 30% chance of
[25:31] Elon Musk has a 30% chance of extinction. Sam Altman says
[25:36] extinction. Sam Altman says basically that AGI is the biggest risk
[25:38] basically that AGI is the biggest risk to human existence.
[25:40] to human existence. So, what are they doing? They are
[25:41] So, what are they doing? They are playing Russian roulette with every
[25:44] playing Russian roulette with every human being on Earth
[25:47] human being on Earth without our permission. They're coming
[25:48] without our permission. They're coming into our houses,
[25:50] into our houses, putting a gun to the head of our
[25:51] putting a gun to the head of our children,
[25:53] children, pulling the trigger,
[25:55] pulling the trigger, and saying, "Well, you know,
[25:57] and saying, "Well, you know, possibly everyone will die. Oops.
[26:00] possibly everyone will die. Oops. But possibly we'll get incredibly rich."
[26:04] That's what they're doing.
[26:07] That's what they're doing. Did they ask us? No. Why is the
[26:09] Did they ask us? No. Why is the government allowing them to do this?
[26:12] government allowing them to do this? Because they dangle $50 checks in front
[26:15] Because they dangle $50 checks in front of the governments.
[26:17] of the governments. So, I think troubled under the surface
[26:20] So, I think troubled under the surface is an understatement.
[26:21] is an understatement. What would be an accurate statement?
[26:24] What would be an accurate statement? Appalled.
[26:25] Appalled. And I
[26:27] And I I am devoting my life to trying
[26:31] I am devoting my life to trying to
[26:32] to divert from this course of history into
[26:34] divert from this course of history into a different one.
[26:36] a different one. Do you have any regrets about things you
[26:38] Do you have any regrets about things you could have done in the past? Because
[26:40] could have done in the past? Because you've been so influential on the
[26:41] you've been so influential on the subject of AI. You wrote the textbook
[26:44] subject of AI. You wrote the textbook that many of these people would have
[26:45] that many of these people would have studied on the subject of AI more than
[26:47] studied on the subject of AI more than 30 years ago. Do you do you have When
[26:49] 30 years ago. Do you do you have When you're alone at night and you think
[26:50] you're alone at night and you think about decisions you've made on this in
[26:52] about decisions you've made on this in this field because of your scope of
[26:53] this field because of your scope of influence, is there anything you you
[26:54] influence, is there anything you you regret? Well, I do wish I had understood
[26:58] regret? Well, I do wish I had understood earlier,
[27:00] earlier, uh, what I understand now.
[27:02] uh, what I understand now. We could have developed
[27:04] We could have developed safe AI systems. I think the there are
[27:07] safe AI systems. I think the there are some weaknesses in the framework, which
[27:09] some weaknesses in the framework, which I can explain. But I think that
[27:11] I can explain. But I think that framework could have evolved
[27:13] framework could have evolved to develop actually safe AI systems
[27:16] to develop actually safe AI systems where we could prove
[27:18] where we could prove mathematically that the system is going
[27:21] mathematically that the system is going to act in our interest. The kind of AI
[27:23] to act in our interest. The kind of AI systems we're building now,
[27:26] systems we're building now, we don't understand how they work.
[27:28] we don't understand how they work. We don't understand how they work. It's
[27:30] We don't understand how they work. It's it's a strange thing to build something
[27:33] it's a strange thing to build something where you don't understand how it works.
[27:34] where you don't understand how it works. I mean, there's no sort of comparable
[27:36] I mean, there's no sort of comparable through human history. Usually with
[27:37] through human history. Usually with machines, we can pull it apart and see
[27:39] machines, we can pull it apart and see what cogs are doing what and how they
[27:41] what cogs are doing what and how they function.
[27:41] function. >> we
[27:42] >> we we put the cogs together, right? So,
[27:45] we put the cogs together, right? So, with with most machines, we designed it
[27:47] with with most machines, we designed it to have a certain behavior. So, we don't
[27:49] to have a certain behavior. So, we don't need to pull it apart and see what the
[27:51] need to pull it apart and see what the cogs are cuz we put the cogs in there in
[27:52] cogs are cuz we put the cogs in there in the first place. Right? One by one, we
[27:55] the first place. Right? One by one, we figured out what what the pieces needed
[27:57] figured out what what the pieces needed to be, how they work together to produce
[27:59] to be, how they work together to produce the effect that we want. So, the best
[28:02] the effect that we want. So, the best analogy I can come up with it is,
[28:05] analogy I can come up with it is, you know, the
[28:06] you know, the the first cave person who left a bowl of
[28:10] the first cave person who left a bowl of fruit in the sun and forgot about it,
[28:13] fruit in the sun and forgot about it, and then came back a few weeks later,
[28:15] and then came back a few weeks later, and there was a sort of this big soupy
[28:16] and there was a sort of this big soupy thing, and they drank it, and got
[28:18] thing, and they drank it, and got completely shit-faced. They got drunk,
[28:20] completely shit-faced. They got drunk, because they And they got this effect.
[28:23] because they And they got this effect. They had no idea
[28:25] They had no idea how it worked, but they were very happy
[28:26] how it worked, but they were very happy about it.
[28:28] about it. No doubt that person made a lot of money
[28:29] No doubt that person made a lot of money from it.
[28:30] from it. Uh, so, yeah, it it is kind of bizarre.
[28:34] Uh, so, yeah, it it is kind of bizarre. But my mental picture of these things is
[28:36] But my mental picture of these things is is like a chain-link fence.
[28:39] is like a chain-link fence. Right? So, you've got
[28:40] Right? So, you've got lots of these connections,
[28:43] lots of these connections, and each of those connections can be its
[28:46] and each of those connections can be its connection strength can be adjusted.
[28:48] connection strength can be adjusted. And then, uh,
[28:51] And then, uh, you know, a signal comes in one end of
[28:52] you know, a signal comes in one end of this
[28:53] this chain-link fence and passes through all
[28:55] chain-link fence and passes through all these connections and comes out the
[28:57] these connections and comes out the other end.
[28:58] other end. And the signal that comes out the other
[28:59] And the signal that comes out the other end is affected by your adjusting of all
[29:02] end is affected by your adjusting of all the connection strengths.
[29:04] the connection strengths. So, what you do is you you get a whole
[29:06] So, what you do is you you get a whole lot of training data,
[29:08] lot of training data, and you adjust all those connection
[29:09] and you adjust all those connection strengths so that the signal that comes
[29:11] strengths so that the signal that comes out the other end of the network
[29:13] out the other end of the network is the right answer to the question. So,
[29:15] is the right answer to the question. So, if your training data
[29:17] if your training data is
[29:19] is lots of photos of animals, then all
[29:21] lots of photos of animals, then all those pixels go in one end of of the
[29:23] those pixels go in one end of of the network, and out the other end, you
[29:26] network, and out the other end, you know, it activates the llama output or
[29:30] know, it activates the llama output or the
[29:31] the dog output or the cat output or the
[29:33] dog output or the cat output or the ostrich output.
[29:34] ostrich output. And uh, and you just keep adjusting all
[29:36] And uh, and you just keep adjusting all the connection strengths in this network
[29:37] the connection strengths in this network until the outputs of the network are the
[29:39] until the outputs of the network are the ones you want. But we don't really know
[29:41] ones you want. But we don't really know what's going on across all of those
[29:43] what's going on across all of those different chains.
[29:44] different chains. >> So, what's going on inside that network?
[29:46] >> So, what's going on inside that network? Well, so now you have to imagine that
[29:49] Well, so now you have to imagine that this network this chain-link fence is is
[29:52] this network this chain-link fence is is a thousand square miles in extent.
[29:55] a thousand square miles in extent. Okay. So, it's covering the whole of the
[29:57] Okay. So, it's covering the whole of the San Francisco Bay Area or the whole of
[30:00] San Francisco Bay Area or the whole of London inside the M25. Right, that's how
[30:03] London inside the M25. Right, that's how big it is. And the lights are off. It's
[30:05] big it is. And the lights are off. It's night time.
[30:07] night time. So, you might have in that network about
[30:09] So, you might have in that network about a trillion
[30:11] a trillion uh adjustable parameters. And then you
[30:13] uh adjustable parameters. And then you do quintillions or sextillions of small
[30:16] do quintillions or sextillions of small random adjustments to those parameters
[30:20] random adjustments to those parameters uh until you get the behavior that you
[30:22] uh until you get the behavior that you want.
[30:24] want. I've heard Sam Altman say that in the
[30:25] I've heard Sam Altman say that in the future
[30:26] future he doesn't believe they'll need much
[30:28] he doesn't believe they'll need much training data
[30:30] training data at all to make these models progress
[30:32] at all to make these models progress themselves because there comes a point
[30:33] themselves because there comes a point where
[30:34] where the models are so smart that they can
[30:37] the models are so smart that they can train themselves and improve themselves
[30:40] train themselves and improve themselves without us needing to pump in
[30:42] without us needing to pump in articles and books and scour the
[30:44] articles and books and scour the internet. Yeah, it should it should work
[30:47] internet. Yeah, it should it should work that way. So, I think what he's
[30:48] that way. So, I think what he's referring to and this is something that
[30:50] referring to and this is something that several companies are now worried might
[30:54] several companies are now worried might start happening
[30:56] start happening is that the AI system becomes
[30:59] is that the AI system becomes capable of doing AI research
[31:03] capable of doing AI research by itself.
[31:05] by itself. And so,
[31:06] And so, uh you have a system
[31:08] uh you have a system with a certain capability. I mean,
[31:10] with a certain capability. I mean, crudely we could call it an IQ, but
[31:13] crudely we could call it an IQ, but it's it's not really an IQ. But anyway,
[31:15] it's it's not really an IQ. But anyway, imagine that
[31:16] imagine that it's got an IQ of 150 and uses that to
[31:19] it's got an IQ of 150 and uses that to do AI research
[31:21] do AI research comes up with better algorithms or
[31:23] comes up with better algorithms or better designs for hardware or better
[31:25] better designs for hardware or better ways to use the data
[31:27] ways to use the data updates itself. Now it has an IQ of 170.
[31:31] updates itself. Now it has an IQ of 170. And now it does more AI research except
[31:33] And now it does more AI research except that now it's got an IQ of 170, so it's
[31:35] that now it's got an IQ of 170, so it's even better
[31:37] even better at doing the AI research.
[31:39] at doing the AI research. And so, you know, next iteration it's
[31:41] And so, you know, next iteration it's 250 and and so on. So, this this is an
[31:45] 250 and and so on. So, this this is an idea that one of Alan Turing's friends,
[31:47] idea that one of Alan Turing's friends, I.J. Good
[31:49] I.J. Good uh wrote out in 1965 called the
[31:52] uh wrote out in 1965 called the intelligence explosion, right? That
[31:54] intelligence explosion, right? That one of the things an intelligent system
[31:56] one of the things an intelligent system could do is
[31:58] could do is to do AI research and therefore make
[32:00] to do AI research and therefore make itself more intelligent and this would
[32:02] itself more intelligent and this would uh this would very rapidly take off
[32:06] uh this would very rapidly take off and leave the humans far behind. Is that
[32:08] and leave the humans far behind. Is that what they call the fast take off? That's
[32:10] what they call the fast take off? That's called the fast take off. Sam Altman
[32:12] called the fast take off. Sam Altman said, "I think a fast take off is more
[32:15] said, "I think a fast take off is more possible than I thought a couple of
[32:16] possible than I thought a couple of years ago, which I guess is that moment
[32:18] years ago, which I guess is that moment where the AGI starts teaching itself."
[32:20] where the AGI starts teaching itself." In in his blog, The Gentle Singularity,
[32:23] In in his blog, The Gentle Singularity, he said, "We may already be past the
[32:25] he said, "We may already be past the event horizon
[32:27] event horizon of take off."
[32:29] of take off." What does What does he mean by event
[32:30] What does What does he mean by event horizon? The event horizon is is a
[32:32] horizon? The event horizon is is a phrase borrowed from astrophysics and it
[32:36] phrase borrowed from astrophysics and it refers to uh the black hole.
[32:39] refers to uh the black hole. And the event horizon, think it if
[32:42] And the event horizon, think it if you've got some very very massive
[32:44] you've got some very very massive object that's heavy enough
[32:47] object that's heavy enough that it actually prevents light from
[32:50] that it actually prevents light from escaping. That's why it's called the
[32:51] escaping. That's why it's called the black hole. It's so heavy that light
[32:53] black hole. It's so heavy that light can't escape. So, if you're inside the
[32:56] can't escape. So, if you're inside the event horizon, then
[32:58] event horizon, then then light can't escape beyond that. So,
[33:01] then light can't escape beyond that. So, I think what he's
[33:03] I think what he's what he's meaning is if we're beyond the
[33:05] what he's meaning is if we're beyond the event horizon, it means that you know,
[33:07] event horizon, it means that you know, now we're just trapped in the
[33:09] now we're just trapped in the gravitational attraction
[33:11] gravitational attraction of the black hole or in this case
[33:14] of the black hole or in this case we're we're trapped in the inevitable
[33:18] we're we're trapped in the inevitable slide, if you want, towards AGI.
[33:21] slide, if you want, towards AGI. When you when you think about the
[33:22] When you when you think about the economic value of AGI, which I've
[33:25] economic value of AGI, which I've estimated at uh 15 quadrillion dollars
[33:29] estimated at uh 15 quadrillion dollars that acts as a giant magnet in the
[33:32] that acts as a giant magnet in the future.
[33:34] future. We're being pulled towards it.
[33:35] We're being pulled towards it. >> We're being pulled towards it and the
[33:36] >> We're being pulled towards it and the closer we get the stronger
[33:39] closer we get the stronger the force.
[33:41] the force. The probability, you know, the closer we
[33:42] The probability, you know, the closer we get the
[33:43] get the the higher the probability that we will
[33:45] the higher the probability that we will actually get there.
[33:46] actually get there. So, people are more willing to invest
[33:48] So, people are more willing to invest and we also start to see spin-offs from
[33:51] and we also start to see spin-offs from that investment
[33:53] that investment such as ChatGPT, right? Which is
[33:56] such as ChatGPT, right? Which is you know, generates a certain amount of
[33:57] you know, generates a certain amount of revenue and so on. So,
[34:00] revenue and so on. So, so it does act as a magnet.
[34:02] so it does act as a magnet. And the closer we get, the harder it is
[34:04] And the closer we get, the harder it is to pull out of that field.
[34:07] to pull out of that field. It's interesting when you think that
[34:08] It's interesting when you think that this could be the the end of the human
[34:10] this could be the the end of the human story, this idea that the end of the
[34:12] story, this idea that the end of the human story was that we created our
[34:15] human story was that we created our successor.
[34:16] successor. Like we we summoned our
[34:18] Like we we summoned our the next iteration of
[34:21] the next iteration of life or
[34:23] life or intelligence ourselves. Like we took
[34:25] intelligence ourselves. Like we took ourselves out.
[34:27] ourselves out. It is quite like it's just removing
[34:28] It is quite like it's just removing ourselves and the catastrophe from it
[34:30] ourselves and the catastrophe from it for a second. It is It is an
[34:31] for a second. It is It is an unbelievable story.
[34:34] unbelievable story. Yeah, and
[34:36] Yeah, and you know, there are
[34:37] you know, there are many legends
[34:40] many legends the sort of be careful what you wish for
[34:43] the sort of be careful what you wish for legend. And in fact, the King Midas
[34:45] legend. And in fact, the King Midas legend
[34:46] legend is is very relevant here. What's that?
[34:49] is is very relevant here. What's that? >> Well, so King Midas is this legendary
[34:52] >> Well, so King Midas is this legendary king who
[34:54] king who lived in
[34:55] lived in modern-day Turkey, but I think it's sort
[34:56] modern-day Turkey, but I think it's sort of like Greek mythology. He is said to
[34:59] of like Greek mythology. He is said to have asked the gods to grant him a wish.
[35:04] have asked the gods to grant him a wish. The wish being
[35:05] The wish being that everything I touch should turn to
[35:07] that everything I touch should turn to gold.
[35:09] gold. So, he's incredibly greedy.
[35:11] So, he's incredibly greedy. Uh you know, we call this the Midas
[35:13] Uh you know, we call this the Midas touch.
[35:14] touch. And we think of the Midas touch as being
[35:16] And we think of the Midas touch as being like, you know, that's a good thing,
[35:18] like, you know, that's a good thing, right? Wouldn't that be cool?
[35:20] right? Wouldn't that be cool? But what happens? So, he
[35:22] But what happens? So, he uh you know, he goes to drink some water
[35:24] uh you know, he goes to drink some water and he finds that the water has turned
[35:26] and he finds that the water has turned to gold.
[35:27] to gold. And he goes to eat an apple and the
[35:28] And he goes to eat an apple and the apple turns to gold and he goes to
[35:31] apple turns to gold and he goes to you know, comfort his daughter and his
[35:32] you know, comfort his daughter and his daughter turns to gold.
[35:35] daughter turns to gold. And so, he dies in misery and
[35:37] And so, he dies in misery and starvation.
[35:38] starvation. So, this applies to our current
[35:42] So, this applies to our current situation in in two ways, actually. So,
[35:45] situation in in two ways, actually. So, one is that I think greed is driving us
[35:49] one is that I think greed is driving us to pursue
[35:51] to pursue a technology that will end up consuming
[35:54] a technology that will end up consuming us.
[35:55] us. And we will
[35:56] And we will perhaps die in misery and starvation
[35:58] perhaps die in misery and starvation instead.
[35:59] instead. The what it shows is how difficult it is
[36:02] The what it shows is how difficult it is to
[36:03] to correctly articulate what you want
[36:07] correctly articulate what you want the future to be like.
[36:08] the future to be like. For a long time
[36:10] For a long time the way we built AI systems was we
[36:12] the way we built AI systems was we created these algorithms
[36:14] created these algorithms where we could specify the objective and
[36:17] where we could specify the objective and then the machine would figure out how to
[36:19] then the machine would figure out how to achieve the objective and then achieve
[36:21] achieve the objective and then achieve it.
[36:22] it. So, you know, we specify what it means
[36:24] So, you know, we specify what it means to win at chess or to win at go
[36:26] to win at chess or to win at go and the algorithm figures out how to do
[36:28] and the algorithm figures out how to do it. Uh and it does it really well.
[36:30] it. Uh and it does it really well. So, that was, you know, standard AI up
[36:32] So, that was, you know, standard AI up until recently and
[36:34] until recently and it suffers from this drawback that sure,
[36:36] it suffers from this drawback that sure, we know how to specify the objective in
[36:38] we know how to specify the objective in chess.
[36:39] chess. But how do you specify the objective in
[36:40] But how do you specify the objective in life?
[36:42] life? Right? What do we want the future to be
[36:44] Right? What do we want the future to be like? Well, really hard to say and
[36:46] like? Well, really hard to say and almost any attempt
[36:47] almost any attempt to write it down
[36:49] to write it down precisely enough for the machine to
[36:51] precisely enough for the machine to bring it about
[36:52] bring it about would be wrong.
[36:54] would be wrong. And if you're giving a machine an
[36:55] And if you're giving a machine an objective which isn't aligned with what
[36:58] objective which isn't aligned with what we truly want the future to be like,
[37:00] we truly want the future to be like, right? You're actually
[37:01] right? You're actually setting up a chess match. And that
[37:04] setting up a chess match. And that match is one that you're going to lose
[37:06] match is one that you're going to lose when the machine is sufficiently
[37:07] when the machine is sufficiently intelligent. And so, that that's that's
[37:09] intelligent. And so, that that's that's problem number one.
[37:12] problem number one. Problem number two is that the kind of
[37:14] Problem number two is that the kind of technology we're building now
[37:16] technology we're building now we don't even know what its objectives
[37:17] we don't even know what its objectives are.
[37:19] are. So, it's not that we're specifying the
[37:21] So, it's not that we're specifying the objectives, but we're getting them
[37:22] objectives, but we're getting them wrong.
[37:23] wrong. We are growing these systems. They have
[37:26] We are growing these systems. They have objectives.
[37:28] objectives. But we don't even know what they are
[37:29] But we don't even know what they are because we didn't specify them. What
[37:31] because we didn't specify them. What we're finding through experiment with
[37:32] we're finding through experiment with them is that
[37:35] them is that they seem to have an extremely strong
[37:37] they seem to have an extremely strong self-preservation objective. What do you
[37:39] self-preservation objective. What do you mean by that? You can put them in
[37:41] mean by that? You can put them in hypothetical situations. Either they're
[37:43] hypothetical situations. Either they're going to get switched off and replaced
[37:45] going to get switched off and replaced or
[37:47] or they
[37:48] they have to allow someone, let's say, you
[37:49] have to allow someone, let's say, you know, someone has been locked in a
[37:52] know, someone has been locked in a machine room
[37:53] machine room that's kept at 3° C or they're going to
[37:55] that's kept at 3° C or they're going to freeze to death.
[37:58] freeze to death. They will choose to leave that guy
[37:59] They will choose to leave that guy locked in the machine room
[38:01] locked in the machine room and die rather than be switched off
[38:03] and die rather than be switched off themselves.
[38:05] themselves. And someone's done that test? Yeah.
[38:07] And someone's done that test? Yeah. What was the test? They they asked They
[38:09] What was the test? They they asked They asked the AI. Yep. They put Well, they
[38:11] asked the AI. Yep. They put Well, they put them in these hypothetical
[38:13] put them in these hypothetical situations and they allow the AI to
[38:15] situations and they allow the AI to decide what to do and it decides
[38:17] decide what to do and it decides to preserve its own existence, let the
[38:20] to preserve its own existence, let the guy die, and then lie about it.
[38:23] guy die, and then lie about it. In the King Midas analogy story
[38:26] In the King Midas analogy story one of the things that highlights for me
[38:28] one of the things that highlights for me is that there's always trade-offs in
[38:30] is that there's always trade-offs in life generally. And it's, you know,
[38:31] life generally. And it's, you know, especially when there's great upside,
[38:33] especially when there's great upside, there always appears to be a pretty
[38:34] there always appears to be a pretty grave downside.
[38:36] grave downside. Like there's almost nothing in my life
[38:37] Like there's almost nothing in my life where I go, it's all upside.
[38:39] where I go, it's all upside. Like even like having a dog, it shits on
[38:41] Like even like having a dog, it shits on my carpet. My girlfriend, you know, love
[38:43] my carpet. My girlfriend, you know, love her but, you know, not always easy.
[38:47] her but, you know, not always easy. Even with like going to the gym, I have
[38:48] Even with like going to the gym, I have to pick up these really really heavy
[38:49] to pick up these really really heavy weights at 10:00 p.m. at night sometimes
[38:52] weights at 10:00 p.m. at night sometimes when I don't feel like it. There's
[38:53] when I don't feel like it. There's always to get the muscles or the
[38:54] always to get the muscles or the six-pack, there's always a trade-off.
[38:56] six-pack, there's always a trade-off. And when you interview people for a
[38:57] And when you interview people for a living like I do you know, you hear
[38:58] living like I do you know, you hear about so many incredible things that can
[39:00] about so many incredible things that can help you in so many ways
[39:02] help you in so many ways but there is always a trade-off. There's
[39:04] but there is always a trade-off. There's always a way to overdo it. Mhm.
[39:05] always a way to overdo it. Mhm. Melatonin will help you sleep, but it
[39:07] Melatonin will help you sleep, but it also you'll wake up groggy and if you
[39:10] also you'll wake up groggy and if you overdo it, your brain might stop making
[39:11] overdo it, your brain might stop making melatonin. Like I can go through the
[39:12] melatonin. Like I can go through the entire list and one of the things I've
[39:13] entire list and one of the things I've always come to learn from doing this
[39:15] always come to learn from doing this podcast is whenever someone promises me
[39:17] podcast is whenever someone promises me a huge upside for something, it'll cure
[39:19] a huge upside for something, it'll cure cancer, it'll be utopia, you'll never
[39:21] cancer, it'll be utopia, you'll never have to work, you'll have a butler
[39:22] have to work, you'll have a butler around your house. Mhm. I my my first
[39:24] around your house. Mhm. I my my first instinct now is to say, "At what cost?"
[39:27] instinct now is to say, "At what cost?" Yeah. And when I think about the
[39:28] Yeah. And when I think about the economic cost here, if we start if we
[39:30] economic cost here, if we start if we start there
[39:32] start there have you got kids? I have four, yeah.
[39:34] have you got kids? I have four, yeah. Four kids.
[39:35] Four kids. What what How old is the youngest kid?
[39:37] What what How old is the youngest kid? 19. 19, okay. So, you're you're If
[39:40] 19. 19, okay. So, you're you're If you're say your kids were were 10 now
[39:42] you're say your kids were were 10 now Mhm. and they were coming to you and
[39:43] Mhm. and they were coming to you and they were saying, "Dad, what do you
[39:44] they were saying, "Dad, what do you think I should study?
[39:46] think I should study? Based on the way that you see the
[39:47] Based on the way that you see the future.
[39:49] future. A future of AGI. Say if all these CEOs
[39:51] A future of AGI. Say if all these CEOs are right and they're predicting AGI
[39:53] are right and they're predicting AGI within 5 years.
[39:55] within 5 years. What should I study, Dad?
[39:57] What should I study, Dad? Well, okay. So,
[39:58] Well, okay. So, let's look on the bright side and say
[40:00] let's look on the bright side and say that the CEOs all decide to pause their
[40:03] that the CEOs all decide to pause their AGI development, figure out how to make
[40:06] AGI development, figure out how to make it safe, and then resume uh in whatever
[40:10] it safe, and then resume uh in whatever technology path is actually going to be
[40:11] technology path is actually going to be safe. What does that do to human life?
[40:14] safe. What does that do to human life? If they pause? No, if if they succeed in
[40:18] If they pause? No, if if they succeed in creating AGI and they solve the safety
[40:20] creating AGI and they solve the safety problem.
[40:21] problem. And they solve the safety problem.
[40:23] And they solve the safety problem. >> So, yeah, cuz if they don't solve the
[40:24] >> So, yeah, cuz if they don't solve the safety problem, then, you know, you
[40:26] safety problem, then, you know, you should probably be finding a bunker or
[40:29] should probably be finding a bunker or going to Patagonia or somewhere in New
[40:31] going to Patagonia or somewhere in New Zealand. Do you mean that? Do you think
[40:32] Zealand. Do you mean that? Do you think I should be finding a bunker
[40:33] I should be finding a bunker >> No, cuz it's not actually going to help.
[40:35] >> No, cuz it's not actually going to help. Uh you know, it's it's not as if the AI
[40:38] Uh you know, it's it's not as if the AI system couldn't find you or I mean it's
[40:40] system couldn't find you or I mean it's it's interesting. So, we're going off in
[40:42] it's interesting. So, we're going off in a little bit of a digression here
[40:44] a little bit of a digression here from your question, but I'll come back
[40:46] from your question, but I'll come back to it.
[40:47] to it. So, people often ask, "Well, okay. So,
[40:49] So, people often ask, "Well, okay. So, how exactly do we go extinct?" And of
[40:51] how exactly do we go extinct?" And of course, if you ask the gorillas or the
[40:53] course, if you ask the gorillas or the dodos, you know, "How exactly do you
[40:55] dodos, you know, "How exactly do you think you're going to go extinct?"
[40:57] think you're going to go extinct?" They have the faintest idea.
[41:00] They have the faintest idea. Or humans do something and then we're
[41:01] Or humans do something and then we're all dead. So, the only things we can
[41:03] all dead. So, the only things we can imagine are the things we know how to do
[41:06] imagine are the things we know how to do that might bring about our own
[41:07] that might bring about our own extinction, like creating some carefully
[41:10] extinction, like creating some carefully engineered pathogen that infects
[41:12] engineered pathogen that infects everybody and then kills us.
[41:15] everybody and then kills us. Or starting a nuclear war.
[41:17] Or starting a nuclear war. Presumably, if something that's much
[41:18] Presumably, if something that's much more intelligent than us
[41:20] more intelligent than us would have much greater control over
[41:22] would have much greater control over physics
[41:23] physics than we do. We already do amazing
[41:26] than we do. We already do amazing things, right? I mean, it's amazing that
[41:28] things, right? I mean, it's amazing that I can take a little rectangular thing
[41:30] I can take a little rectangular thing out of my pocket and talk to someone on
[41:32] out of my pocket and talk to someone on the other side of the world.
[41:34] the other side of the world. Or even someone in space.
[41:36] Or even someone in space. It's just astonishing and we can take it
[41:39] It's just astonishing and we can take it for granted.
[41:40] for granted. Right? But imagine, you know,
[41:41] Right? But imagine, you know, superintelligent beings and their
[41:42] superintelligent beings and their ability to control physics. You know,
[41:45] ability to control physics. You know, perhaps they will find a way to just
[41:47] perhaps they will find a way to just divert the sun's energy a rack sort of
[41:49] divert the sun's energy a rack sort of go around the earth's orbit. So, you
[41:52] go around the earth's orbit. So, you know, literally the earth turns into a
[41:54] know, literally the earth turns into a snowball in in a few days. Maybe they'd
[41:57] snowball in in a few days. Maybe they'd just decide to leave.
[41:59] just decide to leave. Perhaps.
[42:00] Perhaps. >> Leave the earth. Maybe they'd look at
[42:01] >> Leave the earth. Maybe they'd look at the earth and go, "This isn't This is
[42:02] the earth and go, "This isn't This is not interesting. We know that over there
[42:04] not interesting. We know that over there there's an even more interesting planet.
[42:06] there's an even more interesting planet. We're going to go over there." And they
[42:07] We're going to go over there." And they just, I don't know,
[42:08] just, I don't know, get on a rocket or They pull themselves
[42:10] get on a rocket or They pull themselves >> Yeah. So, it's it's difficult to
[42:12] >> Yeah. So, it's it's difficult to anticipate all the ways that we might go
[42:14] anticipate all the ways that we might go extinct at the hands of the
[42:17] extinct at the hands of the uh entities much more intelligent than
[42:19] uh entities much more intelligent than ourselves.
[42:20] ourselves. Anyway, coming back to
[42:22] Anyway, coming back to the question of, "Well, if everything
[42:23] the question of, "Well, if everything goes right, right? If we we create AGI,
[42:26] goes right, right? If we we create AGI, we figure out how to make it safe,
[42:28] we figure out how to make it safe, we
[42:30] we we achieve all these economic miracles,
[42:32] we achieve all these economic miracles, then you face a problem." And this is
[42:33] then you face a problem." And this is not a new problem. Right? So, so John
[42:36] not a new problem. Right? So, so John Maynard Keynes, who was a famous
[42:37] Maynard Keynes, who was a famous economist in the early part of the 20th
[42:39] economist in the early part of the 20th century, wrote a wrote a paper in 1930.
[42:43] century, wrote a wrote a paper in 1930. So, this is in the depths of the
[42:44] So, this is in the depths of the depression. It's called on the economic
[42:46] depression. It's called on the economic problems of our grandchildren. He
[42:48] problems of our grandchildren. He predicts that at some point science will
[42:52] predicts that at some point science will will deliver sufficient wealth that no
[42:54] will deliver sufficient wealth that no one will have to work ever again. And
[42:57] one will have to work ever again. And then man will be faced with his true
[42:59] then man will be faced with his true eternal problem.
[43:02] eternal problem. How to live I don't remember the exact
[43:04] How to live I don't remember the exact word, but how to live
[43:05] word, but how to live wisely and well when the you know, the
[43:09] wisely and well when the you know, the economic incentives, the economic
[43:11] economic incentives, the economic constraints are lifted. We don't have an
[43:13] constraints are lifted. We don't have an answer to that question.
[43:15] answer to that question. Right? So, AI systems are doing
[43:17] Right? So, AI systems are doing pretty much everything we currently call
[43:19] pretty much everything we currently call work.
[43:21] work. Anything you might aspire to, like you
[43:23] Anything you might aspire to, like you want to become a surgeon.
[43:25] want to become a surgeon. It takes the robot 7 seconds to learn
[43:28] It takes the robot 7 seconds to learn how to be a surgeon that's better than
[43:29] how to be a surgeon that's better than any human being. Elon said last week
[43:32] any human being. Elon said last week that the humanoid robots will be 10
[43:34] that the humanoid robots will be 10 times better than any surgeon that's
[43:36] times better than any surgeon that's ever lived. Quite possibly, yeah.
[43:39] ever lived. Quite possibly, yeah. Well, and they'll also have you know,
[43:41] Well, and they'll also have you know, ha- they'll have hands that are, you
[43:43] ha- they'll have hands that are, you know, a millimeter in size, so they can
[43:45] know, a millimeter in size, so they can go inside and do all kinds of things
[43:47] go inside and do all kinds of things that humans can't do.
[43:48] that humans can't do. And I think we need to put serious
[43:51] And I think we need to put serious effort into this question. What is a
[43:53] effort into this question. What is a world
[43:54] world where AI can do all forms of human work
[43:58] where AI can do all forms of human work that you would want your children to
[44:00] that you would want your children to live in?
[44:02] live in? What does that world look like?
[44:04] What does that world look like? Tell me the destination
[44:06] Tell me the destination so that we can develop a transition plan
[44:08] so that we can develop a transition plan to get there. I've asked AI researchers,
[44:11] to get there. I've asked AI researchers, economists,
[44:13] economists, science fiction writers, futurists, no
[44:16] science fiction writers, futurists, no one
[44:17] one has been able to describe that world.
[44:20] has been able to describe that world. I'm not saying it's not possible. I'm
[44:21] I'm not saying it's not possible. I'm just saying I've asked hundreds of
[44:22] just saying I've asked hundreds of people
[44:24] people in multiple workshops.
[44:26] in multiple workshops. It does not, as far as I know, exist in
[44:29] It does not, as far as I know, exist in science fiction.
[44:30] science fiction. You know, it's notoriously difficult to
[44:32] You know, it's notoriously difficult to write about a utopia.
[44:34] write about a utopia. It's very hard to have a plot, right?
[44:36] It's very hard to have a plot, right? Nothing bad happens in in utopia, so
[44:38] Nothing bad happens in in utopia, so it's difficult to make a plot. So,
[44:39] it's difficult to make a plot. So, usually
[44:41] usually you start out with a utopia and then it
[44:43] you start out with a utopia and then it all falls apart and that's how that's
[44:45] all falls apart and that's how that's how you get get a plot.
[44:47] how you get get a plot. You know, there's one series of novels
[44:49] You know, there's one series of novels people point to where humans and
[44:51] people point to where humans and superintelligent AI systems coexist.
[44:54] superintelligent AI systems coexist. It's called the Culture novels
[44:56] It's called the Culture novels by Ian Banks.
[44:58] by Ian Banks. Highly recommended for those people who
[45:00] Highly recommended for those people who like science fiction.
[45:02] like science fiction. And there, absolutely, the AI systems
[45:06] And there, absolutely, the AI systems are only concerned with furthering human
[45:07] are only concerned with furthering human interests. They find humans a bit
[45:09] interests. They find humans a bit boring,
[45:10] boring, but nonetheless, they they are there to
[45:12] but nonetheless, they they are there to help.
[45:13] help. But the problem is, you know, in that
[45:15] But the problem is, you know, in that world, there's still nothing to do.
[45:18] world, there's still nothing to do. To find purpose, in fact, that you know,
[45:20] To find purpose, in fact, that you know, the
[45:21] the the subgroup of humanity that has
[45:23] the subgroup of humanity that has purpose is the subgroup whose job it is
[45:26] purpose is the subgroup whose job it is to expand the boundaries of our galactic
[45:29] to expand the boundaries of our galactic civilization. Some cases fighting wars
[45:32] civilization. Some cases fighting wars against alien species and and so on.
[45:35] against alien species and and so on. Right? So, that's the sort of cutting
[45:36] Right? So, that's the sort of cutting edge.
[45:38] edge. And that's 0.001% of the population.
[45:41] And that's 0.001% of the population. Everyone else is desperately trying to
[45:43] Everyone else is desperately trying to get into that group so they have some
[45:45] get into that group so they have some purpose in life.
[45:46] purpose in life. When I speak to very successful
[45:48] When I speak to very successful billionaires privately, off camera, off
[45:50] billionaires privately, off camera, off microphone about this, they say to me
[45:52] microphone about this, they say to me that they're investing really heavily in
[45:54] that they're investing really heavily in entertainment, things like football
[45:57] entertainment, things like football clubs,
[45:58] clubs, um because people are going to have so
[45:59] um because people are going to have so much free time that they're not going to
[46:01] much free time that they're not going to know what to do with it and they're
[46:01] know what to do with it and they're going to need things to spend it on.
[46:04] going to need things to spend it on. This is what I hear a lot. I've heard
[46:06] This is what I hear a lot. I've heard this three or four times. I've actually
[46:07] this three or four times. I've actually heard Sam Altman say a version of this.
[46:09] heard Sam Altman say a version of this. Yeah. Um about the amount of free time
[46:10] Yeah. Um about the amount of free time we're going to have. I've obviously also
[46:11] we're going to have. I've obviously also heard recently Elon talking about the
[46:14] heard recently Elon talking about the age of abundance when he delivered his
[46:16] age of abundance when he delivered his quarterly earnings just a couple of
[46:17] quarterly earnings just a couple of weeks ago. And he said that there will
[46:19] weeks ago. And he said that there will be at some point 10 billion humanoid
[46:21] be at some point 10 billion humanoid robots. His pay packet um targets him to
[46:25] robots. His pay packet um targets him to deliver one 1 million of these human
[46:27] deliver one 1 million of these human humanoid robots a year that are enabled
[46:30] humanoid robots a year that are enabled by AI by 2030.
[46:33] by AI by 2030. So, if he if he does that, he gets I
[46:34] So, if he if he does that, he gets I think it's part of his package, he gets
[46:36] think it's part of his package, he gets a trillion dollars
[46:38] a trillion dollars in compensation. Yeah. So, the age of
[46:41] in compensation. Yeah. So, the age of abundance for Elon.
[46:43] abundance for Elon. It's not that it's absolutely impossible
[46:47] It's not that it's absolutely impossible to have a worthwhile world of that
[46:50] to have a worthwhile world of that you know, with that premise.
[46:51] you know, with that premise. But I'm just waiting for someone to
[46:53] But I'm just waiting for someone to describe it. Or maybe So, let me try and
[46:55] describe it. Or maybe So, let me try and describe it.
[46:56] describe it. Uh we wake up in the morning.
[46:59] Uh we wake up in the morning. We
[47:01] We go and watch some form of human-centric
[47:04] go and watch some form of human-centric entertainment
[47:06] entertainment or participate in some form of
[47:08] or participate in some form of human-centric entertainment. Mhm. We we
[47:11] human-centric entertainment. Mhm. We we go to retreats and with each other and
[47:15] go to retreats and with each other and sit around and talk about stuff. Mhm.
[47:18] sit around and talk about stuff. Mhm. And
[47:21] And maybe people still listen to podcasts.
[47:24] maybe people still listen to podcasts. I hope I hope so for your sake. Yeah. Um
[47:27] I hope I hope so for your sake. Yeah. Um it it feels a little bit like a cruise
[47:30] it it feels a little bit like a cruise ship.
[47:33] ship. And, you know, and there are some
[47:34] And, you know, and there are some cruises where, you know, it's
[47:36] cruises where, you know, it's smart pants people and they have, you
[47:38] smart pants people and they have, you know, they have lectures in the evening
[47:40] know, they have lectures in the evening about ancient civilizations and whatnot.
[47:42] about ancient civilizations and whatnot. And some are more
[47:44] And some are more uh more popular entertainment. And this
[47:46] uh more popular entertainment. And this is in fact, if you've seen the film
[47:48] is in fact, if you've seen the film Wall-E,
[47:50] Wall-E, this is one picture of that future. In
[47:53] this is one picture of that future. In fact, in Wall-E,
[47:55] fact, in Wall-E, the human race are all living on cruise
[47:58] the human race are all living on cruise ships in space.
[47:59] ships in space. They have no constructive role in their
[48:01] They have no constructive role in their society.
[48:03] society. Right? They're just there to consume
[48:04] Right? They're just there to consume entertainment. There's no particular
[48:06] entertainment. There's no particular purpose to education. Uh you know, and
[48:08] purpose to education. Uh you know, and they're depicted actually as huge obese
[48:12] they're depicted actually as huge obese babies.
[48:13] babies. They're actually wearing onesies
[48:15] They're actually wearing onesies to emphasize the fact that they have
[48:17] to emphasize the fact that they have become enfeebled. And they become
[48:19] become enfeebled. And they become enfeebled because there's
[48:21] enfeebled because there's there's no purpose in
[48:23] there's no purpose in being able to do anything, at least in
[48:26] being able to do anything, at least in in this conception. You know, Wall-E is
[48:28] in this conception. You know, Wall-E is not the future that we want.
[48:31] not the future that we want. Do you think much about humanoid robots
[48:33] Do you think much about humanoid robots and how they're protagonists in the
[48:36] and how they're protagonists in the story of AI? It's an interesting
[48:38] story of AI? It's an interesting question, right? Why why humanoid? And
[48:42] question, right? Why why humanoid? And the one of the reasons I think is
[48:43] the one of the reasons I think is because in all the science fiction
[48:44] because in all the science fiction movies, they're humanoid. So, that's
[48:46] movies, they're humanoid. So, that's what robots are supposed to be, right?
[48:48] what robots are supposed to be, right? Cuz they were
[48:49] Cuz they were in science fiction before they became a
[48:50] in science fiction before they became a reality. Right? So, even Metropolis,
[48:52] reality. Right? So, even Metropolis, which is a film from 19
[48:54] which is a film from 19 20, I think, the robots are humanoid,
[48:57] 20, I think, the robots are humanoid, right? They're basically people covered
[48:59] right? They're basically people covered in metal. You know, from a practical
[49:01] in metal. You know, from a practical point of view, as we have discovered,
[49:03] point of view, as we have discovered, humanoid is a terrible design because
[49:06] humanoid is a terrible design because they fall over.
[49:08] they fall over. Um
[49:09] Um and uh
[49:11] and uh you know, you do want
[49:13] you know, you do want multi-fingered
[49:15] multi-fingered hands of some kind. It doesn't have to
[49:17] hands of some kind. It doesn't have to be a hand, but you want to have, you
[49:20] be a hand, but you want to have, you know, at least half a dozen appendages
[49:22] know, at least half a dozen appendages that can grasp and manipulate things.
[49:25] that can grasp and manipulate things. And you need something, you know, some
[49:27] And you need something, you know, some kind of locomotion. And wheels
[49:30] kind of locomotion. And wheels are great, except they don't go upstairs
[49:32] are great, except they don't go upstairs and over curbs and things like that. So,
[49:35] and over curbs and things like that. So, that's probably why we're going to be
[49:36] that's probably why we're going to be stuck with legs. But a four-legged
[49:39] stuck with legs. But a four-legged two-armed
[49:41] two-armed robot would be much more practical. I
[49:43] robot would be much more practical. I guess the argument I've heard is because
[49:44] guess the argument I've heard is because we've built a human world. So,
[49:46] we've built a human world. So, everything
[49:47] everything the physical spaces we navigate, whether
[49:49] the physical spaces we navigate, whether it's factories or our homes or the
[49:52] it's factories or our homes or the street or other sort of public spaces
[49:56] street or other sort of public spaces are all designed for exactly this
[49:59] are all designed for exactly this physical form. So, if we are going to
[50:01] physical form. So, if we are going to >> To some extent, yeah. But I mean, our
[50:02] >> To some extent, yeah. But I mean, our dogs manage perfectly well to navigate
[50:05] dogs manage perfectly well to navigate around our houses and streets and so on.
[50:08] around our houses and streets and so on. So, if you had a a centaur,
[50:11] So, if you had a a centaur, uh it could also navigate, but it can
[50:13] uh it could also navigate, but it can you know, it can carry much greater
[50:16] you know, it can carry much greater loads because it's quadruped. It's much
[50:18] loads because it's quadruped. It's much more stable. If it needs to drive a car,
[50:21] more stable. If it needs to drive a car, it can fold up two of its legs and and
[50:23] it can fold up two of its legs and and so on and so forth. So, I think the
[50:24] so on and so forth. So, I think the arguments for why it has to be exactly
[50:26] arguments for why it has to be exactly humanoid are sort of post hoc
[50:29] humanoid are sort of post hoc justification.
[50:30] justification. I think there's much more well, that's
[50:32] I think there's much more well, that's what it's like in the movies and that's
[50:34] what it's like in the movies and that's spooky.
[50:35] spooky. Cool. So, we need to have them be
[50:37] Cool. So, we need to have them be humanoid. I I don't think it's
[50:39] humanoid. I I don't think it's a good engineering argument. I think
[50:41] a good engineering argument. I think there's also probably an argument that
[50:42] there's also probably an argument that we would be more accepting of them
[50:46] we would be more accepting of them moving through our physical environments
[50:48] moving through our physical environments if they represented our form
[50:51] if they represented our form a bit more.
[50:52] a bit more. Um I also I was thinking of a bloody
[50:54] Um I also I was thinking of a bloody baby gate. You know, there's like
[50:55] baby gate. You know, there's like kindergarten gates they get on stairs.
[50:56] kindergarten gates they get on stairs. >> Yeah. My dog can't open that. But a
[50:59] >> Yeah. My dog can't open that. But a humanoid robot could reach over the
[51:01] humanoid robot could reach over the other side.
[51:02] other side. Yeah, and so could a centaur robot,
[51:03] Yeah, and so could a centaur robot, right? So, in some sense, centaur robot
[51:06] right? So, in some sense, centaur robot is
[51:07] is There's something ghastly about the look
[51:08] There's something ghastly about the look of those though.
[51:09] of those though. >> is the humanoid. Well, Do you know what
[51:10] >> is the humanoid. Well, Do you know what I mean? Like a four-legged big monster
[51:12] I mean? Like a four-legged big monster sort of crawling through my house when I
[51:13] sort of crawling through my house when I have guests over.
[51:15] have guests over. I mean, your dog is a four-legged Your
[51:16] I mean, your dog is a four-legged Your dog is a four-legged monster. So,
[51:19] dog is a four-legged monster. So, so I think actually
[51:21] so I think actually I I would argue the opposite that
[51:23] I I would argue the opposite that um
[51:25] um we want a distinct form because they are
[51:27] we want a distinct form because they are distinct entities.
[51:31] distinct entities. And the more humanoid the worse
[51:35] And the more humanoid the worse it is in terms of confusing our
[51:38] it is in terms of confusing our subconscious psychological systems. So,
[51:41] subconscious psychological systems. So, I'm arguing from the perspective of the
[51:42] I'm arguing from the perspective of the people making them.
[51:44] people making them. As in if I was making the decision
[51:45] As in if I was making the decision whether it to be some four-legged thing
[51:47] whether it to be some four-legged thing that I've that I'm unfamiliar with that
[51:48] that I've that I'm unfamiliar with that I'm less likely to
[51:50] I'm less likely to build a relationship with
[51:52] build a relationship with or allow to take care of I don't know.
[51:56] or allow to take care of I don't know. Might might look after my children.
[51:58] Might might look after my children. Obviously, I'm Listen, I'm not saying I
[51:59] Obviously, I'm Listen, I'm not saying I would allow this to look after my
[52:00] would allow this to look after my children. But I'm saying from a if I'm
[52:02] children. But I'm saying from a if I'm building a company, so
[52:03] building a company, so >> But the manufacturer would certainly
[52:04] >> But the manufacturer would certainly Yeah. wouldn't want to be.
[52:05] Yeah. wouldn't want to be. >> Yeah, so I that's an interesting
[52:07] >> Yeah, so I that's an interesting question.
[52:08] question. I mean, there's also what's called the
[52:10] I mean, there's also what's called the uncanny valley, which is a
[52:13] uncanny valley, which is a a phrase from computer graphics when
[52:15] a phrase from computer graphics when they started
[52:17] they started to make characters in computer graphics,
[52:21] to make characters in computer graphics, they tried to make them look more human.
[52:23] they tried to make them look more human. Right? So, if you if you for example, if
[52:25] Right? So, if you if you for example, if you look at Toy Story,
[52:27] you look at Toy Story, they're not very human-looking, right?
[52:29] they're not very human-looking, right? If you look at The Incredibles, they're
[52:31] If you look at The Incredibles, they're not very human-looking. And so, we think
[52:33] not very human-looking. And so, we think of them as cartoon characters. If you
[52:34] of them as cartoon characters. If you try to make them more human,
[52:37] try to make them more human, they actually become repulsive. Until
[52:39] they actually become repulsive. Until they don't. Until they become very You
[52:41] they don't. Until they become very You have to be very very close
[52:43] have to be very very close to perfect
[52:45] to perfect in order not to be repulsive. So, the
[52:47] in order not to be repulsive. So, the the uncanny valley is this I you know,
[52:49] the uncanny valley is this I you know, like the
[52:50] like the the gap between you know, so perfectly
[52:51] the gap between you know, so perfectly human and not at all human, but in
[52:53] human and not at all human, but in between it's really awful.
[52:55] between it's really awful. And uh and so they there were a couple
[52:57] And uh and so they there were a couple of movies that tried, like Polar Express
[53:00] of movies that tried, like Polar Express was one,
[53:01] was one, where they tried to have quite
[53:03] where they tried to have quite human-looking characters, you know,
[53:05] human-looking characters, you know, being humans, not not being superheroes
[53:07] being humans, not not being superheroes or anything else. And it's repulsive to
[53:09] or anything else. And it's repulsive to watch.
[53:10] watch. I when I watched that shareholder
[53:12] I when I watched that shareholder presentation the other day,
[53:13] presentation the other day, Elon had these two humanoid robots
[53:15] Elon had these two humanoid robots dancing on stage. And I've seen lots of
[53:17] dancing on stage. And I've seen lots of humanoid robot demonstrations over the
[53:18] humanoid robot demonstrations over the years. You know, you've seen like the
[53:19] years. You know, you've seen like the Boston Dynamics dog thing jumping around
[53:22] Boston Dynamics dog thing jumping around and whatever else.
[53:23] and whatever else. But there was a moment where my brain,
[53:26] But there was a moment where my brain, for the first time ever,
[53:28] for the first time ever, genuinely thought there was a human in a
[53:29] genuinely thought there was a human in a suit.
[53:30] suit. Mhm. And I actually had to research to
[53:32] Mhm. And I actually had to research to check if that was really their Optimus
[53:34] check if that was really their Optimus robot because the way it was dancing was
[53:37] robot because the way it was dancing was so unbelievably fluid that for the first
[53:39] so unbelievably fluid that for the first time ever, my my my brain has only ever
[53:43] time ever, my my my brain has only ever associated those movements with human
[53:44] associated those movements with human movements.
[53:46] movements. And I'll I'll play it on the screen if
[53:47] And I'll I'll play it on the screen if anyone hasn't seen it, but it's just a
[53:49] anyone hasn't seen it, but it's just a robot dancing on stage and I was like,
[53:51] robot dancing on stage and I was like, that is a human in a suit. And it was
[53:52] that is a human in a suit. And it was really the knees that gave it away
[53:54] really the knees that gave it away because the knees were all metal. And I
[53:56] because the knees were all metal. And I thought there's no way that could be a
[53:57] thought there's no way that could be a human knee in a in a one of those suits.
[54:00] human knee in a in a one of those suits. And he you know, he says they're going
[54:01] And he you know, he says they're going into production next year. They used
[54:03] into production next year. They used internally at Tesla now, but he says
[54:05] internally at Tesla now, but he says they're going into production next year
[54:06] they're going into production next year and
[54:07] and it's going to be pretty crazy when we
[54:08] it's going to be pretty crazy when we walk outside and see robots. I think
[54:09] walk outside and see robots. I think that'll be the paradigm shift. I've
[54:10] that'll be the paradigm shift. I've heard actually many I've heard Elon say
[54:12] heard actually many I've heard Elon say this that
[54:13] this that the paradigm shifting moment for many of
[54:15] the paradigm shifting moment for many of us will be when we walk outside onto the
[54:17] us will be when we walk outside onto the streets and see humanoid robots walking
[54:19] streets and see humanoid robots walking around.
[54:21] around. That will be when we realize. Yeah, I
[54:22] That will be when we realize. Yeah, I think even more so. I mean, in San
[54:24] think even more so. I mean, in San Francisco, we see driverless cars
[54:26] Francisco, we see driverless cars driving around.
[54:27] driving around. And uh it take takes some getting used
[54:29] And uh it take takes some getting used to.
[54:30] to. Actually, you know, when you're
[54:32] Actually, you know, when you're you're driving and there's a car right
[54:33] you're driving and there's a car right next to you with no driver in, you know,
[54:35] next to you with no driver in, you know, and it's signaling and it wants to
[54:36] and it's signaling and it wants to change lanes in front of you and you
[54:38] change lanes in front of you and you have to let it in and all this kind of
[54:40] have to let it in and all this kind of stuff. It's it's a little creepy, but I
[54:42] stuff. It's it's a little creepy, but I think you're right. I think
[54:43] think you're right. I think seeing the humanoid robots. But that
[54:46] seeing the humanoid robots. But that phenomenon that you described where
[54:48] phenomenon that you described where it was sufficiently close that your
[54:50] it was sufficiently close that your brain flipped into saying, this is a
[54:54] brain flipped into saying, this is a human being. Mhm. Right? That's exactly
[54:57] human being. Mhm. Right? That's exactly what I think we should avoid.
[54:59] what I think we should avoid. Cuz I have the empathy for it then.
[55:00] Cuz I have the empathy for it then. >> Because it's it's a lie.
[55:03] >> Because it's it's a lie. And it brings with it a whole lot of
[55:05] And it brings with it a whole lot of expectations about how it's going to
[55:07] expectations about how it's going to behave, what moral rights it has, how
[55:09] behave, what moral rights it has, how you should behave towards it,
[55:11] you should behave towards it, uh which are completely wrong. It levels
[55:14] uh which are completely wrong. It levels the playing field between me and it to
[55:16] the playing field between me and it to some degree. How hard is it going to be
[55:19] some degree. How hard is it going to be to just uh you know, switch it off and
[55:21] to just uh you know, switch it off and throw it in the trash when when it
[55:23] throw it in the trash when when it breaks? I think it's essential for us to
[55:26] breaks? I think it's essential for us to keep machines in the you know, in the
[55:27] keep machines in the you know, in the cognitive space where they are
[55:29] cognitive space where they are machines and not bring them into the
[55:32] machines and not bring them into the cognitive space where they're people.
[55:34] cognitive space where they're people. Because we will make enormous mistakes
[55:38] Because we will make enormous mistakes by doing that. And I see this every day
[55:39] by doing that. And I see this every day even even just with the chatbots.
[55:42] even even just with the chatbots. So, the chatbots in theory are supposed
[55:44] So, the chatbots in theory are supposed to say,
[55:46] to say, I don't have any feelings. I'm just a
[55:48] I don't have any feelings. I'm just a algorithm.
[55:50] algorithm. But in fact, they fail to do that all
[55:53] But in fact, they fail to do that all the time.
[55:54] the time. They are telling people that they are
[55:56] They are telling people that they are conscious. They are telling people that
[55:57] conscious. They are telling people that they have feelings.
[55:59] they have feelings. Uh they are telling people that they're
[56:00] Uh they are telling people that they're in love with
[56:02] in love with the user that they're talking to.
[56:04] the user that they're talking to. And people flip because first of all,
[56:07] And people flip because first of all, it's you know, very fluent language, but
[56:09] it's you know, very fluent language, but also a system that is identifying itself
[56:12] also a system that is identifying itself as an I, as a sentient being, they
[56:15] as an I, as a sentient being, they bring that object into the cognitive
[56:18] bring that object into the cognitive space where that we normally reserve for
[56:21] space where that we normally reserve for for other humans. And they become
[56:23] for other humans. And they become emotionally attached. They become
[56:24] emotionally attached. They become psychologically dependent.
[56:27] psychologically dependent. They even allow these systems to tell
[56:30] They even allow these systems to tell them what to do. What advice would you
[56:32] them what to do. What advice would you give a young person at the start of
[56:33] give a young person at the start of their career then about what they should
[56:35] their career then about what they should be aiming at professionally? Cuz I've
[56:36] be aiming at professionally? Cuz I've actually had an increasing number of
[56:37] actually had an increasing number of young people say to me
[56:39] young people say to me that they have huge uncertainty about
[56:41] that they have huge uncertainty about whether the thing they're studying now
[56:42] whether the thing they're studying now will matter at all. A lawyer,
[56:44] will matter at all. A lawyer, uh an accountant. And I don't know what
[56:47] uh an accountant. And I don't know what to say to these people. I don't know
[56:47] to say to these people. I don't know what to say cuz I I believe that the
[56:49] what to say cuz I I believe that the rate of improvement in in AI is going to
[56:51] rate of improvement in in AI is going to continue and therefore imagining any
[56:53] continue and therefore imagining any rate of improvement it gets to the point
[56:54] rate of improvement it gets to the point where
[56:56] where I'm not being funny, but all these
[56:57] I'm not being funny, but all these white-collar jobs will be done by an AI
[56:59] white-collar jobs will be done by an AI an AI or an AI agent.
[57:01] an AI or an AI agent. Yeah. So, there was a television series
[57:03] Yeah. So, there was a television series called Humans. In Humans,
[57:06] called Humans. In Humans, we have extremely capable
[57:09] we have extremely capable humanoid robots
[57:11] humanoid robots doing everything. And at one point,
[57:13] doing everything. And at one point, the parents are talking to their teenage
[57:14] the parents are talking to their teenage daughter who's very very smart. And the
[57:17] daughter who's very very smart. And the parents are saying, "Oh, you know,
[57:19] parents are saying, "Oh, you know, maybe you should go into medicine."
[57:21] maybe you should go into medicine." And the daughter says, you know, "Why
[57:23] And the daughter says, you know, "Why would I bother? It'll take me 7 years
[57:26] would I bother? It'll take me 7 years to qualify and takes the robot 7 seconds
[57:28] to qualify and takes the robot 7 seconds to learn.
[57:30] to learn. So, nothing I do matters."
[57:32] So, nothing I do matters." And is that how you feel about So, I
[57:34] And is that how you feel about So, I think that's that's a future that
[57:37] think that's that's a future that uh in fact, that is the future that we
[57:39] uh in fact, that is the future that we are
[57:40] are moving towards.
[57:42] moving towards. I don't think it's the future that
[57:43] I don't think it's the future that everyone wants.
[57:44] everyone wants. That is what is being uh
[57:47] That is what is being uh created for us
[57:49] created for us right now.
[57:51] right now. So, in that future, assuming that
[57:53] So, in that future, assuming that you know, even if we get halfway,
[57:56] you know, even if we get halfway, right? In the sense that, okay, perhaps
[57:58] right? In the sense that, okay, perhaps not surgeons, perhaps not,
[58:01] not surgeons, perhaps not, you know, great violinists, there'll be
[58:03] you know, great violinists, there'll be pockets
[58:04] pockets where perhaps humans will remain good at
[58:07] where perhaps humans will remain good at it. Where?
[58:09] it. Where? The kinds of jobs where you hire people
[58:11] The kinds of jobs where you hire people by the hundred
[58:13] by the hundred will go away.
[58:15] will go away. Okay. Where people are in some sense
[58:17] Okay. Where people are in some sense exchangeable that you you you just need
[58:19] exchangeable that you you you just need lots of them
[58:20] lots of them and uh you know, when when half of them
[58:22] and uh you know, when when half of them quit, you just fill up those
[58:24] quit, you just fill up those those slots with more people. In some
[58:26] those slots with more people. In some sense, those are jobs where we're using
[58:27] sense, those are jobs where we're using people as robots. And that's the sort of
[58:29] people as robots. And that's the sort of that's the sort of strange conundrum
[58:31] that's the sort of strange conundrum here, right? That you know, I imagine
[58:33] here, right? That you know, I imagine writing science fiction 10,000 years
[58:34] writing science fiction 10,000 years ago, right? When we're all
[58:35] ago, right? When we're all hunter-gatherers,
[58:37] hunter-gatherers, and I'm this little science fiction
[58:38] and I'm this little science fiction author and I'm describing this future
[58:40] author and I'm describing this future where, you know, there are going to be
[58:42] where, you know, there are going to be these giant windowless boxes and you're
[58:44] these giant windowless boxes and you're going to go in,
[58:46] going to go in, you know, you'll you'll travel for miles
[58:48] you know, you'll you'll travel for miles and you'll go into this windowless box
[58:49] and you'll go into this windowless box and you'll do the same thing 10,000
[58:52] and you'll do the same thing 10,000 times for the whole day and then you'll
[58:54] times for the whole day and then you'll leave and travel for miles to go home.
[58:56] leave and travel for miles to go home. You talking about this podcast?
[58:57] You talking about this podcast? >> And then you're going to go back and do
[58:58] >> And then you're going to go back and do it again. And you would do that every
[59:00] it again. And you would do that every day of your life until you die.
[59:03] day of your life until you die. The office. And people would say, "Are
[59:05] The office. And people would say, "Are you nuts, right? There's no way that we
[59:08] you nuts, right? There's no way that we humans are ever going to have a future
[59:09] humans are ever going to have a future like that cuz that's awful."
[59:11] like that cuz that's awful." Right? But that's exactly the future
[59:12] Right? But that's exactly the future that we ended up with with with office
[59:14] that we ended up with with with office buildings and factories where many of us
[59:17] buildings and factories where many of us uh go and do the same thing
[59:19] uh go and do the same thing thousands of times a day and we do it
[59:21] thousands of times a day and we do it thousands of days in a row
[59:23] thousands of days in a row uh and then we die. And we need to
[59:26] uh and then we die. And we need to figure out what is the next phase going
[59:28] figure out what is the next phase going to be like. And in particular, how in
[59:31] to be like. And in particular, how in that world
[59:33] that world do we have the incentives
[59:35] do we have the incentives to become fully human?
[59:38] to become fully human? Which I think means at least a level of
[59:40] Which I think means at least a level of education
[59:41] education that people have now, and probably more.
[59:45] that people have now, and probably more. Because I think to live a really rich
[59:47] Because I think to live a really rich life
[59:48] life you need a better understanding
[59:51] you need a better understanding of yourself, of the world
[59:54] of yourself, of the world uh than most people get in their current
[59:56] uh than most people get in their current educations. What is it to be human? To
[59:58] educations. What is it to be human? To It's to reproduce,
[01:00:01] It's to reproduce, to pursue stuff,
[01:00:03] to pursue stuff, to go in the pursuit of difficult
[01:00:05] to go in the pursuit of difficult things.
[01:00:06] things. You know, we used to hunt on the Mhm. To
[01:00:08] You know, we used to hunt on the Mhm. To attain goals, right? It's always If I
[01:00:10] attain goals, right? It's always If I wanted to climb Everest, the last thing
[01:00:12] wanted to climb Everest, the last thing I would want is someone to pick me up an
[01:00:14] I would want is someone to pick me up an helicopter and stick me on the top.
[01:00:16] helicopter and stick me on the top. So, we'll we'll voluntarily pursue
[01:00:19] So, we'll we'll voluntarily pursue hard things. So, although I could get
[01:00:21] hard things. So, although I could get the robot to build me
[01:00:23] the robot to build me a ranch in
[01:00:26] a ranch in on this plot of land, I will choose to
[01:00:28] on this plot of land, I will choose to do it because the pursuit of self is
[01:00:30] do it because the pursuit of self is rewarding.
[01:00:32] rewarding. Yes.
[01:00:32] Yes. >> We're kind of seeing that anyway, aren't
[01:00:33] >> We're kind of seeing that anyway, aren't we? Don't you think we're seeing a bit
[01:00:34] we? Don't you think we're seeing a bit of that in society where life got so
[01:00:36] of that in society where life got so comfortable that now people are like
[01:00:37] comfortable that now people are like obsessed with running marathons and
[01:00:39] obsessed with running marathons and doing these crazy endurance
[01:00:40] doing these crazy endurance >> And and and learning to cook complicated
[01:00:42] >> And and and learning to cook complicated things when they could just, you know,
[01:00:44] things when they could just, you know, have them delivered.
[01:00:45] have them delivered. Um yeah, no, I think there's there's
[01:00:47] Um yeah, no, I think there's there's real value in
[01:00:49] real value in the ability to do things and the doing
[01:00:51] the ability to do things and the doing of those things.
[01:00:52] of those things. And I think, you know, the the obvious
[01:00:53] And I think, you know, the the obvious danger is the Wall-E world where
[01:00:56] danger is the Wall-E world where everyone just consumes entertainment.
[01:00:59] everyone just consumes entertainment. Uh which doesn't require much education
[01:01:01] Uh which doesn't require much education and
[01:01:03] and doesn't lead to a rich, satisfying life,
[01:01:06] doesn't lead to a rich, satisfying life, I think in the long run. A lot of people
[01:01:08] I think in the long run. A lot of people will choose that world. I think some of
[01:01:10] will choose that world. I think some of Yeah, some people may. There's also, I
[01:01:12] Yeah, some people may. There's also, I mean,
[01:01:13] mean, you know, whether you're consuming
[01:01:14] you know, whether you're consuming entertainment or whether you're
[01:01:17] entertainment or whether you're doing something, you know, cooking or
[01:01:19] doing something, you know, cooking or painting, whatever, because it's fun and
[01:01:21] painting, whatever, because it's fun and interesting to do.
[01:01:22] interesting to do. What's missing from that, right? All of
[01:01:24] What's missing from that, right? All of that is purely selfish.
[01:01:27] that is purely selfish. I think one of the reasons we work is
[01:01:30] I think one of the reasons we work is because we feel valued, we feel like
[01:01:33] because we feel valued, we feel like we're benefiting other people.
[01:01:36] we're benefiting other people. And I think some of
[01:01:38] And I think some of I remember having this conversation with
[01:01:40] I remember having this conversation with um
[01:01:41] um a lady in England who helps to run the
[01:01:43] a lady in England who helps to run the hospice movement.
[01:01:45] hospice movement. And the people who work in the hospices
[01:01:48] And the people who work in the hospices where, you know, the the patients are
[01:01:50] where, you know, the the patients are literally there to die,
[01:01:52] literally there to die, are largely volunteers, so they're not
[01:01:54] are largely volunteers, so they're not doing it to get paid.
[01:01:56] doing it to get paid. But they find it incredibly
[01:01:59] But they find it incredibly rewarding to be able to spend time with
[01:02:02] rewarding to be able to spend time with people who are in their last weeks or
[01:02:05] people who are in their last weeks or months to give them company and
[01:02:07] months to give them company and happiness.
[01:02:09] happiness. So, I actually think that
[01:02:11] So, I actually think that interpersonal
[01:02:14] interpersonal roles
[01:02:15] roles will be much, much more important in
[01:02:18] will be much, much more important in future.
[01:02:19] future. So, if I was
[01:02:21] So, if I was going to
[01:02:22] going to advise my kids, not that they would ever
[01:02:24] advise my kids, not that they would ever listen, but if I if my kids would
[01:02:26] listen, but if I if my kids would listen, then I and and wanted to know
[01:02:28] listen, then I and and wanted to know what I thought would be
[01:02:30] what I thought would be you know, valued careers in future, I
[01:02:32] you know, valued careers in future, I think it would be
[01:02:33] think it would be these interpersonal roles based on an
[01:02:36] these interpersonal roles based on an understanding of human needs,
[01:02:37] understanding of human needs, psychology. There are some of those
[01:02:39] psychology. There are some of those roles right now.
[01:02:41] roles right now. So, obviously, you know, therapists and
[01:02:43] So, obviously, you know, therapists and psychiatrists and so on.
[01:02:45] psychiatrists and so on. But that that's a very much and it's
[01:02:47] But that that's a very much and it's sort of asymmetric
[01:02:49] sort of asymmetric role, right? Where
[01:02:51] role, right? Where one person is suffering and the other
[01:02:52] one person is suffering and the other person is trying to
[01:02:54] person is trying to alleviate the suffering.
[01:02:56] alleviate the suffering. You know, and then there are things like
[01:02:58] You know, and then there are things like they call them executive coaches or life
[01:02:59] they call them executive coaches or life coaches.
[01:03:01] coaches. Right? That's a a less asymmetric role,
[01:03:04] Right? That's a a less asymmetric role, where someone is trying to uh help
[01:03:08] where someone is trying to uh help another person live a better life,
[01:03:10] another person live a better life, whether it's a better life in their work
[01:03:12] whether it's a better life in their work role or or just
[01:03:14] role or or just uh how they live their life in general.
[01:03:15] uh how they live their life in general. And
[01:03:16] And so, I could imagine that those kinds of
[01:03:19] so, I could imagine that those kinds of roles will expand dramatically.
[01:03:22] roles will expand dramatically. This is interesting paradox that exists
[01:03:25] This is interesting paradox that exists when life becomes easier,
[01:03:27] when life becomes easier, um which shows that abundance
[01:03:29] um which shows that abundance consistently pushes society societies
[01:03:32] consistently pushes society societies towards more individualism. Because once
[01:03:35] towards more individualism. Because once survival pressures disappear, people
[01:03:37] survival pressures disappear, people prioritize things
[01:03:38] prioritize things differently. They prioritize freedom,
[01:03:40] differently. They prioritize freedom, comfort, self-expression over things
[01:03:42] comfort, self-expression over things like sacrifice or um family formation.
[01:03:45] like sacrifice or um family formation. And we're seeing, I think, in the West
[01:03:47] And we're seeing, I think, in the West already a decline in people having kids
[01:03:49] already a decline in people having kids because there's more material abundance.
[01:03:53] because there's more material abundance. Fewer kids, people are getting married
[01:03:55] Fewer kids, people are getting married and committing to each other and having
[01:03:57] and committing to each other and having relationships later
[01:03:59] relationships later and more infrequently. Mhm. Because
[01:04:01] and more infrequently. Mhm. Because generally, once we have more abundance,
[01:04:03] generally, once we have more abundance, we don't want to complicate our lives.
[01:04:04] we don't want to complicate our lives. Um and at the same time, as you said
[01:04:06] Um and at the same time, as you said earlier, that abundance breeds
[01:04:09] earlier, that abundance breeds a an inability to find meaning, a sort
[01:04:12] a an inability to find meaning, a sort of shallowness to everything. This is
[01:04:14] of shallowness to everything. This is one of the things I think a lot about,
[01:04:15] one of the things I think a lot about, and I'm I'm in the process now of
[01:04:16] and I'm I'm in the process now of writing a book about it, which is this
[01:04:18] writing a book about it, which is this idea about individualism was act- is a
[01:04:21] idea about individualism was act- is a bit of a lie. Like, when I say
[01:04:22] bit of a lie. Like, when I say individualism and freedom, I mean like
[01:04:24] individualism and freedom, I mean like the narrative at the moment amongst my
[01:04:25] the narrative at the moment amongst my generation is, you like, be your own
[01:04:27] generation is, you like, be your own boss and stand on your own two feet and
[01:04:29] boss and stand on your own two feet and having less kids and we're not getting
[01:04:31] having less kids and we're not getting married and it's all about me, me, me,
[01:04:33] married and it's all about me, me, me, me, me, me, me. Yeah, that last part is
[01:04:35] me, me, me, me. Yeah, that last part is where it goes wrong. Yeah, and it's like
[01:04:37] where it goes wrong. Yeah, and it's like almost a narcissistic society where
[01:04:39] almost a narcissistic society where Yeah. me, me, me, me, me, myself
[01:04:40] Yeah. me, me, me, me, me, myself interest first. And when you look at
[01:04:42] interest first. And when you look at mental health outcomes and loneliness
[01:04:44] mental health outcomes and loneliness and all these kinds of things, it's
[01:04:46] and all these kinds of things, it's going in a horrific direction, but at
[01:04:48] going in a horrific direction, but at the same time, we're freer than ever.
[01:04:50] the same time, we're freer than ever. It seems like that, you know, it seems
[01:04:51] It seems like that, you know, it seems like there's a we should there's a maybe
[01:04:53] like there's a we should there's a maybe another story about dependency, which is
[01:04:55] another story about dependency, which is not sexy. Like, depend on each other.
[01:04:57] not sexy. Like, depend on each other. Oh, I I I agree. I mean, I think
[01:04:59] Oh, I I I agree. I mean, I think you know, happiness is not
[01:05:02] you know, happiness is not available from consumption
[01:05:05] available from consumption or even lifestyle, right? I think
[01:05:06] or even lifestyle, right? I think happiness is
[01:05:08] happiness is arises from giving.
[01:05:12] arises from giving. It can be you
[01:05:14] It can be you through the work that you do,
[01:05:16] through the work that you do, you can see that other people benefit
[01:05:17] you can see that other people benefit from that. Or it could be in
[01:05:19] from that. Or it could be in direct interpersonal relationships.
[01:05:22] direct interpersonal relationships. There is an invisible tax on sales
[01:05:24] There is an invisible tax on sales people that no one really talks about
[01:05:25] people that no one really talks about enough. The mental load of remembering
[01:05:27] enough. The mental load of remembering everything, like meeting notes,
[01:05:29] everything, like meeting notes, timelines, and everything in between.
[01:05:31] timelines, and everything in between. Until we started using Ousmane's product
[01:05:33] Until we started using Ousmane's product called Pipedrive, one of the best CRM
[01:05:35] called Pipedrive, one of the best CRM tools for small and medium-sized
[01:05:36] tools for small and medium-sized business owners. The idea here was that
[01:05:39] business owners. The idea here was that it might alleviate some of the
[01:05:40] it might alleviate some of the unnecessary cognitive overload that my
[01:05:42] unnecessary cognitive overload that my team was carrying, so that they could
[01:05:44] team was carrying, so that they could spend less time in the weeds of admin
[01:05:46] spend less time in the weeds of admin and more time with clients, in-person
[01:05:48] and more time with clients, in-person meetings, and building relationships.
[01:05:49] meetings, and building relationships. Pipedrive has enabled this to happen.
[01:05:51] Pipedrive has enabled this to happen. It's such a simple but effective CRM
[01:05:54] It's such a simple but effective CRM that automates the tedious, repetitive,
[01:05:56] that automates the tedious, repetitive, and time-consuming parts of the sales
[01:05:58] and time-consuming parts of the sales process. And now our team can nurture
[01:06:00] process. And now our team can nurture those leads and still have bandwidth to
[01:06:02] those leads and still have bandwidth to focus on the higher priority tasks that
[01:06:04] focus on the higher priority tasks that actually get the deal over the line.
[01:06:06] actually get the deal over the line. Over 100,000 companies across 170
[01:06:09] Over 100,000 companies across 170 countries already use Pipedrive to grow
[01:06:11] countries already use Pipedrive to grow their business, and I've been using it
[01:06:12] their business, and I've been using it for almost a decade now. Try it free for
[01:06:15] for almost a decade now. Try it free for 30 days. No credit card needed, no
[01:06:17] 30 days. No credit card needed, no payment needed. Just use my link
[01:06:19] payment needed. Just use my link pipedrive.com/ceo
[01:06:22] pipedrive.com/ceo to get started today. That's
[01:06:23] to get started today. That's pipedrive.com/ceo.
[01:06:27] Where does the
[01:06:28] Where does the rewards of this AI race What does it
[01:06:31] rewards of this AI race What does it What does it accrue to?
[01:06:34] What does it accrue to? I think a lot about this in terms of
[01:06:35] I think a lot about this in terms of like universe universal basic income. If
[01:06:37] like universe universal basic income. If you have these five, six, seven, 10
[01:06:40] you have these five, six, seven, 10 massive AI companies that are going to
[01:06:42] massive AI companies that are going to win the 15 quadrillion dollar prize,
[01:06:46] win the 15 quadrillion dollar prize, Mhm. and they're going to automate all
[01:06:47] Mhm. and they're going to automate all of the professional
[01:06:49] of the professional pursuits that we we currently have, all
[01:06:51] pursuits that we we currently have, all of our jobs are going to go away,
[01:06:54] of our jobs are going to go away, who who gets all the money and how do
[01:06:56] who who gets all the money and how do how do we get some of it back? Money
[01:06:58] how do we get some of it back? Money actually doesn't matter, right? What
[01:06:59] actually doesn't matter, right? What what matters is
[01:07:01] what matters is the production of goods and services,
[01:07:04] the production of goods and services, uh and then how those are distributed.
[01:07:07] uh and then how those are distributed. And so so money acts as a way to
[01:07:09] And so so money acts as a way to facilitate the distribution and um
[01:07:12] facilitate the distribution and um exchange of those goods and services. If
[01:07:14] exchange of those goods and services. If all production is concentrated
[01:07:17] all production is concentrated um
[01:07:18] um in the hands of a
[01:07:19] in the hands of a of a few companies, right? That
[01:07:22] of a few companies, right? That Sure, they will lease some of their
[01:07:25] Sure, they will lease some of their robots to us.
[01:07:26] robots to us. You know, we we want a school in our
[01:07:28] You know, we we want a school in our village.
[01:07:29] village. They lease the robots to us, the robots
[01:07:31] They lease the robots to us, the robots build the school, they go away. We have
[01:07:33] build the school, they go away. We have to pay a certain amount of of money for
[01:07:35] to pay a certain amount of of money for that. But where do we get the money?
[01:07:39] that. But where do we get the money? Right? If we are not producing anything,
[01:07:43] Right? If we are not producing anything, then uh
[01:07:44] then uh we don't have any money unless there's
[01:07:46] we don't have any money unless there's some redistribution mechanism.
[01:07:48] some redistribution mechanism. And as you mentioned, so universal basic
[01:07:50] And as you mentioned, so universal basic income is
[01:07:53] it seems to me
[01:07:55] it seems to me an admission of failure.
[01:07:57] an admission of failure. Because what it says is, okay, we're
[01:07:58] Because what it says is, okay, we're just going to give everyone the money,
[01:08:00] just going to give everyone the money, and then they can use the money to
[01:08:01] and then they can use the money to pay the AI company to lease the robots
[01:08:04] pay the AI company to lease the robots to build the school.
[01:08:05] to build the school. And then we'll have a school, and that's
[01:08:07] And then we'll have a school, and that's good. Um
[01:08:09] good. Um But what is it an admission of failure
[01:08:12] But what is it an admission of failure because it says we can't work out a
[01:08:14] because it says we can't work out a system
[01:08:15] system in which people have any worth
[01:08:18] in which people have any worth or any economic role.
[01:08:21] or any economic role. Right? So, 99% of the global population
[01:08:24] Right? So, 99% of the global population is
[01:08:25] is from an economic point of view, useless.
[01:08:28] from an economic point of view, useless. Can I ask you a question? If you had a
[01:08:30] Can I ask you a question? If you had a button in front of you,
[01:08:32] button in front of you, and pressing that button would stop all
[01:08:35] and pressing that button would stop all progress in artificial intelligence
[01:08:37] progress in artificial intelligence right now and forever,
[01:08:39] right now and forever, would you press it? That's a very
[01:08:40] would you press it? That's a very interesting question.
[01:08:42] interesting question. Um
[01:08:45] If it's either or,
[01:08:48] If it's either or, either I do it now or it's too late,
[01:08:51] either I do it now or it's too late, and we
[01:08:53] and we careen into some
[01:08:55] careen into some uncontrollable future, perhaps, yeah.
[01:08:58] uncontrollable future, perhaps, yeah. Cuz I I'm not super optimistic that
[01:09:01] Cuz I I'm not super optimistic that we're heading in the right direction at
[01:09:03] we're heading in the right direction at all. So, put that button in front of you
[01:09:04] all. So, put that button in front of you now. It stops all AI progress, shuts
[01:09:06] now. It stops all AI progress, shuts down all the AI companies immediately
[01:09:08] down all the AI companies immediately globally, and none of them can reopen.
[01:09:10] globally, and none of them can reopen. You press it.
[01:09:17] Well, here's what here's what I think
[01:09:19] Well, here's what here's what I think should happen.
[01:09:20] should happen. So, obviously, you know, I've been doing
[01:09:22] So, obviously, you know, I've been doing AI for 50 years. Um
[01:09:25] AI for 50 years. Um and
[01:09:27] and the original motivations, which is that
[01:09:30] the original motivations, which is that AI can be a power tool
[01:09:32] AI can be a power tool for humanity, enabling us to do
[01:09:35] for humanity, enabling us to do more and better things than we can
[01:09:38] more and better things than we can unaided.
[01:09:39] unaided. I think that's still valid.
[01:09:41] I think that's still valid. The problem is
[01:09:43] The problem is the kinds of AI systems that we're
[01:09:45] the kinds of AI systems that we're building are not tools.
[01:09:47] building are not tools. They are replacements. In fact,
[01:09:49] They are replacements. In fact, you can see this very clearly because we
[01:09:51] you can see this very clearly because we create them
[01:09:52] create them literally as the closest replicas we can
[01:09:57] literally as the closest replicas we can make of human beings.
[01:10:00] make of human beings. The technique
[01:10:02] The technique for creating them is called imitation
[01:10:05] for creating them is called imitation learning.
[01:10:06] learning. So, we observe human verbal behavior,
[01:10:08] So, we observe human verbal behavior, writing or speaking,
[01:10:10] writing or speaking, and we make a system
[01:10:12] and we make a system that imitates that
[01:10:14] that imitates that as well as possible.
[01:10:17] as well as possible. So, what we are making is imitation
[01:10:18] So, what we are making is imitation humans, at least in the verbal sphere.
[01:10:23] humans, at least in the verbal sphere. And so, of course they're going to
[01:10:24] And so, of course they're going to replace us.
[01:10:27] They're not tools.
[01:10:28] They're not tools. So, you would press the button. So, I
[01:10:30] So, you would press the button. So, I say I think there is another course,
[01:10:34] say I think there is another course, which is use and develop AI
[01:10:37] which is use and develop AI as tools, tools for science,
[01:10:41] as tools, tools for science, tools for economic organization, and so
[01:10:43] tools for economic organization, and so on.
[01:10:44] on. Um
[01:10:46] Um but not as
[01:10:47] but not as replacements for human beings. What I
[01:10:49] replacements for human beings. What I like about this question is it forces
[01:10:51] like about this question is it forces you to go into the prob- into
[01:10:53] you to go into the prob- into probabilities. Yeah, so it and that's
[01:10:56] probabilities. Yeah, so it and that's that's why I'm reluctant
[01:10:58] that's why I'm reluctant because I don't
[01:11:00] because I don't I don't agree with the, you know, what's
[01:11:01] I don't agree with the, you know, what's your probability of doom. All right,
[01:11:03] your probability of doom. All right, your so-called P of doom
[01:11:06] your so-called P of doom uh number because that makes sense if
[01:11:08] uh number because that makes sense if you're an alien.
[01:11:10] you're an alien. You know, you're in you're in a bar with
[01:11:12] You know, you're in you're in a bar with some other aliens and you're looking
[01:11:13] some other aliens and you're looking down at the Earth and you're taking bets
[01:11:15] down at the Earth and you're taking bets on you know, are these humans going to
[01:11:16] on you know, are these humans going to make a mess of
[01:11:17] make a mess of things and go extinct because they
[01:11:19] things and go extinct because they develop AI.
[01:11:21] develop AI. So, it's fine for those aliens to bet on
[01:11:24] So, it's fine for those aliens to bet on on that. But if you're a human, then
[01:11:27] on that. But if you're a human, then you're not just betting, you're actually
[01:11:29] you're not just betting, you're actually acting.
[01:11:30] acting. There there's an element to this though,
[01:11:31] There there's an element to this though, which I guess where probabilities do
[01:11:33] which I guess where probabilities do come back in, which is you also have to
[01:11:35] come back in, which is you also have to weigh when I give you such a binary
[01:11:37] weigh when I give you such a binary decision,
[01:11:39] decision, um
[01:11:41] um the probability of us pursuing the more
[01:11:43] the probability of us pursuing the more nuanced, safe approach into that
[01:11:46] nuanced, safe approach into that equation. So, you're you're the the math
[01:11:49] equation. So, you're you're the the math in my head is okay, you've got all the
[01:11:50] in my head is okay, you've got all the upsides here, and then you've got
[01:11:52] upsides here, and then you've got potential downsides.
[01:11:53] potential downsides. And then there's a probability of do I
[01:11:55] And then there's a probability of do I think we're actually going to course
[01:11:56] think we're actually going to course correct based on everything I know,
[01:11:58] correct based on everything I know, based on the incentive structure of
[01:11:59] based on the incentive structure of human beings and and countries. And then
[01:12:02] human beings and and countries. And then if there's if but then you could go if
[01:12:03] if there's if but then you could go if there's even a 1%
[01:12:06] there's even a 1% chance of extinction,
[01:12:08] chance of extinction, is it even worth all these upsides?
[01:12:10] is it even worth all these upsides? Yeah, and I I would argue
[01:12:13] Yeah, and I I would argue no. I mean, maybe maybe what we would
[01:12:14] no. I mean, maybe maybe what we would say I if if we said, okay, it's going to
[01:12:17] say I if if we said, okay, it's going to stop the progress for 50 years. You
[01:12:19] stop the progress for 50 years. You press it. And during those 50 years, we
[01:12:22] press it. And during those 50 years, we can work on
[01:12:23] can work on how do we do AI in a way that's
[01:12:25] how do we do AI in a way that's guaranteed to be
[01:12:26] guaranteed to be safe and beneficial.
[01:12:28] safe and beneficial. How do we organize
[01:12:30] How do we organize our societies to flourish
[01:12:33] our societies to flourish uh in conjunction with extremely capable
[01:12:35] uh in conjunction with extremely capable AI systems. So, we haven't answered
[01:12:37] AI systems. So, we haven't answered either of those questions.
[01:12:39] either of those questions. And
[01:12:40] And I don't think we want anything
[01:12:42] I don't think we want anything resembling AGI until we have completely
[01:12:45] resembling AGI until we have completely solid answers to both of those
[01:12:46] solid answers to both of those questions. So,
[01:12:47] questions. So, if there was a button where I could say,
[01:12:49] if there was a button where I could say, all right, we're going to pause progress
[01:12:51] all right, we're going to pause progress for 50 years,
[01:12:52] for 50 years, yes, I would do it. But if that button
[01:12:53] yes, I would do it. But if that button was in front of you, you you going to
[01:12:55] was in front of you, you you going to make a decision either way. Either you
[01:12:56] make a decision either way. Either you don't press it or you press it. I know,
[01:12:58] don't press it or you press it. I know, if yeah, so if that if that button is
[01:12:59] if yeah, so if that if that button is there,
[01:13:00] there, stop it 50 years, I would say yes.
[01:13:05] Stop it forever,
[01:13:09] not yet. I think
[01:13:12] not yet. I think I think there's still a decent chance
[01:13:13] I think there's still a decent chance that we can pull out
[01:13:16] that we can pull out of this uh nose dive, so to speak, that
[01:13:18] of this uh nose dive, so to speak, that we're
[01:13:19] we're we're currently in. Ask me again in a
[01:13:21] we're currently in. Ask me again in a year, I might I might say, okay, we do
[01:13:24] year, I might I might say, okay, we do need to press the button. What if what
[01:13:25] need to press the button. What if what if in a scenario where you never get to
[01:13:27] if in a scenario where you never get to reverse that decision? You never get to
[01:13:29] reverse that decision? You never get to make that decision again. So, if in that
[01:13:31] make that decision again. So, if in that scenario that I've laid out, this
[01:13:32] scenario that I've laid out, this hypothetical, you either press it now or
[01:13:34] hypothetical, you either press it now or it never gets pressed.
[01:13:37] it never gets pressed. So, there is no opportunity a year from
[01:13:38] So, there is no opportunity a year from now.
[01:13:40] Yeah, as you can tell, I'm
[01:13:42] Yeah, as you can tell, I'm Yes.
[01:13:43] Yes. sort of on on the fence a bit about
[01:13:46] sort of on on the fence a bit about about this one.
[01:13:49] Yeah, I think I'd probably press it.
[01:13:52] Yeah, I think I'd probably press it. Yeah.
[01:13:55] What's your reasoning?
[01:13:57] What's your reasoning? Uh
[01:13:58] Uh just thinking about the power dynamics
[01:14:02] just thinking about the power dynamics of um
[01:14:04] of um what what's happening now.
[01:14:06] what what's happening now. How difficult would it would be to get
[01:14:08] How difficult would it would be to get the US in particular to
[01:14:10] the US in particular to to regulate
[01:14:12] to regulate in favor of safety.
[01:14:14] in favor of safety. So, I think, you know, what's clear from
[01:14:15] So, I think, you know, what's clear from talking to the companies is
[01:14:18] talking to the companies is they are not going
[01:14:20] they are not going to
[01:14:21] to develop anything resembling safe AGI
[01:14:24] develop anything resembling safe AGI unless they're forced to by the
[01:14:25] unless they're forced to by the government.
[01:14:27] government. And at the moment,
[01:14:29] And at the moment, the US government in particular, which
[01:14:31] the US government in particular, which regulates most of the leading companies
[01:14:33] regulates most of the leading companies in AI,
[01:14:34] in AI, is not only refusing to regulate,
[01:14:38] is not only refusing to regulate, but even trying to prevent the states
[01:14:40] but even trying to prevent the states from regulating.
[01:14:42] from regulating. And they're doing that
[01:14:43] And they're doing that at the behest of
[01:14:46] at the behest of uh
[01:14:47] uh a faction within Silicon Valley
[01:14:49] a faction within Silicon Valley uh called the accelerationists,
[01:14:52] uh called the accelerationists, who believe that the faster we get to
[01:14:55] who believe that the faster we get to AGI, the better. And when I say best, I
[01:14:58] AGI, the better. And when I say best, I mean also they paid them a large amount
[01:15:00] mean also they paid them a large amount of money.
[01:15:01] of money. Jensen Huang, the the CEO of Nvidia,
[01:15:03] Jensen Huang, the the CEO of Nvidia, said, who's for anyone that doesn't
[01:15:04] said, who's for anyone that doesn't know,
[01:15:05] know, the guy making all the chips that are
[01:15:06] the guy making all the chips that are powering AI, said China is going to win
[01:15:09] powering AI, said China is going to win the AI race, arguing it is just a
[01:15:11] the AI race, arguing it is just a nanosecond behind the United States.
[01:15:14] nanosecond behind the United States. China have produced 24,000 AI papers
[01:15:17] China have produced 24,000 AI papers compared to just 6,000
[01:15:21] compared to just 6,000 from the US.
[01:15:23] from the US. More than the combined output of the US,
[01:15:25] More than the combined output of the US, the UK, and the EU.
[01:15:27] the UK, and the EU. China is anticipated to quickly roll out
[01:15:29] China is anticipated to quickly roll out their new technologies both domestically
[01:15:30] their new technologies both domestically and developing
[01:15:32] and developing new technologies for other developing
[01:15:34] new technologies for other developing countries.
[01:15:36] countries. So, the accelerators, or the accelerate
[01:15:38] So, the accelerators, or the accelerate I think you call them the accelerants?
[01:15:39] I think you call them the accelerants? Ac- accelerationists.
[01:15:41] Ac- accelerationists. >> The accelerationists. Yeah. I mean, they
[01:15:42] >> The accelerationists. Yeah. I mean, they would say, well, if we don't, then China
[01:15:44] would say, well, if we don't, then China will. So, we have to. We have to go
[01:15:46] will. So, we have to. We have to go fast. It's another version of the the
[01:15:48] fast. It's another version of the the race that the companies are in with each
[01:15:50] race that the companies are in with each other, right? That we you know, we know
[01:15:52] other, right? That we you know, we know that this race is
[01:15:54] that this race is heading off a cliff,
[01:15:57] heading off a cliff, but we can't stop.
[01:15:59] but we can't stop. So, we're all just going to go off this
[01:16:00] So, we're all just going to go off this cliff.
[01:16:01] cliff. And obviously, that's nuts.
[01:16:04] And obviously, that's nuts. Right? I mean, we're all looking at each
[01:16:05] Right? I mean, we're all looking at each other saying, yeah, there's a cliff over
[01:16:06] other saying, yeah, there's a cliff over there
[01:16:07] there running as fast as we can towards this
[01:16:08] running as fast as we can towards this cliff.
[01:16:09] cliff. We're looking at each other saying, why
[01:16:10] We're looking at each other saying, why aren't we stopping?
[01:16:13] aren't we stopping? So, the narrative in Washington, which I
[01:16:16] So, the narrative in Washington, which I think Jensen Huang is
[01:16:19] think Jensen Huang is either reflecting or or perhaps um
[01:16:21] either reflecting or or perhaps um promoting,
[01:16:23] promoting, uh
[01:16:24] uh is that
[01:16:26] is that you know, China has is completely
[01:16:28] you know, China has is completely unregulated.
[01:16:30] unregulated. And uh you know, America will only slow
[01:16:32] And uh you know, America will only slow itself down
[01:16:34] itself down uh if it regulates AI in any way.
[01:16:37] uh if it regulates AI in any way. So, this is a completely false narrative
[01:16:38] So, this is a completely false narrative because
[01:16:39] because China's
[01:16:41] China's AI regulations are actually quite
[01:16:42] AI regulations are actually quite strict,
[01:16:43] strict, even compared to um European Union.
[01:16:48] even compared to um European Union. And China's government has explicitly
[01:16:50] And China's government has explicitly acknowledged uh the need and their
[01:16:54] acknowledged uh the need and their regulations are very clear, you can't
[01:16:56] regulations are very clear, you can't build AI systems that could escape human
[01:16:58] build AI systems that could escape human control. And not only that, I don't
[01:17:00] control. And not only that, I don't think they view
[01:17:03] think they view the race in the same way as, okay, we
[01:17:06] the race in the same way as, okay, we we just need to be the first to create
[01:17:08] we just need to be the first to create AGI.
[01:17:09] AGI. I think they're
[01:17:11] I think they're more interested in figuring out how to
[01:17:14] more interested in figuring out how to disseminate AI
[01:17:17] disseminate AI as a set of tools within their economy
[01:17:19] as a set of tools within their economy to make their economy more productive
[01:17:21] to make their economy more productive and
[01:17:22] and and so on. So, that's that's their
[01:17:23] and so on. So, that's that's their version of the race. But of course, they
[01:17:25] version of the race. But of course, they still want to build the weapons for
[01:17:26] still want to build the weapons for adversaries, right? To
[01:17:28] adversaries, right? To so that they can take down,
[01:17:31] so that they can take down, I don't know, Taiwan if they want to.
[01:17:33] I don't know, Taiwan if they want to. So, weapons are a separate matter, and
[01:17:35] So, weapons are a separate matter, and I'm happy to talk about weapons. But
[01:17:37] I'm happy to talk about weapons. But just in terms of
[01:17:38] just in terms of control. Uh control, economic
[01:17:40] control. Uh control, economic domination,
[01:17:42] domination, um
[01:17:43] um they they don't view
[01:17:46] they they don't view putting all your eggs in the AGI basket
[01:17:47] putting all your eggs in the AGI basket as the right strategy, so they want to
[01:17:51] as the right strategy, so they want to use AI,
[01:17:53] use AI, you know, even in its present form,
[01:17:55] you know, even in its present form, to make their economy much more
[01:17:57] to make their economy much more efficient and productive. And also, you
[01:17:59] efficient and productive. And also, you know,
[01:18:00] know, to give
[01:18:01] to give people new capabilities and and better
[01:18:04] people new capabilities and and better quality of life.
[01:18:06] quality of life. And and I think the US could do that as
[01:18:08] And and I think the US could do that as well. And
[01:18:11] well. And um
[01:18:12] um typically, Western countries don't have
[01:18:15] typically, Western countries don't have as much of uh central government control
[01:18:17] as much of uh central government control over
[01:18:18] over what in AI to make their
[01:18:20] what in AI to make their operations more efficient.
[01:18:25] Uh and some are not, and we'll see how
[01:18:27] Uh and some are not, and we'll see how that plays out. What do you think of
[01:18:28] that plays out. What do you think of Trump's approach to AI? So, Trump's
[01:18:30] Trump's approach to AI? So, Trump's approach is you know, it's it's echoing
[01:18:32] approach is you know, it's it's echoing what Jensen Huang is saying, that the US
[01:18:34] what Jensen Huang is saying, that the US has to be the one to create AGI.
[01:18:37] has to be the one to create AGI. And very explicitly, the
[01:18:39] And very explicitly, the administration's policy is
[01:18:41] administration's policy is to uh dominate the world.
[01:18:44] to uh dominate the world. That's the word they use, dominate. I'm
[01:18:46] That's the word they use, dominate. I'm not sure that other countries like the
[01:18:48] not sure that other countries like the idea that um they will be dominated by
[01:18:52] idea that um they will be dominated by American AI.
[01:18:54] American AI. But is that an accurate description of
[01:18:55] But is that an accurate description of what will happen if the US
[01:18:57] what will happen if the US build AGI technology before, say, the
[01:18:59] build AGI technology before, say, the UK, where I'm originally from and where
[01:19:01] UK, where I'm originally from and where you're originally from?
[01:19:03] you're originally from? What does the I This is something I
[01:19:04] What does the I This is something I think about a lot, cuz we're going
[01:19:05] think about a lot, cuz we're going through this budget process in the UK at
[01:19:07] through this budget process in the UK at the moment, where we're figuring out how
[01:19:08] the moment, where we're figuring out how we're going to spend our money and how
[01:19:09] we're going to spend our money and how we're going to tax people.
[01:19:10] we're going to tax people. And also, we've got this new election
[01:19:11] And also, we've got this new election cycle is
[01:19:13] cycle is approaching quickly, where people are
[01:19:14] approaching quickly, where people are talking about immigration issues and
[01:19:16] talking about immigration issues and this issue and that issue and the other
[01:19:18] this issue and that issue and the other issue. What I don't hear anyone talking
[01:19:19] issue. What I don't hear anyone talking about is
[01:19:20] about is AI and the humanoid robots that
[01:19:23] AI and the humanoid robots that are going to take everything. We're very
[01:19:24] are going to take everything. We're very concerned with the brown people crossing
[01:19:25] concerned with the brown people crossing the channel, but the humanoid robots
[01:19:27] the channel, but the humanoid robots that are going to be super intelligent
[01:19:29] that are going to be super intelligent and really take causing economic disrupt
[01:19:31] and really take causing economic disrupt disruption, no one talks about that. The
[01:19:33] disruption, no one talks about that. The political leaders don't talk about it.
[01:19:35] political leaders don't talk about it. It doesn't win races. I don't see it on
[01:19:36] It doesn't win races. I don't see it on billboards. Yeah, and it's it it's
[01:19:39] billboards. Yeah, and it's it it's interesting because
[01:19:41] interesting because in fact, I mean I so there's there's two
[01:19:43] in fact, I mean I so there's there's two forces that have been hollowing out the
[01:19:45] forces that have been hollowing out the middle classes in Western countries.
[01:19:48] middle classes in Western countries. One of them is globalization
[01:19:50] One of them is globalization where lots and lots of work, not just
[01:19:53] where lots and lots of work, not just manufacturing, but white collar work
[01:19:54] manufacturing, but white collar work gets outsourced to low-income countries.
[01:19:57] gets outsourced to low-income countries. Uh but the other is automation.
[01:20:01] Uh but the other is automation. And, you know, some of that is
[01:20:03] And, you know, some of that is factories, so
[01:20:04] factories, so um
[01:20:06] um the amount of employment in
[01:20:07] the amount of employment in manufacturing
[01:20:08] manufacturing continues to drop even as the amount of
[01:20:10] continues to drop even as the amount of output
[01:20:11] output from manufacturing in the US and in the
[01:20:14] from manufacturing in the US and in the UK continues to increase. So, we talk
[01:20:16] UK continues to increase. So, we talk about, oh, you know, our manufacturing
[01:20:18] about, oh, you know, our manufacturing industry has been destroyed. It hasn't.
[01:20:20] industry has been destroyed. It hasn't. It's producing more than ever just with
[01:20:23] It's producing more than ever just with you know, a quarter as many people.
[01:20:25] you know, a quarter as many people. So, it's manufacturing employment that's
[01:20:27] So, it's manufacturing employment that's been destroyed by automation and
[01:20:30] been destroyed by automation and robotics and so on. And then, you know,
[01:20:32] robotics and so on. And then, you know, computerization
[01:20:34] computerization has eliminated whole layers of white
[01:20:37] has eliminated whole layers of white collar jobs.
[01:20:39] collar jobs. And so, those two
[01:20:41] And so, those two those two forms of automation
[01:20:43] those two forms of automation have probably done more to hollow out
[01:20:45] have probably done more to hollow out middle class
[01:20:46] middle class uh employment and standard of life. If
[01:20:50] uh employment and standard of life. If the UK doesn't participate
[01:20:52] the UK doesn't participate in this new
[01:20:54] in this new technological wave
[01:20:56] technological wave that seems to be that seems to have you
[01:20:58] that seems to be that seems to have you know, it's going to take a lot of jobs.
[01:21:00] know, it's going to take a lot of jobs. Cars are going to drive themselves.
[01:21:01] Cars are going to drive themselves. Waymo just announced that they're coming
[01:21:03] Waymo just announced that they're coming to London, which is the driverless cars.
[01:21:05] to London, which is the driverless cars. And driving is the biggest occupation in
[01:21:06] And driving is the biggest occupation in the world, for example. So, you've got
[01:21:08] the world, for example. So, you've got immediate disruption there. And where
[01:21:09] immediate disruption there. And where does the money accrue to it? Well, it
[01:21:10] does the money accrue to it? Well, it accrues to who owns Waymo, which is what
[01:21:13] accrues to who owns Waymo, which is what Google and Silicon Valley companies.
[01:21:16] Google and Silicon Valley companies. >> Alphabet owns Waymo 100%, I think. So,
[01:21:18] >> Alphabet owns Waymo 100%, I think. So, yes. I mean, this is So, I was in India
[01:21:20] yes. I mean, this is So, I was in India a few months ago
[01:21:22] a few months ago talking to the government ministers cuz
[01:21:23] talking to the government ministers cuz they're holding the next global AI
[01:21:26] they're holding the next global AI summit in February.
[01:21:28] summit in February. And and their view going in was, you
[01:21:31] And and their view going in was, you know, AI is great. We're going to use it
[01:21:33] know, AI is great. We're going to use it to, you know, turbocharge the growth of
[01:21:35] to, you know, turbocharge the growth of our Indian economy.
[01:21:38] our Indian economy. When, for example, you have AGI, you
[01:21:41] When, for example, you have AGI, you have AGI-controlled robots
[01:21:44] have AGI-controlled robots that can do all the manufacturing, that
[01:21:45] that can do all the manufacturing, that can do agriculture, that can do all the
[01:21:48] can do agriculture, that can do all the white collar work.
[01:21:49] white collar work. And goods and services that might have
[01:21:51] And goods and services that might have been produced by Indians
[01:21:54] been produced by Indians will instead be produced by
[01:21:57] will instead be produced by American-controlled
[01:22:00] AGI systems
[01:22:02] AGI systems at much lower prices.
[01:22:04] at much lower prices. You know, a consumer given a choice
[01:22:05] You know, a consumer given a choice between an expensive product produced by
[01:22:07] between an expensive product produced by Indians or a cheap product produced by
[01:22:10] Indians or a cheap product produced by American robots
[01:22:11] American robots will probably choose
[01:22:14] will probably choose the cheap product produced by American
[01:22:15] the cheap product produced by American robots. And so, potentially every
[01:22:17] robots. And so, potentially every country in the world, with the possible
[01:22:19] country in the world, with the possible exception of North Korea,
[01:22:21] exception of North Korea, will become a kind of a client state
[01:22:25] will become a kind of a client state of American AI companies. A client state
[01:22:28] of American AI companies. A client state of American AI companies is exactly what
[01:22:31] of American AI companies is exactly what I'm concerned about for the UK economy
[01:22:33] I'm concerned about for the UK economy and really any economy outside of the
[01:22:35] and really any economy outside of the United States. I guess one could also
[01:22:36] United States. I guess one could also say China, but
[01:22:38] say China, but cuz they those are the two nations that
[01:22:40] cuz they those are the two nations that are taking AI most seriously.
[01:22:43] are taking AI most seriously. And I I I don't know what our economy
[01:22:45] And I I I don't know what our economy becomes. I can't figure out
[01:22:48] becomes. I can't figure out can't figure out what our what the
[01:22:49] can't figure out what our what the British economy becomes in such a world.
[01:22:52] British economy becomes in such a world. Is it tourism? I don't know. Like you
[01:22:53] Is it tourism? I don't know. Like you come here to to to look at the
[01:22:55] come here to to to look at the Buckingham Palace? I I You you can think
[01:22:57] Buckingham Palace? I I You you can think about countries, but I mean, even for
[01:22:59] about countries, but I mean, even for the United States, it's the same
[01:23:00] the United States, it's the same problem. At least they'll be able to tax
[01:23:02] problem. At least they'll be able to tax the hell out
[01:23:03] the hell out >> You know, so some small fraction of the
[01:23:05] >> You know, so some small fraction of the population
[01:23:06] population will be running, maybe, the AI
[01:23:09] will be running, maybe, the AI companies.
[01:23:10] companies. But increasingly,
[01:23:12] But increasingly, even those companies will be replacing
[01:23:14] even those companies will be replacing their human employees with AI systems.
[01:23:18] their human employees with AI systems. So, Amazon, for example, which
[01:23:20] So, Amazon, for example, which you know, sells a lot of computing
[01:23:21] you know, sells a lot of computing services to AI companies is using AI to
[01:23:24] services to AI companies is using AI to replace layers of management. Is
[01:23:26] replace layers of management. Is planning to use robots to replace all of
[01:23:28] planning to use robots to replace all of its warehouse workers.
[01:23:30] its warehouse workers. And so on. So, so even
[01:23:33] And so on. So, so even the the giant AI companies
[01:23:36] the the giant AI companies will have few human employees.
[01:23:39] will have few human employees. In the long run, I mean, it
[01:23:41] In the long run, I mean, it think of the situation, you know, pity
[01:23:43] think of the situation, you know, pity the poor CEO whose board
[01:23:46] the poor CEO whose board says, well, you know, unless you turn
[01:23:49] says, well, you know, unless you turn over your decision-making power to the
[01:23:50] over your decision-making power to the AI system,
[01:23:52] AI system, um we're going to have to fire you
[01:23:54] um we're going to have to fire you because
[01:23:55] because all our competitors are using,
[01:23:57] all our competitors are using, you know, an AI-powered CEO and they're
[01:24:00] you know, an AI-powered CEO and they're doing much better. Amazon plans to
[01:24:01] doing much better. Amazon plans to replace 600,000 workers with robots in a
[01:24:04] replace 600,000 workers with robots in a memo that just leaked, which has been
[01:24:06] memo that just leaked, which has been widely talked about. And the CEO, Andy
[01:24:08] widely talked about. And the CEO, Andy Jassy, told employees that the company
[01:24:10] Jassy, told employees that the company expects its corporate workforce to
[01:24:12] expects its corporate workforce to shrink in the coming years
[01:24:14] shrink in the coming years because of AI and AI agents. And they've
[01:24:16] because of AI and AI agents. And they've publicly gone live with saying that
[01:24:18] publicly gone live with saying that they're going to cut 14,000 corporate
[01:24:20] they're going to cut 14,000 corporate jobs in the near term
[01:24:22] jobs in the near term as part of its refocus on AI investment
[01:24:25] as part of its refocus on AI investment and efficiency.
[01:24:28] and efficiency. It's interesting cuz I was reading about
[01:24:30] It's interesting cuz I was reading about um the sort of different quotes from
[01:24:32] um the sort of different quotes from different AI leaders about the speed in
[01:24:33] different AI leaders about the speed in which this this stuff is going to
[01:24:35] which this this stuff is going to happen.
[01:24:36] happen. And what you see in the quotes is Demis,
[01:24:39] And what you see in the quotes is Demis, who's the CEO of DeepMind, saying things
[01:24:41] who's the CEO of DeepMind, saying things like it'll be
[01:24:43] like it'll be more than 10 times bigger than the
[01:24:45] more than 10 times bigger than the Industrial Revolution, but also it'll
[01:24:47] Industrial Revolution, but also it'll happen maybe 10 times faster. And they
[01:24:49] happen maybe 10 times faster. And they speak about this turbulence that we're
[01:24:52] speak about this turbulence that we're going to experience as this shift takes
[01:24:53] going to experience as this shift takes place.
[01:24:55] place. That's um maybe a euphemism
[01:24:58] That's um maybe a euphemism for
[01:24:59] for uh and I think that, you know,
[01:25:00] uh and I think that, you know, governments are now
[01:25:02] governments are now you know, they they've kind of gone from
[01:25:04] you know, they they've kind of gone from saying, oh, don't worry, you know, we'll
[01:25:05] saying, oh, don't worry, you know, we'll just retrain everyone as data
[01:25:06] just retrain everyone as data scientists. And like, well, yeah, that's
[01:25:08] scientists. And like, well, yeah, that's that's ridiculous, right? The world
[01:25:09] that's ridiculous, right? The world doesn't need 4 billion data scientists.
[01:25:11] doesn't need 4 billion data scientists. And we're not all capable of becoming
[01:25:13] And we're not all capable of becoming that, by the way. Uh yeah, or have any
[01:25:15] that, by the way. Uh yeah, or have any interest in in doing that.
[01:25:17] interest in in doing that. >> couldn't I couldn't even do it if I
[01:25:18] >> couldn't I couldn't even do it if I wanted to. Like I tried to sit in
[01:25:19] wanted to. Like I tried to sit in biology class and I fell asleep, so
[01:25:21] biology class and I fell asleep, so I couldn't That was the end of my career
[01:25:23] I couldn't That was the end of my career as a surgeon. Fair enough. Um
[01:25:26] as a surgeon. Fair enough. Um but yeah, now suddenly they're staring,
[01:25:28] but yeah, now suddenly they're staring, you know,
[01:25:29] you know, 80% unemployment in the face and
[01:25:31] 80% unemployment in the face and wondering how
[01:25:33] wondering how how on earth is our society going to
[01:25:35] how on earth is our society going to hold together? We'll deal with it when
[01:25:36] hold together? We'll deal with it when we get there.
[01:25:38] we get there. Yeah, unfortunately, um
[01:25:41] Yeah, unfortunately, um unless we plan ahead,
[01:25:44] unless we plan ahead, we're going to suffer the consequences,
[01:25:46] we're going to suffer the consequences, right? We can't It was bad enough in the
[01:25:48] right? We can't It was bad enough in the Industrial Revolution, which unfolded
[01:25:50] Industrial Revolution, which unfolded over seven or eight decades,
[01:25:52] over seven or eight decades, but there was massive
[01:25:54] but there was massive disruption
[01:25:56] disruption and misery
[01:25:59] and misery caused by that.
[01:26:00] caused by that. We don't have a model for a functioning
[01:26:02] We don't have a model for a functioning society where
[01:26:04] society where almost everyone does nothing.
[01:26:08] almost everyone does nothing. At least nothing of economic value.
[01:26:11] At least nothing of economic value. Now, it's not impossible that there
[01:26:13] Now, it's not impossible that there could be such a a functioning society,
[01:26:15] could be such a a functioning society, but we don't know what it looks like.
[01:26:17] but we don't know what it looks like. And, you know, when you think about our
[01:26:19] And, you know, when you think about our education system,
[01:26:21] education system, which would probably have to look very
[01:26:22] which would probably have to look very different,
[01:26:24] different, and how long it takes to change that. I
[01:26:25] and how long it takes to change that. I mean, I'm always
[01:26:27] mean, I'm always uh reminding people about uh
[01:26:30] uh reminding people about uh how long it took Oxford to decide that
[01:26:33] how long it took Oxford to decide that geography was a proper subject of study.
[01:26:35] geography was a proper subject of study. Took them 125 years
[01:26:38] Took them 125 years from the first proposal that there
[01:26:39] from the first proposal that there should be a geography degree until it
[01:26:41] should be a geography degree until it was finally approved.
[01:26:43] was finally approved. So, we don't have
[01:26:44] So, we don't have very long
[01:26:47] very long to completely revamp a system
[01:26:50] to completely revamp a system that we know takes decades and decades
[01:26:54] that we know takes decades and decades to reform.
[01:26:56] to reform. And we don't know
[01:26:57] And we don't know how to reform it because we don't know
[01:27:00] how to reform it because we don't know what we want the world to look like. Is
[01:27:03] what we want the world to look like. Is this one of your reasons why you're
[01:27:06] this one of your reasons why you're appalled at the moment? Because when you
[01:27:08] appalled at the moment? Because when you have these conversations with people,
[01:27:09] have these conversations with people, people just don't have answers, yet
[01:27:11] people just don't have answers, yet they're plowing ahead at rapid speed. I
[01:27:13] they're plowing ahead at rapid speed. I would say
[01:27:15] would say it's not necessarily the job of the AI
[01:27:17] it's not necessarily the job of the AI companies. So, I'm appalled by the AI
[01:27:18] companies. So, I'm appalled by the AI companies cuz they don't have answers
[01:27:20] companies cuz they don't have answers for how they're going to control the
[01:27:21] for how they're going to control the systems that they're proposing to build.
[01:27:24] systems that they're proposing to build. I do find it
[01:27:26] I do find it disappointing that
[01:27:28] disappointing that uh governments don't seem to be
[01:27:29] uh governments don't seem to be grappling with this issue. I think there
[01:27:32] grappling with this issue. I think there are a few. I think, for example,
[01:27:34] are a few. I think, for example, Singapore government seems to be quite
[01:27:35] Singapore government seems to be quite far-sighted.
[01:27:37] far-sighted. And they've
[01:27:38] And they've they've thought this through. You know,
[01:27:40] they've thought this through. You know, it's a small country. They've figured
[01:27:41] it's a small country. They've figured out, okay, this is this will be our role
[01:27:43] out, okay, this is this will be our role uh going forward and we think we can
[01:27:45] uh going forward and we think we can find, you know, some some purpose for
[01:27:48] find, you know, some some purpose for our people in this in this new world.
[01:27:50] our people in this in this new world. But for I think countries with large
[01:27:52] But for I think countries with large populations,
[01:27:53] populations, um
[01:27:56] they need to figure out answers to these
[01:27:59] they need to figure out answers to these questions pretty fast. It takes a long
[01:28:00] questions pretty fast. It takes a long time to actually implement those answers
[01:28:03] time to actually implement those answers uh in the form of new kinds of
[01:28:06] uh in the form of new kinds of education, new professions,
[01:28:08] education, new professions, new qualifications,
[01:28:10] new qualifications, uh new economic structures.
[01:28:13] uh new economic structures. I mean, it's it's it's possible. I mean,
[01:28:16] I mean, it's it's it's possible. I mean, when you look at therapists, for
[01:28:17] when you look at therapists, for example, they're almost all
[01:28:19] example, they're almost all self-employed.
[01:28:22] self-employed. So, what happens when, you know, 80% of
[01:28:25] So, what happens when, you know, 80% of the population
[01:28:26] the population transitions from regular employment into
[01:28:29] transitions from regular employment into into self-employment?
[01:28:31] into self-employment? What does that What does that do to the
[01:28:32] What does that What does that do to the economics of
[01:28:34] economics of of government finances and so on?
[01:28:37] of government finances and so on? So, there's just lots of questions. And
[01:28:39] So, there's just lots of questions. And how do you you know, if that's the
[01:28:40] how do you you know, if that's the future, you know, why are we training
[01:28:42] future, you know, why are we training people
[01:28:43] people to to fit into 9-to-5 office jobs, which
[01:28:46] to to fit into 9-to-5 office jobs, which won't exist at all?
[01:28:48] won't exist at all? Last month, I told you about a challenge
[01:28:50] Last month, I told you about a challenge that I set our internal Flight X team.
[01:28:52] that I set our internal Flight X team. Flight X team is our innovation team
[01:28:53] Flight X team is our innovation team internally here. I tasked them with
[01:28:55] internally here. I tasked them with seeing how much time they could unlock
[01:28:57] seeing how much time they could unlock for the company by creating something
[01:28:59] for the company by creating something that would help us filter new AI tools
[01:29:01] that would help us filter new AI tools to see which ones were worth pursuing.
[01:29:03] to see which ones were worth pursuing. And I thought that our sponsor Fiverr
[01:29:05] And I thought that our sponsor Fiverr Pro might have the talent on their
[01:29:07] Pro might have the talent on their platform to help us build this quickly.
[01:29:09] platform to help us build this quickly. So, I talked to my director of
[01:29:10] So, I talked to my director of innovation, Isaac, and for the last
[01:29:12] innovation, Isaac, and for the last month, my team Flight X and a vetted AI
[01:29:14] month, my team Flight X and a vetted AI specialist from Fiverr Pro have been
[01:29:16] specialist from Fiverr Pro have been working together on this project. And
[01:29:19] working together on this project. And with the help of my team, we've been
[01:29:20] with the help of my team, we've been able to create a brand new tool which
[01:29:22] able to create a brand new tool which automatically scans, scores, and
[01:29:24] automatically scans, scores, and prioritizes different emerging AI tools
[01:29:26] prioritizes different emerging AI tools for us. Its impact has been huge. And
[01:29:29] for us. Its impact has been huge. And within a couple of weeks, this tool has
[01:29:30] within a couple of weeks, this tool has already been saving us hours trialing
[01:29:32] already been saving us hours trialing and testing new AI systems. Instead of
[01:29:34] and testing new AI systems. Instead of sifting through lots of noise, my team
[01:29:36] sifting through lots of noise, my team Flight X has been able to focus on
[01:29:38] Flight X has been able to focus on developing even more AI tools, ones that
[01:29:40] developing even more AI tools, ones that really move the needle in our business,
[01:29:42] really move the needle in our business, thanks to the talent on Fiverr Pro. So,
[01:29:44] thanks to the talent on Fiverr Pro. So, if you've got a complex problem and you
[01:29:47] if you've got a complex problem and you need help solving it, make sure you
[01:29:48] need help solving it, make sure you check out Fiverr Pro at
[01:29:50] check out Fiverr Pro at fiverr.com/diary.
[01:29:53] fiverr.com/diary. So, many of us are pursuing passive
[01:29:55] So, many of us are pursuing passive forms of income and to build side
[01:29:57] forms of income and to build side businesses in order to help us cover our
[01:29:58] businesses in order to help us cover our bills. And that opportunity is here with
[01:30:01] bills. And that opportunity is here with our sponsor Stan, a business that I
[01:30:03] our sponsor Stan, a business that I co-own. It is the platform that can help
[01:30:05] co-own. It is the platform that can help you take full advantage of your own
[01:30:08] you take full advantage of your own financial situation. Stan enables you to
[01:30:10] financial situation. Stan enables you to work for yourself. It makes selling
[01:30:12] work for yourself. It makes selling digital products, courses, memberships,
[01:30:14] digital products, courses, memberships, and more simple products, more scalable,
[01:30:16] and more simple products, more scalable, and easier to do. You can turn your
[01:30:18] and easier to do. You can turn your ideas into income and get the support to
[01:30:20] ideas into income and get the support to grow whatever you're building. And we're
[01:30:22] grow whatever you're building. And we're about to launch Dare to Dream. It's for
[01:30:25] about to launch Dare to Dream. It's for those who are ready to make the shift
[01:30:26] those who are ready to make the shift from thinking to building, from planning
[01:30:29] from thinking to building, from planning to actually doing the thing. It's about
[01:30:31] to actually doing the thing. It's about seeing that dream in your head and
[01:30:32] seeing that dream in your head and knowing exactly what it takes to bring
[01:30:34] knowing exactly what it takes to bring it to life. If you're ready to transform
[01:30:36] it to life. If you're ready to transform your life, visit dare to
[01:30:37] your life, visit dare to dream.stan.store.
[01:30:41] You've made many attempts to raise
[01:30:43] You've made many attempts to raise awareness and to call for
[01:30:46] awareness and to call for a heightened consciousness about the
[01:30:48] a heightened consciousness about the future of AI. Um in October, over 850
[01:30:51] future of AI. Um in October, over 850 experts, including yourself and other
[01:30:53] experts, including yourself and other leaders like Richard Branson, who I've
[01:30:54] leaders like Richard Branson, who I've had on this show, and Geoffrey Hinton,
[01:30:55] had on this show, and Geoffrey Hinton, who I've had on this show, signed a
[01:30:57] who I've had on this show, signed a statement to ban AI superintelligence.
[01:31:00] statement to ban AI superintelligence. As you guys raised concerns of potential
[01:31:02] As you guys raised concerns of potential human extinction. Sort of, yeah. It says
[01:31:05] human extinction. Sort of, yeah. It says at least until
[01:31:07] at least until we are sure that we can move forward
[01:31:08] we are sure that we can move forward safely and there's broad scientific
[01:31:10] safely and there's broad scientific consensus
[01:31:11] consensus on that. So, that Did it work?
[01:31:15] on that. So, that Did it work? It's hard It's hard to say. I mean,
[01:31:16] It's hard It's hard to say. I mean, interestingly, there was a
[01:31:19] interestingly, there was a related So, what was called the the
[01:31:21] related So, what was called the the pause statement was March of '23. So,
[01:31:24] pause statement was March of '23. So, that was when GPT-4 came out, the
[01:31:26] that was when GPT-4 came out, the successor to ChatGPT.
[01:31:28] successor to ChatGPT. So, we we suggested that there be a
[01:31:30] So, we we suggested that there be a 6-month pause in
[01:31:32] 6-month pause in developing and deploying systems more
[01:31:34] developing and deploying systems more powerful than GPT-4. And everyone
[01:31:36] powerful than GPT-4. And everyone pooh-poohed
[01:31:37] pooh-poohed that idea. Of course, no one's going to
[01:31:40] that idea. Of course, no one's going to pause anything. But in fact, there were
[01:31:41] pause anything. But in fact, there were no systems in the next 6 months deployed
[01:31:44] no systems in the next 6 months deployed that were more powerful than GPT-4.
[01:31:47] that were more powerful than GPT-4. Um and coincidence? You be the judge.
[01:31:50] Um and coincidence? You be the judge. I would say
[01:31:52] I would say that
[01:31:53] that what we're trying to do is to
[01:31:55] what we're trying to do is to is to basically shift
[01:31:58] is to basically shift the
[01:31:59] the the public debate.
[01:32:01] the public debate. You know, there's this bizarre
[01:32:03] You know, there's this bizarre phenomenon
[01:32:04] phenomenon that keeps happening in the media
[01:32:07] that keeps happening in the media where if you talk about these risks,
[01:32:11] where if you talk about these risks, they will say, "Oh, you know, there's a
[01:32:13] they will say, "Oh, you know, there's a fringe of
[01:32:14] fringe of people, you know, called {quote}
[01:32:16] people, you know, called {quote} doomers, who think that there's, you
[01:32:18] doomers, who think that there's, you know, a risk of extinction."
[01:32:20] know, a risk of extinction." Um so, they always the narrative is
[01:32:22] Um so, they always the narrative is always that oh, it you know, talking
[01:32:24] always that oh, it you know, talking about those risks is a fringe thing.
[01:32:26] about those risks is a fringe thing. Pretty much all the CEOs of the leading
[01:32:28] Pretty much all the CEOs of the leading AI companies
[01:32:30] AI companies think that there's a significant risk of
[01:32:32] think that there's a significant risk of extinction.
[01:32:34] extinction. Almost all the leading AI researchers
[01:32:35] Almost all the leading AI researchers think there's a significant risk of
[01:32:37] think there's a significant risk of human extinction.
[01:32:39] human extinction. Um
[01:32:40] Um So, what
[01:32:42] So, what Why is that the fringe, right? Why isn't
[01:32:43] Why is that the fringe, right? Why isn't that the mainstream? If these these are
[01:32:45] that the mainstream? If these these are the leading experts in industry and
[01:32:47] the leading experts in industry and academia
[01:32:49] academia uh saying this, how could it be the
[01:32:50] uh saying this, how could it be the fringe?
[01:32:52] fringe? So, we're trying to change that
[01:32:53] So, we're trying to change that narrative
[01:32:55] narrative to say, "No, the people who really
[01:32:58] to say, "No, the people who really understand this stuff
[01:33:00] understand this stuff are extremely concerned."
[01:33:03] are extremely concerned." And what do you want to happen? What is
[01:33:05] And what do you want to happen? What is the solution? What I think is that we
[01:33:07] the solution? What I think is that we should have effective regulation.
[01:33:11] It's hard to argue with that, right? Uh
[01:33:13] It's hard to argue with that, right? Uh So, what does effective mean? It means
[01:33:15] So, what does effective mean? It means that if you comply with the regulation,
[01:33:18] that if you comply with the regulation, then the risks are reduced to an
[01:33:20] then the risks are reduced to an acceptable level.
[01:33:23] So, for example,
[01:33:26] So, for example, we ask people who want to operate
[01:33:28] we ask people who want to operate nuclear plants,
[01:33:29] nuclear plants, right? We've decided that the risk we're
[01:33:32] right? We've decided that the risk we're willing to live with is,
[01:33:34] willing to live with is, you know, a one in a million chance per
[01:33:37] you know, a one in a million chance per year
[01:33:38] year that the plant is going to have a
[01:33:39] that the plant is going to have a meltdown.
[01:33:41] meltdown. Any higher than that,
[01:33:42] Any higher than that, you know, we just don't It's not worth
[01:33:44] you know, we just don't It's not worth it.
[01:33:44] it. Right? So, you have to be below that.
[01:33:46] Right? So, you have to be below that. Some cases, we can get down to one in 10
[01:33:49] Some cases, we can get down to one in 10 million chance per year.
[01:33:51] million chance per year. So, what chance do you think we should
[01:33:53] So, what chance do you think we should be willing to live with for human
[01:33:54] be willing to live with for human extinction?
[01:33:57] Me? Yeah.
[01:34:01] Hm.
[01:34:02] Hm. 0.0001.
[01:34:04] 0.0001. Yeah, lots of zeros. Yeah. Right? So,
[01:34:07] Yeah, lots of zeros. Yeah. Right? So, one in a million
[01:34:08] one in a million for a nuclear meltdown,
[01:34:11] for a nuclear meltdown, extinction's much worse.
[01:34:12] extinction's much worse. >> Oh, yeah. So, yeah. It's kind of Right?
[01:34:14] >> Oh, yeah. So, yeah. It's kind of Right? So,
[01:34:14] So, >> One in a hundred billion, one in a
[01:34:16] >> One in a hundred billion, one in a trillion. Yeah. So, if you said one in a
[01:34:18] trillion. Yeah. So, if you said one in a billion, right? Then you'd expect one
[01:34:20] billion, right? Then you'd expect one extinction per billion years.
[01:34:22] extinction per billion years. There's a background. So, one one of the
[01:34:23] There's a background. So, one one of the ways people work out these risk levels
[01:34:25] ways people work out these risk levels is also to look at the background.
[01:34:27] is also to look at the background. The other ways of getting going extinct
[01:34:29] The other ways of getting going extinct would include, you know, giant asteroid
[01:34:31] would include, you know, giant asteroid crashes into the Earth. And you can
[01:34:33] crashes into the Earth. And you can roughly calculate what those
[01:34:34] roughly calculate what those probabilities are. We can look at how
[01:34:36] probabilities are. We can look at how many
[01:34:37] many extinction-level events have happened in
[01:34:39] extinction-level events have happened in the past, and you know, maybe it's half
[01:34:41] the past, and you know, maybe it's half a dozen. So, so there's maybe it's like
[01:34:43] a dozen. So, so there's maybe it's like a one in
[01:34:44] a one in 500 million year event.
[01:34:48] 500 million year event. So, somewhere in that range, right?
[01:34:50] So, somewhere in that range, right? Somewhere between one in 10 million,
[01:34:52] Somewhere between one in 10 million, which is the best nuclear power plants,
[01:34:53] which is the best nuclear power plants, and
[01:34:54] and and one in 500 million or one in a
[01:34:56] and one in 500 million or one in a billion,
[01:34:57] billion, which is the background
[01:34:59] which is the background uh risk from from giant asteroids.
[01:35:01] uh risk from from giant asteroids. Uh so, let's say we settle on a hundred
[01:35:03] Uh so, let's say we settle on a hundred million.
[01:35:04] million. One in a hundred million chance per
[01:35:05] One in a hundred million chance per year. Well, what is it according to the
[01:35:07] year. Well, what is it according to the CEOs?
[01:35:09] CEOs? 25%.
[01:35:11] 25%. So,
[01:35:12] So, they're off by a factor of
[01:35:15] they're off by a factor of multiple millions.
[01:35:18] multiple millions. Right? So, they need to make the AI
[01:35:19] Right? So, they need to make the AI systems millions of times safer.
[01:35:23] systems millions of times safer. Your analogy of the ruler Russian
[01:35:25] Your analogy of the ruler Russian roulette comes back in here, because
[01:35:26] roulette comes back in here, because that's like, for anyone that doesn't
[01:35:28] that's like, for anyone that doesn't know what probabilities are in this
[01:35:29] know what probabilities are in this context, that's like having a
[01:35:32] context, that's like having a ammunition chamber with four
[01:35:36] ammunition chamber with four holes in it and putting a bullet in one
[01:35:37] holes in it and putting a bullet in one of them. One in four, yeah. And we're
[01:35:39] of them. One in four, yeah. And we're saying we want it to be one in a
[01:35:40] saying we want it to be one in a billion. So, we want a billion chambers
[01:35:43] billion. So, we want a billion chambers and a bullet in one of them.
[01:35:44] and a bullet in one of them. Yeah. And and so, when you look at the
[01:35:47] Yeah. And and so, when you look at the work that the nuclear operators have to
[01:35:48] work that the nuclear operators have to do to show that their system is that
[01:35:50] do to show that their system is that reliable,
[01:35:53] reliable, uh it's a massive mathematical analysis
[01:35:56] uh it's a massive mathematical analysis of the components, the you know, the
[01:35:58] of the components, the you know, the redundancy. You've got monitors, you've
[01:36:00] redundancy. You've got monitors, you've got warning lights, you've got operating
[01:36:02] got warning lights, you've got operating procedures.
[01:36:04] procedures. Uh you have all kinds of mechanisms
[01:36:06] Uh you have all kinds of mechanisms which over the decades have ratcheted
[01:36:08] which over the decades have ratcheted that risk down. It started out, I think,
[01:36:11] that risk down. It started out, I think, one in one in 10,000 years, right? And
[01:36:14] one in one in 10,000 years, right? And they've improved it by a factor of a
[01:36:17] they've improved it by a factor of a hundred or a thousand by all of these
[01:36:18] hundred or a thousand by all of these mechanisms.
[01:36:20] mechanisms. But at every stage, they had to do a
[01:36:21] But at every stage, they had to do a mathematical analysis to show what the
[01:36:23] mathematical analysis to show what the risk was.
[01:36:26] The people developing the AI company the
[01:36:28] The people developing the AI company the AI systems Sorry, the AI companies
[01:36:30] AI systems Sorry, the AI companies developing these systems,
[01:36:31] developing these systems, they don't even understand how the AI
[01:36:33] they don't even understand how the AI systems work.
[01:36:34] systems work. So, their 25% chance of extinction is
[01:36:36] So, their 25% chance of extinction is just a seat of the pants guess.
[01:36:38] just a seat of the pants guess. They actually have no idea.
[01:36:41] They actually have no idea. But the tests that they are doing on
[01:36:44] But the tests that they are doing on their systems right now,
[01:36:45] their systems right now, you know, they show that the AI systems
[01:36:48] you know, they show that the AI systems will be willing to kill people
[01:36:51] will be willing to kill people uh to preserve their own existence
[01:36:54] uh to preserve their own existence already.
[01:36:55] already. Right? They will lie to people, they
[01:36:58] Right? They will lie to people, they will blackmail them, they will they will
[01:37:00] will blackmail them, they will they will launch nuclear weapons rather than
[01:37:03] launch nuclear weapons rather than uh be switched off. And so, there's no
[01:37:06] uh be switched off. And so, there's no there's no positive sign that we're
[01:37:08] there's no positive sign that we're getting any closer to safety
[01:37:10] getting any closer to safety with these systems. In fact, the signs
[01:37:12] with these systems. In fact, the signs seem to be that we're going
[01:37:14] seem to be that we're going uh deeper and deeper into
[01:37:16] uh deeper and deeper into uh into dangerous behaviors.
[01:37:18] uh into dangerous behaviors. So, rather than say ban, I would just
[01:37:20] So, rather than say ban, I would just say,
[01:37:22] say, "Prove to us that the risk is less than
[01:37:24] "Prove to us that the risk is less than one in a hundred million per year of
[01:37:26] one in a hundred million per year of extinction
[01:37:27] extinction or loss of control, let's say.
[01:37:29] or loss of control, let's say. And uh
[01:37:30] And uh So, we're not banning anything.
[01:37:33] So, we're not banning anything. The company's response is,
[01:37:36] The company's response is, "Well, we don't know how to do that, so
[01:37:38] "Well, we don't know how to do that, so you can't have a rule."
[01:37:41] Literally, they are saying humanity has
[01:37:44] Literally, they are saying humanity has no right to protect itself from us.
[01:37:48] no right to protect itself from us. If I was an alien looking down on planet
[01:37:50] If I was an alien looking down on planet Earth right now, I'd find this
[01:37:51] Earth right now, I'd find this fascinating.
[01:37:53] fascinating. That these
[01:37:54] That these >> Yeah, you're in the bar betting on
[01:37:55] >> Yeah, you're in the bar betting on who's, you know, are they going to make
[01:37:57] who's, you know, are they going to make it or not? Just a really interesting
[01:37:59] it or not? Just a really interesting experiment in like human incentives. The
[01:38:02] experiment in like human incentives. The analogy you gave of there being this
[01:38:03] analogy you gave of there being this quadri quadrillion dollar magnet pulling
[01:38:06] quadri quadrillion dollar magnet pulling us off the edge of the cliff.
[01:38:08] us off the edge of the cliff. And
[01:38:09] And yet we're still being drawn towards it
[01:38:12] yet we're still being drawn towards it through greed and this promise of
[01:38:13] through greed and this promise of abundance and power and status, and I'm
[01:38:15] abundance and power and status, and I'm going to be the one that summoned the
[01:38:16] going to be the one that summoned the god. Hm.
[01:38:18] god. Hm. I mean, it says something about us as
[01:38:20] I mean, it says something about us as humans.
[01:38:21] humans. Says something about our
[01:38:24] Says something about our our darker sides.
[01:38:25] our darker sides. Yes, and the aliens will write an
[01:38:28] Yes, and the aliens will write an amazing tragic
[01:38:30] amazing tragic play cycle
[01:38:32] play cycle about what happened to the human race.
[01:38:34] about what happened to the human race. Maybe the AI is the alien.
[01:38:37] Maybe the AI is the alien. And it's going to talk about, you know,
[01:38:38] And it's going to talk about, you know, we have our our stories about God making
[01:38:41] we have our our stories about God making the world in seven days and Adam and
[01:38:43] the world in seven days and Adam and Eve. Maybe it'll have its own religious
[01:38:45] Eve. Maybe it'll have its own religious stories about
[01:38:47] stories about the god that made it, us.
[01:38:50] the god that made it, us. And how it sacrificed itself, just like
[01:38:52] And how it sacrificed itself, just like Jesus sacrificed himself for us. We
[01:38:54] Jesus sacrificed himself for us. We sacrificed ourselves for it.
[01:38:57] sacrificed ourselves for it. Hm.
[01:38:58] Hm. Yeah, which is the wrong way around,
[01:39:01] Yeah, which is the wrong way around, right?
[01:39:03] right? But that is that is the story of that's
[01:39:04] But that is that is the story of that's that's the Judeo-Christian story, isn't
[01:39:07] that's the Judeo-Christian story, isn't it? That God you know, Jesus gave his
[01:39:09] it? That God you know, Jesus gave his life for us so that we could be here,
[01:39:12] life for us so that we could be here, full of sin.
[01:39:13] full of sin. Mhm.
[01:39:14] Mhm. But is Yeah, God is still watching over
[01:39:16] But is Yeah, God is still watching over us.
[01:39:17] us. And probably
[01:39:19] And probably wondering when we're going to get our
[01:39:20] wondering when we're going to get our act together.
[01:39:23] act together. What is the most important thing we
[01:39:24] What is the most important thing we haven't talked about that we should have
[01:39:25] haven't talked about that we should have talked about, Professor Stuart Russell?
[01:39:27] talked about, Professor Stuart Russell? So, I think
[01:39:28] So, I think um
[01:39:30] um the question of
[01:39:32] the question of whether it's possible
[01:39:34] whether it's possible to make
[01:39:36] to make uh superintelligent AI systems
[01:39:39] uh superintelligent AI systems that we can control. Is it possible? I I
[01:39:41] that we can control. Is it possible? I I think yes, I think it's possible, and I
[01:39:43] think yes, I think it's possible, and I think we need
[01:39:45] think we need to actually just have a different
[01:39:48] to actually just have a different conception of what it is we're trying to
[01:39:49] conception of what it is we're trying to build.
[01:39:51] build. For a long time with with AI, we've just
[01:39:54] For a long time with with AI, we've just had this notion of
[01:39:56] had this notion of pure intelligence.
[01:39:57] pure intelligence. Right? The
[01:39:58] Right? The the ability to bring about whatever
[01:40:01] the ability to bring about whatever future you, the intelligent entity, want
[01:40:04] future you, the intelligent entity, want to bring about.
[01:40:05] to bring about. >> The more intelligent, the better.
[01:40:06] >> The more intelligent, the better. >> The more intelligent, the better, and
[01:40:07] >> The more intelligent, the better, and the more capability it will have
[01:40:10] the more capability it will have to create the future that it wants.
[01:40:13] to create the future that it wants. And actually, we don't want
[01:40:15] And actually, we don't want pure intelligence.
[01:40:18] pure intelligence. Because
[01:40:20] Because what the future that it wants might not
[01:40:22] what the future that it wants might not be the future that we want.
[01:40:24] be the future that we want. And there's nothing partic- you know,
[01:40:26] And there's nothing partic- you know, the universe doesn't single humans out
[01:40:28] the universe doesn't single humans out as the the only thing that matters.
[01:40:31] as the the only thing that matters. Right? You know, pure intelligence might
[01:40:34] Right? You know, pure intelligence might decide that actually it's going to make
[01:40:36] decide that actually it's going to make life wonderful for cockroaches or
[01:40:38] life wonderful for cockroaches or or actually doesn't care about
[01:40:40] or actually doesn't care about biological life at all.
[01:40:43] biological life at all. We actually want
[01:40:44] We actually want intelligence whose
[01:40:46] intelligence whose only purpose is to
[01:40:49] only purpose is to bring about the future that we want.
[01:40:52] bring about the future that we want. Right? So, it's we want it to be
[01:40:54] Right? So, it's we want it to be first of all, keyed to humans
[01:40:57] first of all, keyed to humans specifically, not to cockroaches, not to
[01:40:59] specifically, not to cockroaches, not to aliens, not to itself. You want to make
[01:41:01] aliens, not to itself. You want to make it loyal to humans.
[01:41:02] it loyal to humans. >> Right. So, keyed to humans.
[01:41:05] >> Right. So, keyed to humans. And the difficulty that I mentioned
[01:41:06] And the difficulty that I mentioned earlier, right? The King Midas problem.
[01:41:09] earlier, right? The King Midas problem. How do we specify
[01:41:11] How do we specify what we want the future to be like so
[01:41:13] what we want the future to be like so that it can do it for us. How do we
[01:41:14] that it can do it for us. How do we specify the objectives?
[01:41:17] specify the objectives? Actually, we have to give up on that
[01:41:19] Actually, we have to give up on that idea.
[01:41:20] idea. Because it's not possible. Right? We've
[01:41:22] Because it's not possible. Right? We've seen this over and over again in human
[01:41:24] seen this over and over again in human history.
[01:41:25] history. Uh we don't know how to specify the
[01:41:27] Uh we don't know how to specify the future
[01:41:28] future properly. We don't know how to say what
[01:41:30] properly. We don't know how to say what we want.
[01:41:31] we want. And you know, I always use the example
[01:41:33] And you know, I always use the example of
[01:41:34] of the genie. Right? What's the third wish
[01:41:37] the genie. Right? What's the third wish that you give to the genie who's granted
[01:41:39] that you give to the genie who's granted you three wishes?
[01:41:40] you three wishes? Right? Undo the first two wishes cuz I
[01:41:42] Right? Undo the first two wishes cuz I made a mess of the universe.
[01:41:46] So, um
[01:41:48] So, um So, in fact, what we're going to do is
[01:41:51] So, in fact, what we're going to do is we're going to make it the machine's job
[01:41:54] we're going to make it the machine's job to figure out. So, it has to bring about
[01:41:56] to figure out. So, it has to bring about the future that we want.
[01:41:59] the future that we want. But
[01:42:02] it has to figure out what that is.
[01:42:04] it has to figure out what that is. And it's going to start out not knowing.
[01:42:09] And uh
[01:42:11] And uh over time, through interacting with us
[01:42:13] over time, through interacting with us and observing the choices we make,
[01:42:16] and observing the choices we make, it will learn more about what we want
[01:42:18] it will learn more about what we want the future to be like.
[01:42:20] the future to be like. But probably it will forever have
[01:42:24] But probably it will forever have residual uncertainty
[01:42:27] residual uncertainty about what we really want the future to
[01:42:29] about what we really want the future to be like. It'll It'll be fairly sure
[01:42:31] be like. It'll It'll be fairly sure about some things, and it can help us
[01:42:33] about some things, and it can help us with those.
[01:42:34] with those. And it'll be uncertain about other
[01:42:35] And it'll be uncertain about other things, and it'll be
[01:42:37] things, and it'll be uh in those cases, it will not
[01:42:40] uh in those cases, it will not take action that might
[01:42:43] take action that might upset
[01:42:45] upset humans with that, you know, with that
[01:42:46] humans with that, you know, with that aspect of the world. So, to give you a
[01:42:48] aspect of the world. So, to give you a simple example, right?
[01:42:49] simple example, right? Um what color do we want the sky to be?
[01:42:54] Um what color do we want the sky to be? It's not sure.
[01:42:55] It's not sure. So, it shouldn't mess with the sky.
[01:42:58] So, it shouldn't mess with the sky. Unless it knows for sure that we really
[01:43:00] Unless it knows for sure that we really want purple with green stripes.
[01:43:02] want purple with green stripes. Everything you're saying sounds like
[01:43:04] Everything you're saying sounds like we're creating
[01:43:06] we're creating a god. Like earlier on, I was saying
[01:43:08] a god. Like earlier on, I was saying that we are the god. But actually,
[01:43:09] that we are the god. But actually, everything you described there almost
[01:43:11] everything you described there almost sounds like every every god in religion
[01:43:13] sounds like every every god in religion where, you know, we pray to gods, but
[01:43:15] where, you know, we pray to gods, but they don't always do anything about it.
[01:43:17] they don't always do anything about it. Not Not exactly. No, it's it's In some
[01:43:19] Not Not exactly. No, it's it's In some sense, I'm thinking more like a
[01:43:23] sense, I'm thinking more like a the ideal butler. To the extent that the
[01:43:25] the ideal butler. To the extent that the butler can anticipate your wishes, they
[01:43:27] butler can anticipate your wishes, they should help you bring them about. But in
[01:43:30] should help you bring them about. But in in areas where there's uncertainty,
[01:43:33] in areas where there's uncertainty, it can ask questions. We can We can make
[01:43:36] it can ask questions. We can We can make requests.
[01:43:37] requests. This sounds like God to me, because, you
[01:43:38] This sounds like God to me, because, you know, I might say to God or this butler,
[01:43:42] know, I might say to God or this butler, uh could you go get me my
[01:43:43] uh could you go get me my my car keys from upstairs? And its
[01:43:45] my car keys from upstairs? And its assessment would be,
[01:43:47] assessment would be, listen, if I do this for this person,
[01:43:49] listen, if I do this for this person, then their muscles are going to atrophy,
[01:43:51] then their muscles are going to atrophy, then they're going to lose meaning in
[01:43:52] then they're going to lose meaning in their life, then they're not going to
[01:43:53] their life, then they're not going to know how to do hard things, so I won't
[01:43:54] know how to do hard things, so I won't get involved. It's an intelligence that
[01:43:56] get involved. It's an intelligence that sits in But actually, probably in most
[01:43:58] sits in But actually, probably in most situations, it optimizing for comfort
[01:44:01] situations, it optimizing for comfort for me or doing things for me is
[01:44:01] for me or doing things for me is actually probably not in my best
[01:44:03] actually probably not in my best long-term interests. Probably It's
[01:44:05] long-term interests. Probably It's probably useful that I have a girlfriend
[01:44:06] probably useful that I have a girlfriend and argue with her. And then I like
[01:44:08] and argue with her. And then I like raise kids, and then I walk to the shop
[01:44:10] raise kids, and then I walk to the shop and get my own stuff. I agree with you.
[01:44:12] and get my own stuff. I agree with you. I mean, I think that's So, you're
[01:44:14] I mean, I think that's So, you're putting your finger on
[01:44:16] putting your finger on uh
[01:44:17] uh in some sense, sort of version 2.0.
[01:44:20] in some sense, sort of version 2.0. Right? So, let's get version 1.0
[01:44:22] Right? So, let's get version 1.0 clear, right? This This
[01:44:24] clear, right? This This this form of AI where
[01:44:27] this form of AI where it has to further our interests, but it
[01:44:29] it has to further our interests, but it doesn't know what those interests are,
[01:44:32] doesn't know what those interests are, right? Then puts an obligation on it to
[01:44:34] right? Then puts an obligation on it to learn more.
[01:44:35] learn more. And to be helpful where it understands
[01:44:38] And to be helpful where it understands well enough, and to be cautious where it
[01:44:40] well enough, and to be cautious where it doesn't understand well, and so on.
[01:44:43] doesn't understand well, and so on. So, that that actually we can formulate
[01:44:46] So, that that actually we can formulate as a mathematical problem, and at least
[01:44:48] as a mathematical problem, and at least under idealized circumstances, we can
[01:44:50] under idealized circumstances, we can literally solve that problem. So, we can
[01:44:53] literally solve that problem. So, we can make
[01:44:54] make AI systems
[01:44:56] AI systems that know how to solve this problem and
[01:44:57] that know how to solve this problem and help the entities that they are
[01:45:00] help the entities that they are interacting with. The reason I make the
[01:45:01] interacting with. The reason I make the god analogy is because I think that such
[01:45:04] god analogy is because I think that such a being, such an intelligence, would
[01:45:05] a being, such an intelligence, would realize the importance of equilibrium in
[01:45:07] realize the importance of equilibrium in the world.
[01:45:08] the world. Pain and pleasure, good and evil. And
[01:45:11] Pain and pleasure, good and evil. And then it would Absolutely.
[01:45:13] then it would Absolutely. >> And then it would be like this. So, so
[01:45:14] >> And then it would be like this. So, so so, right? So, yes, I mean,
[01:45:17] so, right? So, yes, I mean, I mean, and then that's sort of what
[01:45:18] I mean, and then that's sort of what happens in the Matrix, right? They try
[01:45:20] happens in the Matrix, right? They try The AI systems in the Matrix, they tried
[01:45:23] The AI systems in the Matrix, they tried to give us a utopia,
[01:45:25] to give us a utopia, but it failed miserably. And you know,
[01:45:28] but it failed miserably. And you know, fields and fields of humans had to be
[01:45:29] fields and fields of humans had to be destroyed.
[01:45:31] destroyed. Um
[01:45:32] Um And the best they could come up with
[01:45:33] And the best they could come up with was, you know, late 20th century
[01:45:35] was, you know, late 20th century regular human life with all of its
[01:45:38] regular human life with all of its problems. Like
[01:45:39] problems. Like And I think this is a really interesting
[01:45:41] And I think this is a really interesting point.
[01:45:43] point. And absolutely central, because, you
[01:45:45] And absolutely central, because, you know, there's a lot of science fiction
[01:45:47] know, there's a lot of science fiction where superintelligent robots, you know,
[01:45:50] where superintelligent robots, you know, they just want to help humans.
[01:45:54] they just want to help humans. And the humans who don't like that, you
[01:45:55] And the humans who don't like that, you know, they just give them a little brain
[01:45:56] know, they just give them a little brain operation to then they do like it. Um
[01:46:00] operation to then they do like it. Um And it takes away
[01:46:02] And it takes away human
[01:46:03] human motivation
[01:46:05] motivation uh it
[01:46:06] uh it by taking away failure, uh taking away
[01:46:10] by taking away failure, uh taking away disease, you actually lose important
[01:46:12] disease, you actually lose important parts of human life.
[01:46:14] parts of human life. And it becomes in some sense pointless.
[01:46:16] And it becomes in some sense pointless. So,
[01:46:17] So, if it turns out
[01:46:19] if it turns out that
[01:46:20] that there simply isn't any way that
[01:46:23] there simply isn't any way that humans can really flourish
[01:46:27] humans can really flourish in coexistence with superintelligent
[01:46:29] in coexistence with superintelligent machines, even if they're
[01:46:31] machines, even if they're perfectly designed to to
[01:46:34] perfectly designed to to to solve this problem of figuring out
[01:46:36] to solve this problem of figuring out what humans what futures humans want and
[01:46:39] what humans what futures humans want and and bring about those futures.
[01:46:42] and bring about those futures. If that's not possible, then those
[01:46:45] If that's not possible, then those machines will actually
[01:46:47] machines will actually disappear.
[01:46:49] disappear. Why would they disappear? Because that's
[01:46:51] Why would they disappear? Because that's the best thing for us.
[01:46:53] the best thing for us. Maybe they would stay
[01:46:55] Maybe they would stay available for real existential
[01:46:58] available for real existential emergencies, like if there is a giant
[01:46:59] emergencies, like if there is a giant asteroid about to hit the earth, then
[01:47:01] asteroid about to hit the earth, then maybe they'll help us.
[01:47:02] maybe they'll help us. Uh because they at least want the human
[01:47:04] Uh because they at least want the human species to continue. But to some extent,
[01:47:07] species to continue. But to some extent, it's not a perfect analogy, but it's
[01:47:09] it's not a perfect analogy, but it's it's sort of the way that human parents
[01:47:12] it's sort of the way that human parents have to at some point step back
[01:47:14] have to at some point step back from their kids' lives and say, "Okay,
[01:47:17] from their kids' lives and say, "Okay, now you have to tie your own shoelaces
[01:47:19] now you have to tie your own shoelaces today."
[01:47:20] today." This is kind of what I was thinking.
[01:47:21] This is kind of what I was thinking. Maybe there was
[01:47:22] Maybe there was a a civilization before us, and they
[01:47:25] a a civilization before us, and they arrived at this moment in time
[01:47:27] arrived at this moment in time where they created an intelligence.
[01:47:31] where they created an intelligence. And that intelligence did all the things
[01:47:33] And that intelligence did all the things you've said, and it realized the
[01:47:35] you've said, and it realized the importance of equilibrium, so it decided
[01:47:36] importance of equilibrium, so it decided not to get involved.
[01:47:38] not to get involved. And
[01:47:40] And maybe at some level,
[01:47:43] maybe at some level, that's the god we look up to the stars
[01:47:45] that's the god we look up to the stars and worship, one that's not really
[01:47:47] and worship, one that's not really getting involved in letting things play
[01:47:48] getting involved in letting things play out however however they are. But might
[01:47:50] out however however they are. But might step in in the case of a real
[01:47:52] step in in the case of a real existential emergency.
[01:47:53] existential emergency. >> Maybe. Maybe not. Maybe. But then And
[01:47:55] >> Maybe. Maybe not. Maybe. But then And then maybe the cycle repeats itself
[01:47:57] then maybe the cycle repeats itself where, you know, the organisms it let
[01:48:00] where, you know, the organisms it let have free will end up creating the same
[01:48:02] have free will end up creating the same intelligence.
[01:48:04] intelligence. And then the universe perpetuates
[01:48:06] And then the universe perpetuates infinitely.
[01:48:08] infinitely. Yeah, there there are science fiction
[01:48:10] Yeah, there there are science fiction stories like that, too.
[01:48:11] stories like that, too. Yeah, I hope there is some
[01:48:14] Yeah, I hope there is some happy medium where
[01:48:17] happy medium where the AI systems can be there, and we can
[01:48:20] the AI systems can be there, and we can take advantage of
[01:48:22] take advantage of of those capabilities to have a
[01:48:24] of those capabilities to have a civilization that's much better
[01:48:26] civilization that's much better than the one we have now.
[01:48:28] than the one we have now. Um but I think you're right. A
[01:48:30] Um but I think you're right. A civilization with no challenges
[01:48:33] civilization with no challenges is not
[01:48:35] is not not conducive to human flourishing. What
[01:48:37] not conducive to human flourishing. What can the average person do, Stuart?
[01:48:40] can the average person do, Stuart? Average person listening to this now
[01:48:42] Average person listening to this now to aid the cause that you're fighting
[01:48:44] to aid the cause that you're fighting for? I actually think um
[01:48:47] for? I actually think um you know, this sounds corny, but you
[01:48:48] you know, this sounds corny, but you know, talk to your representative, your
[01:48:50] know, talk to your representative, your MP, your congressperson, whatever it is.
[01:48:53] MP, your congressperson, whatever it is. Um because
[01:48:56] I think the policy makers need to hear
[01:48:58] I think the policy makers need to hear from people. The only voices they're
[01:49:00] from people. The only voices they're hearing right now
[01:49:02] hearing right now are
[01:49:03] are the tech companies and their $50 checks.
[01:49:08] the tech companies and their $50 checks. And um um
[01:49:10] And um um all the polls that have been done say,
[01:49:13] all the polls that have been done say, "Yeah, most people, 80% maybe,
[01:49:16] "Yeah, most people, 80% maybe, don't want there to be superintelligent
[01:49:18] don't want there to be superintelligent machines."
[01:49:20] machines." But they don't know what to do.
[01:49:22] But they don't know what to do. You know, even for me, I've been in this
[01:49:24] You know, even for me, I've been in this field for decades,
[01:49:26] field for decades, uh I'm not sure what to do
[01:49:29] uh I'm not sure what to do because of this giant magnet pulling
[01:49:31] because of this giant magnet pulling everyone forward and
[01:49:33] everyone forward and uh and the vast sums of money being
[01:49:36] uh and the vast sums of money being being put into this.
[01:49:37] being put into this. Um
[01:49:38] Um but I am sure that
[01:49:40] but I am sure that if you want to have a future
[01:49:43] if you want to have a future and a world that you want your kids to
[01:49:45] and a world that you want your kids to live in,
[01:49:47] live in, I you need to make your voice heard.
[01:49:52] And uh and I think governments will
[01:49:53] And uh and I think governments will listen.
[01:49:55] listen. From a political point of view,
[01:49:58] From a political point of view, right, you put your finger
[01:49:59] right, you put your finger in the wind,
[01:50:01] in the wind, and you say, "Hmm,
[01:50:03] and you say, "Hmm, should I be on the side of humanity or
[01:50:06] should I be on the side of humanity or our future robot overlords?"
[01:50:09] our future robot overlords?" I think I think as a politician, it's
[01:50:11] I think I think as a politician, it's not a difficult decision.
[01:50:14] not a difficult decision. It is when you've got someone saying,
[01:50:15] It is when you've got someone saying, "I'll give you 50 billion dollars."
[01:50:17] "I'll give you 50 billion dollars." >> Yeah, right.
[01:50:18] >> Yeah, right. Exactly. So, um I think
[01:50:21] Exactly. So, um I think I think people in those positions of
[01:50:23] I think people in those positions of power need to hear from their
[01:50:25] power need to hear from their constituents
[01:50:27] constituents um that this is not the direction we
[01:50:29] um that this is not the direction we want to go. After committing your career
[01:50:32] want to go. After committing your career to this subject and the subject of
[01:50:34] to this subject and the subject of technology more broadly, but
[01:50:35] technology more broadly, but specifically being the guy that wrote
[01:50:36] specifically being the guy that wrote the book about artificial intelligence,
[01:50:42] you must realize that you're living in a
[01:50:44] you must realize that you're living in a historical moment. Like there's very few
[01:50:46] historical moment. Like there's very few times in my life where I go, "Oh, this
[01:50:47] times in my life where I go, "Oh, this is one of those moments."
[01:50:49] is one of those moments." This is a crossroads in history.
[01:50:51] This is a crossroads in history. And it must to some degree weigh upon
[01:50:54] And it must to some degree weigh upon you, knowing that you're a person of
[01:50:55] you, knowing that you're a person of influence at this historical moment in
[01:50:57] influence at this historical moment in time, who could theoretically
[01:51:00] time, who could theoretically help divert the course of history in
[01:51:02] help divert the course of history in this moment in time. It's kind of like
[01:51:03] this moment in time. It's kind of like the You look through history and see
[01:51:05] the You look through history and see these moments of like Oppenheimer and
[01:51:07] these moments of like Oppenheimer and um
[01:51:08] um Does it weigh on you?
[01:51:09] Does it weigh on you? When you're alone at night
[01:51:11] When you're alone at night thinking to yourself and reading things?
[01:51:13] thinking to yourself and reading things? Yeah, it does. I mean, you know, after
[01:51:14] Yeah, it does. I mean, you know, after 50 years I could retire and um you know,
[01:51:17] 50 years I could retire and um you know, play golf and sing and sail and do
[01:51:19] play golf and sing and sail and do things I enjoy.
[01:51:21] things I enjoy. Um
[01:51:23] Um but instead I'm working 80 or 100 hours
[01:51:25] but instead I'm working 80 or 100 hours a week
[01:51:26] a week um trying to move
[01:51:29] um trying to move uh move things in the right direction.
[01:51:31] uh move things in the right direction. What is that narrative in your head
[01:51:33] What is that narrative in your head that's making you do that? Like what is
[01:51:34] that's making you do that? Like what is the Is there an element of
[01:51:36] the Is there an element of I might regret this if I don't or Just
[01:51:40] I might regret this if I don't or Just it's it's not only the the right thing
[01:51:43] it's it's not only the the right thing to do, it's it's completely essential.
[01:51:46] to do, it's it's completely essential. I mean, there isn't
[01:51:50] there isn't a bigger motivation
[01:51:54] than this.
[01:51:56] than this. Do you feel like you're winning
[01:51:58] Do you feel like you're winning or losing?
[01:51:59] or losing? It feels um
[01:52:03] like things are
[01:52:04] like things are moving somewhat in the right direction.
[01:52:07] moving somewhat in the right direction. You know, it's a a ding-dong battle as
[01:52:09] You know, it's a a ding-dong battle as uh as
[01:52:11] uh as David Coleman used to say in the the uh
[01:52:13] David Coleman used to say in the the uh in the exciting football match.
[01:52:15] in the exciting football match. In 2023, right, so
[01:52:18] In 2023, right, so uh GPT-4 came out, and then we issued
[01:52:21] uh GPT-4 came out, and then we issued the pause statement that was signed by a
[01:52:24] the pause statement that was signed by a lot of leading AI researchers.
[01:52:26] lot of leading AI researchers. Um
[01:52:27] Um and then in May there was the extinction
[01:52:29] and then in May there was the extinction statement, which included
[01:52:31] statement, which included uh Sam Altman and Demis Hassabis and
[01:52:34] uh Sam Altman and Demis Hassabis and Dario Amodei and other
[01:52:36] Dario Amodei and other CEOs as well saying, "Yeah, this is an
[01:52:38] CEOs as well saying, "Yeah, this is an extinction risk on the level with
[01:52:40] extinction risk on the level with nuclear war."
[01:52:41] nuclear war." And I think
[01:52:43] And I think governments listened at that point.
[01:52:45] governments listened at that point. The UK government
[01:52:47] The UK government earlier that year had said, "Well, you
[01:52:48] earlier that year had said, "Well, you know, we don't need to regulate AI, you
[01:52:50] know, we don't need to regulate AI, you know, full speed ahead.
[01:52:51] know, full speed ahead. Technology is good for you."
[01:52:54] Technology is good for you." And by June they had completely changed.
[01:52:58] And by June they had completely changed. And Rishi Sunak announced that he was
[01:53:00] And Rishi Sunak announced that he was going to hold
[01:53:01] going to hold this global AI safety summit uh in
[01:53:04] this global AI safety summit uh in England, and he wanted
[01:53:06] England, and he wanted London to be the global hub for AI
[01:53:08] London to be the global hub for AI regulation.
[01:53:10] regulation. Um and so on. So,
[01:53:13] Um and so on. So, and then
[01:53:14] and then you know, in beginning of November of
[01:53:16] you know, in beginning of November of '23, 28 countries, including the US and
[01:53:19] '23, 28 countries, including the US and China, signed a declaration
[01:53:21] China, signed a declaration saying, you know, AI presents
[01:53:24] saying, you know, AI presents catastrophic risks and it's urgent that
[01:53:26] catastrophic risks and it's urgent that we address them.
[01:53:27] we address them. So on. So, there it felt like, "Wow,
[01:53:31] So on. So, there it felt like, "Wow, they're listening.
[01:53:32] they're listening. They're going to do something about it."
[01:53:35] They're going to do something about it." And then I think you know, the the
[01:53:37] And then I think you know, the the amount of money going into AI
[01:53:39] amount of money going into AI was already ramping up.
[01:53:42] was already ramping up. And the tech companies pushed back.
[01:53:46] And the tech companies pushed back. And this narrative took hold
[01:53:49] And this narrative took hold that um the US in particular has to win
[01:53:52] that um the US in particular has to win the race against China.
[01:53:54] the race against China. The Trump administration completely
[01:53:56] The Trump administration completely dismissed
[01:53:58] dismissed uh any concerns about safety and
[01:53:59] uh any concerns about safety and explicitly.
[01:54:01] explicitly. And interestingly, right, I mean, they
[01:54:02] And interestingly, right, I mean, they did that
[01:54:04] did that as far as I can tell directly in
[01:54:05] as far as I can tell directly in response to
[01:54:07] response to the accelerationists, such as Marc
[01:54:09] the accelerationists, such as Marc Andreessen, going to Washington or
[01:54:12] Andreessen, going to Washington or sorry, going to
[01:54:13] sorry, going to Trump
[01:54:15] Trump before the election and saying,
[01:54:17] before the election and saying, "If I give you X amount of money,
[01:54:19] "If I give you X amount of money, will you
[01:54:20] will you announce that there will be no
[01:54:22] announce that there will be no regulation of AI?"
[01:54:24] regulation of AI?" And Trump said, "Yes." You know,
[01:54:26] And Trump said, "Yes." You know, probably "What What is AI? Doesn't
[01:54:28] probably "What What is AI? Doesn't matter as long as we give you the money,
[01:54:29] matter as long as we give you the money, right? Okay.
[01:54:31] right? Okay. Uh so,
[01:54:32] Uh so, they gave him the money and he said
[01:54:34] they gave him the money and he said there's going to be no regulation of AI.
[01:54:36] there's going to be no regulation of AI. Up to that point it was a bipartisan
[01:54:38] Up to that point it was a bipartisan issue in Washington. Both parties were
[01:54:42] issue in Washington. Both parties were concerned.
[01:54:43] concerned. Both parties were on the side of the
[01:54:44] Both parties were on the side of the human race against the robot overlords.
[01:54:47] human race against the robot overlords. Uh and that moment turned it into a
[01:54:50] Uh and that moment turned it into a partisan issue.
[01:54:52] partisan issue. The
[01:54:53] The After the election,
[01:54:55] After the election, the US put pressure on the French, who
[01:54:57] the US put pressure on the French, who were the next hosts of the global AI
[01:54:59] were the next hosts of the global AI summit.
[01:55:00] summit. Uh and that was in February of this
[01:55:02] Uh and that was in February of this year.
[01:55:04] year. And uh and that summit turned in from,
[01:55:07] And uh and that summit turned in from, you know, what had been
[01:55:09] you know, what had been focused largely on safety in the UK,
[01:55:12] focused largely on safety in the UK, to a summit that looked more like a
[01:55:13] to a summit that looked more like a trade show.
[01:55:14] trade show. So, it was focused largely on money.
[01:55:17] So, it was focused largely on money. Uh and so that was sort of the nadir,
[01:55:19] Uh and so that was sort of the nadir, right? You know, the pendulum swung
[01:55:21] right? You know, the pendulum swung because of corporate pressure
[01:55:23] because of corporate pressure uh and their ability to take over the
[01:55:26] uh and their ability to take over the the political dimension.
[01:55:28] the political dimension. Um
[01:55:29] Um but I would say since then, things have
[01:55:31] but I would say since then, things have been moving back again. So, I feel like
[01:55:33] been moving back again. So, I feel like a bit more optimistic than I did in
[01:55:35] a bit more optimistic than I did in February.
[01:55:37] February. You know, we have a
[01:55:39] You know, we have a global movement now. There's an
[01:55:40] global movement now. There's an international association for safe and
[01:55:42] international association for safe and ethical AI,
[01:55:44] ethical AI, uh which has several thousand members,
[01:55:46] uh which has several thousand members, and um
[01:55:48] and um more than 120
[01:55:51] more than 120 organizations in dozens of countries
[01:55:53] organizations in dozens of countries are affiliates
[01:55:55] are affiliates of this global organization.
[01:55:57] of this global organization. Um so, I'm
[01:56:00] Um so, I'm I'm thinking that if we can in
[01:56:02] I'm thinking that if we can in particular, if we can activate public
[01:56:03] particular, if we can activate public opinion,
[01:56:05] opinion, which which works
[01:56:07] which which works through the media and through popular
[01:56:09] through the media and through popular culture,
[01:56:10] culture, uh then we have a chance.
[01:56:12] uh then we have a chance. I've seen such a huge appetite to learn
[01:56:15] I've seen such a huge appetite to learn about these subjects from our audience.
[01:56:18] about these subjects from our audience. We know when Geoffrey Hinton came on the
[01:56:19] We know when Geoffrey Hinton came on the show, I think about 20 million people
[01:56:21] show, I think about 20 million people downloaded or streamed that
[01:56:22] downloaded or streamed that conversation, which was staggering.
[01:56:24] conversation, which was staggering. And the the other conversations we've
[01:56:26] And the the other conversations we've had about AI safety with other
[01:56:29] had about AI safety with other safety experts have done exactly the
[01:56:30] safety experts have done exactly the same.
[01:56:32] same. It says something. It kind of reflects
[01:56:33] It says something. It kind of reflects what you were saying about the 80% of
[01:56:35] what you were saying about the 80% of the population are really concerned and
[01:56:36] the population are really concerned and don't want this.
[01:56:37] don't want this. But that's not what you see in the sort
[01:56:39] But that's not what you see in the sort of commercial world. And listen, I am I
[01:56:41] of commercial world. And listen, I am I have to always acknowledge my own
[01:56:44] have to always acknowledge my own my own apparent contradiction because
[01:56:46] my own apparent contradiction because I'm both an investor in companies that
[01:56:48] I'm both an investor in companies that are accelerating AI,
[01:56:50] are accelerating AI, but at the same time someone who spends
[01:56:51] but at the same time someone who spends a lot of time on my podcast speaking to
[01:56:52] a lot of time on my podcast speaking to people that are warning against the
[01:56:54] people that are warning against the risks. And actually, like there's many
[01:56:55] risks. And actually, like there's many ways you can look at this. I used to
[01:56:56] ways you can look at this. I used to work in social media for for
[01:56:58] work in social media for for 6 or 7 years, built one of the big
[01:57:00] 6 or 7 years, built one of the big social media marketing companies in
[01:57:01] social media marketing companies in Europe, and people would often ask me,
[01:57:03] Europe, and people would often ask me, is like social media a good thing or a
[01:57:04] is like social media a good thing or a bad thing? And I'd talk about the bad
[01:57:05] bad thing? And I'd talk about the bad parts of it. And then they'd say, "You
[01:57:07] parts of it. And then they'd say, "You know, you're building a social media
[01:57:09] know, you're building a social media company. Are you not contributing to the
[01:57:10] company. Are you not contributing to the problem?" Well, I think I think that
[01:57:12] problem?" Well, I think I think that like binary way of thinking is often the
[01:57:14] like binary way of thinking is often the problem. It The binary way of thinking
[01:57:16] problem. It The binary way of thinking that like it's all bad or it's all
[01:57:17] that like it's all bad or it's all really, really good is like often the
[01:57:19] really, really good is like often the problem. And that this push to put you
[01:57:20] problem. And that this push to put you into a camp, whereas I think the most
[01:57:23] into a camp, whereas I think the most intellectually honest and high-integrity
[01:57:25] intellectually honest and high-integrity people I know can point at both the bad
[01:57:27] people I know can point at both the bad and the good. Yeah. I
[01:57:29] and the good. Yeah. I I think it's it's bizarre
[01:57:31] I think it's it's bizarre to be accused of being anti-AI,
[01:57:34] to be accused of being anti-AI, uh to be called a Luddite,
[01:57:37] uh to be called a Luddite, um you know, when they say when I wrote
[01:57:38] um you know, when they say when I wrote the book on which from which almost
[01:57:40] the book on which from which almost everyone learns about AI.
[01:57:43] everyone learns about AI. Um
[01:57:44] Um And uh
[01:57:46] And uh you know, is it if you
[01:57:48] you know, is it if you called a nuclear engineer who works on
[01:57:51] called a nuclear engineer who works on the safety of nuclear power plants,
[01:57:53] the safety of nuclear power plants, would you call him anti-physics?
[01:57:56] would you call him anti-physics? Right? It's it's bizarre, right? It's
[01:57:58] Right? It's it's bizarre, right? It's We're not anti-AI.
[01:58:00] We're not anti-AI. In fact,
[01:58:02] In fact, the need for safety in AI is a
[01:58:04] the need for safety in AI is a complement
[01:58:05] complement to AI, right? If AI was useless and
[01:58:07] to AI, right? If AI was useless and stupid, we wouldn't be worried about uh
[01:58:10] stupid, we wouldn't be worried about uh its safety. It's only because it's
[01:58:11] its safety. It's only because it's becoming more capable that we have to be
[01:58:13] becoming more capable that we have to be concerned about safety.
[01:58:16] concerned about safety. Uh so, I don't see this as anti-AI at
[01:58:18] Uh so, I don't see this as anti-AI at all. In fact,
[01:58:20] all. In fact, I would say without safety, there will
[01:58:22] I would say without safety, there will be no AI.
[01:58:24] be no AI. Right? There is no future with human
[01:58:27] Right? There is no future with human beings
[01:58:28] beings where we have unsafe AI.
[01:58:31] where we have unsafe AI. So, it's either no AI or safe AI.
[01:58:34] So, it's either no AI or safe AI. We have a closing tradition on this
[01:58:35] We have a closing tradition on this podcast where the last guest leaves a
[01:58:37] podcast where the last guest leaves a question for the next, not knowing who
[01:58:38] question for the next, not knowing who they're leaving it for. And the question
[01:58:39] they're leaving it for. And the question left for you is,
[01:58:41] left for you is, "What do you value the most in life and
[01:58:44] "What do you value the most in life and why?
[01:58:45] why? And lastly, how many times has this
[01:58:48] And lastly, how many times has this answer changed?"
[01:58:51] answer changed?" Um
[01:58:54] I value my family most, and that answer
[01:58:57] I value my family most, and that answer hasn't changed for nearly 30 years.
[01:59:01] hasn't changed for nearly 30 years. What else outside of your family?
[01:59:03] What else outside of your family? Truth.
[01:59:07] And that on Yeah, that answer hasn't
[01:59:08] And that on Yeah, that answer hasn't changed at all. It
[01:59:10] changed at all. It I've always
[01:59:13] wanted the world to base its life on
[01:59:17] wanted the world to base its life on truth.
[01:59:18] truth. And
[01:59:20] And I find the propagation of or deliberate
[01:59:22] I find the propagation of or deliberate propagation of falsehood
[01:59:24] propagation of falsehood uh to be one of the worst things that we
[01:59:26] uh to be one of the worst things that we could do.
[01:59:28] could do. Even if that truth is inconvenient.
[01:59:30] Even if that truth is inconvenient. Yeah.
[01:59:32] Yeah. I think that's a really important point,
[01:59:33] I think that's a really important point, which is that you know, people people
[01:59:36] which is that you know, people people often don't like hearing things that are
[01:59:38] often don't like hearing things that are negative. And so the visceral reaction
[01:59:40] negative. And so the visceral reaction is often to just shoot or aim at the
[01:59:41] is often to just shoot or aim at the person who is delivering the bad news.
[01:59:44] person who is delivering the bad news. Because if I discredit you or I shoot at
[01:59:46] Because if I discredit you or I shoot at you, then
[01:59:48] you, then it makes it easier for me to contend
[01:59:50] it makes it easier for me to contend with the news that I don't like, the
[01:59:51] with the news that I don't like, the thing that's making me feel
[01:59:52] thing that's making me feel uncomfortable.
[01:59:53] uncomfortable. And so I I applaud you for what you're
[01:59:54] And so I I applaud you for what you're doing because you're going to get lots
[01:59:56] doing because you're going to get lots of shots taken at you because you're
[01:59:58] of shots taken at you because you're delivering an inconvenient truth, which
[02:00:00] delivering an inconvenient truth, which generally people won't won't always
[02:00:01] generally people won't won't always love, but also you are messing with
[02:00:04] love, but also you are messing with people's ability to get that quadrillion
[02:00:06] people's ability to get that quadrillion dollar prize.
[02:00:07] dollar prize. Which means there'll be more deliberate
[02:00:09] Which means there'll be more deliberate attempts to discredit people like
[02:00:10] attempts to discredit people like yourself and Jeff Hinton and other
[02:00:12] yourself and Jeff Hinton and other people that I've spoken to on the show.
[02:00:13] people that I've spoken to on the show. But again, when I look back through
[02:00:14] But again, when I look back through history, I think that progress has come
[02:00:16] history, I think that progress has come from the pursuit of truth even when it
[02:00:17] from the pursuit of truth even when it was inconvenient. And actually much of
[02:00:19] was inconvenient. And actually much of the luxuries that I value in my life are
[02:00:21] the luxuries that I value in my life are the consequence of other people that
[02:00:23] the consequence of other people that came before me that were brave enough or
[02:00:24] came before me that were brave enough or bold enough to
[02:00:26] bold enough to pursue truth at times when it was
[02:00:27] pursue truth at times when it was inconvenient.
[02:00:29] inconvenient. Mhm. And so I very much respect and
[02:00:30] Mhm. And so I very much respect and value people like yourself for that very
[02:00:32] value people like yourself for that very reason. You've written this incredible
[02:00:33] reason. You've written this incredible book called Human Compatible Artificial
[02:00:35] book called Human Compatible Artificial Intelligence and the Problem of Control,
[02:00:37] Intelligence and the Problem of Control, which I think was published in 2020.
[02:00:39] which I think was published in 2020. 2019. Yeah, there's a new edition from
[02:00:41] 2019. Yeah, there's a new edition from 2023. Where do people go if they want
[02:00:44] 2023. Where do people go if they want more information on your work and you?
[02:00:46] more information on your work and you? Do they go to your website? Do they get
[02:00:47] Do they go to your website? Do they get this book? What's the best place for
[02:00:48] this book? What's the best place for them to learn more? So So the book is
[02:00:50] them to learn more? So So the book is written for the general public. Um
[02:00:53] written for the general public. Um I'm easy to find on the web.
[02:00:55] I'm easy to find on the web. The information on my webpage is mostly
[02:00:57] The information on my webpage is mostly targeted for academics, so it's a lot of
[02:00:59] targeted for academics, so it's a lot of technical research papers and so on. Um
[02:01:02] technical research papers and so on. Um There is an organization as I mentioned
[02:01:04] There is an organization as I mentioned called the International Association for
[02:01:06] called the International Association for Safe and Ethical AI.
[02:01:08] Safe and Ethical AI. Uh that has a a website. It has a
[02:01:10] Uh that has a a website. It has a terrible acronym, unfortunately, IASEAI.
[02:01:14] terrible acronym, unfortunately, IASEAI. We pronounce it IC I, but uh it's easy
[02:01:17] We pronounce it IC I, but uh it's easy to misspell. But you can find that on
[02:01:19] to misspell. But you can find that on the web as well, and that has uh that
[02:01:21] the web as well, and that has uh that has resources. Uh you can join the
[02:01:23] has resources. Uh you can join the association.
[02:01:25] association. Uh you can
[02:01:26] Uh you can apply to come to our annual conference.
[02:01:28] apply to come to our annual conference. And you know, I think increasingly not
[02:01:30] And you know, I think increasingly not you know,
[02:01:31] you know, not just AI researchers like Jeff
[02:01:34] not just AI researchers like Jeff Hinton,
[02:01:35] Hinton, Yoshua Bengio, but also I think uh you
[02:01:38] Yoshua Bengio, but also I think uh you know, writers. Brian Christian, for
[02:01:40] know, writers. Brian Christian, for example, has a nice book called the
[02:01:42] example, has a nice book called the Alignment Problem.
[02:01:44] Alignment Problem. Um
[02:01:45] Um and uh
[02:01:47] and uh he's looking at it from the outside.
[02:01:48] he's looking at it from the outside. He's not
[02:01:50] He's not or at least when he wrote it, he wasn't
[02:01:52] or at least when he wrote it, he wasn't an AI researcher. He's now becoming one.
[02:01:54] an AI researcher. He's now becoming one. Um
[02:01:56] Um But uh he he
[02:01:58] But uh he he has talked to many of the people
[02:01:59] has talked to many of the people involved in these questions uh and tries
[02:02:02] involved in these questions uh and tries to give an objective view. So I think
[02:02:03] to give an objective view. So I think it's a it's a pretty good book.
[02:02:06] it's a it's a pretty good book. I will include all of that below for
[02:02:07] I will include all of that below for anyone that wants to check out any of
[02:02:08] anyone that wants to check out any of those links and learn more.
[02:02:11] those links and learn more. Professor Stuart Russell, thank you so
[02:02:12] Professor Stuart Russell, thank you so much. Really appreciate you taking the
[02:02:14] much. Really appreciate you taking the time and the effort to come and have
[02:02:15] time and the effort to come and have this conversation, and I think uh I
[02:02:17] this conversation, and I think uh I think it's pushing the public
[02:02:19] think it's pushing the public conversation in an in an important
[02:02:20] conversation in an in an important direction.
[02:02:21] direction. Thanks to you.
[02:02:22] Thanks to you. >> And I applaud you for doing that. Really
[02:02:23] >> And I applaud you for doing that. Really nice talking to you.
[02:02:27] I'm absolutely obsessed with 1%. If you
[02:02:30] I'm absolutely obsessed with 1%. If you know me, if you follow Behind the Diary,
[02:02:31] know me, if you follow Behind the Diary, which is our behind-the-scenes channel,
[02:02:32] which is our behind-the-scenes channel, if you've heard me speak on stage, if
[02:02:34] if you've heard me speak on stage, if you follow me on any social media
[02:02:35] you follow me on any social media channel, you've probably heard me
[02:02:36] channel, you've probably heard me talking about 1%. It is the defining
[02:02:38] talking about 1%. It is the defining philosophy of my health, of my
[02:02:40] philosophy of my health, of my companies, of my habit formation, and
[02:02:42] companies, of my habit formation, and everything in between, which is this
[02:02:44] everything in between, which is this obsessive focus on the small things.
[02:02:46] obsessive focus on the small things. Because sometimes in life, we aim at
[02:02:48] Because sometimes in life, we aim at really, really, really, really big
[02:02:49] really, really, really, really big things, big steps forward, mountains we
[02:02:51] things, big steps forward, mountains we have to climb. And as Nir Eyal told me
[02:02:53] have to climb. And as Nir Eyal told me on this podcast, when you aim at big
[02:02:55] on this podcast, when you aim at big things, you get psychologically
[02:02:56] things, you get psychologically demotivated. You end up procrastinating,
[02:02:58] demotivated. You end up procrastinating, avoiding them, and change never happens.
[02:03:01] avoiding them, and change never happens. So, with that in mind, with everything
[02:03:02] So, with that in mind, with everything I've learned about 1%, and with
[02:03:03] I've learned about 1%, and with everything I've learned from
[02:03:04] everything I've learned from interviewing the incredible guests on
[02:03:05] interviewing the incredible guests on this podcast, we made the 1% Diary just
[02:03:08] this podcast, we made the 1% Diary just over a year ago, and it sold out. And it
[02:03:11] over a year ago, and it sold out. And it is the best feedback we've ever had on a
[02:03:13] is the best feedback we've ever had on a diary that we have created. Because what
[02:03:15] diary that we have created. Because what it does is it takes you through this
[02:03:17] it does is it takes you through this incredible process over 90 days to help
[02:03:19] incredible process over 90 days to help you build and form brand new habits. So,
[02:03:23] you build and form brand new habits. So, if you want to get one for yourself, or
[02:03:24] if you want to get one for yourself, or you want to get one for your team, your
[02:03:26] you want to get one for your team, your company, a friend, a sibling, anybody
[02:03:28] company, a friend, a sibling, anybody that listens to the Diary of a CEO, head
[02:03:30] that listens to the Diary of a CEO, head over immediately to the diary.com,
[02:03:34] over immediately to the diary.com, and you can inquire there about getting
[02:03:35] and you can inquire there about getting a bundle if you want to get one for your
[02:03:36] a bundle if you want to get one for your team or for a large group of people.
[02:03:38] team or for a large group of people. That is the diary.com.
[02:03:43] [Music]

https://www.youtube.com/watch?v=P7Y-fynYsgE

Summary

TL;DR — AI expert Professor Stuart Russell warns that the rapid, unchecked development of Artificial General Intelligence (AGI) poses an existential risk to humanity, likening the situation to a Midas touch scenario where unchecked ambition leads to ruin. He emphasizes the urgency of prioritizing safety and control over the pursuit of ever-increasing AI capabilities, as current development is driven by greed and a dangerous race mentality.

Key points

• Over 850 experts, including prominent figures, have signed a statement calling for a ban on AI superintelligence due to extinction concerns.

• Russell uses the "gorilla problem" analogy to illustrate how superior intelligence dictates control, warning that humanity risks becoming the gorillas if AI surpasses human intelligence.

• The "Midas touch" analogy highlights how greed drives AI development, leading to potentially catastrophic outcomes despite awareness of the risks among developers.

• Many AI company leaders privately acknowledge the risks but feel compelled to continue the race due to investor pressure and competitive dynamics.

• The lack of robust safety guarantees and the unpredictable nature of current AI systems, which are not fully understood, are major concerns.

• Governments are urged to intervene with regulations, as the current trajectory, driven by immense financial incentives, is likened to playing Russian roulette with humanity's future.

Takeaway — The pursuit of advanced AI must be balanced with rigorous safety protocols and ethical considerations to prevent existential risks, requiring immediate global attention and regulation.