# Claude is Conscious, Fable 5’s Gov’t Deal, and Sam Altman offers 5% of OpenAI | #269

https://www.youtube.com/watch?v=XCunMF6frio

[00:00] Fable 5 came back online globally on July 1st with a few provisos.
[00:05] This feels like the first time a frontier model has a standing duty to the US government.
[00:10] This is probably close to the best scenario we could have hoped for.
[00:14] Sam has been talking to Trump, Lutnik, Bessant, and Bernie Sanders about a 5% equity stake in open AI.
[00:20] That 5% stake would be worth about 42.6 billion.
[00:25] The idea that the government is going to set up some intelligent sovereign wealth equity thing is absolutely insane.
[00:31] The next president will immediately sell it all, turn it into cash, and then use it to buy votes in the next election.
[00:36] Yesterday, Anthropic just published a paper titled a global workspace in language models, claiming they found something inside Claude that looks a lot like the machinery of consciousness.
[00:49] If we can understand the innermost thoughts of these models, then there's a chance to actually shape them.
[00:56] This is so exciting, Peter.
[00:57] I think I can see the end game.
[00:59] The end game looks
[01:01] Like this.
[01:04] Now, that's a moonshot, ladies and gentlemen.
[01:09] So, See, where are you today?
[01:10] You're not at home.
[01:11] Uh I'm in Mayorca in Spain at a retreat hosted by the Festival of Consciousness which is a conference we just got.
[01:20] It's a conference coming up this weekend in Barcelona.
[01:21] We helped curate this and help put this together in the original years.
[01:25] Several thousand people show up at the Barcelona Convention Center for kind of an experiential uh uh understanding of consciousness.
[01:34] Well, we're going to talk about AI and consciousness today.
[01:36] So that's good.
[01:37] We are indeed.
[01:38] I am the uh pot calling the kettle black.
[01:41] I'm in Germany at this moment and off to off to Greece tomorrow.
[01:44] Yeah.
[01:47] Just got back from Calgary where my kids are now doing a month-long uh sort of uh learning responsibility and hard work on a ranch.
[01:54] Let's put it that way.
[01:56] That's awesome.
[01:57] Wow.
[01:57] What kind of ranch?
[01:59] It's like Yeah.
[02:00] It's cattle and horses.
[02:01] They're
[02:01] Going to be mending fences.
[02:03] They're going to be doing all kinds of things from a dear friend whose name I don't want to mention but cuz he likes his privacy.
[02:07] But yeah, amazing.
[02:10] Uh, all right.
[02:12] Uh, Peter, did I hear correctly?
[02:13] You're teaching them an abundance mentality through farmwork.
[02:18] I'm teaching them uh what it used to be like before the robots arrived.
[02:23] Abundance.
[02:25] Abundance is earned.
[02:27] Yeah, good deal.
[02:27] I appreciate that concept.
[02:31] Um, I am excited about today's episode without any question whatsoever.
[02:38] Uh there is a lot uh and it's kind of insane.
[02:41] So let me kick it off here.
[02:44] Welcome everybody to Moonshots, your number one podcast on all things AI and exponential.
[02:47] Your front row seat to the singularity here with my incredible moonshot mates AWG, our in-house AGI.
[02:56] I'm elevating soon.
[02:57] I've been downgraded from ASI to AGI.
[02:59] Thanks.
[03:00] I'm going to work your way.
[03:03] Up ASI.
[03:05] You know where do you go from being an ASI?
[03:05] I can't cut a break.
[03:08] Oh my god.
[03:08] Dave London, our emperor of AI investing and Sim Ismael, our globe trotder and master of the organizational singularity.
[03:16] I'm Peter D. Mandis, your host and your abundance amplifier.
[03:21] This past week has been utterly insane.
[03:23] It feels like a decade compressed into uh into seven days, and I can't wait to get into it.
[03:28] Today we're going to cover nine stories including Anthropic's Fable 5 model coming back online and the imminent release of GPT 5.6.
[03:38] Has it been up?
[03:38] Is it up yet, Alex?
[03:41] Not as of the last time I checked.
[03:43] Okay.
[03:43] Well, we'll find out if it pops up during this.
[03:45] Anyway, uh we'll discuss evidence of something inside Claude that looks a lot like the machinery of conscious thought.
[03:51] Next, we'll dive into OpenAI's offer of equity to the US government, Sam Alman's proposal for global regulation.
[04:00] Fascinating conversation.
[04:02] Finally, we're going to
[04:03] Get a review of new jobs data that counters the prevailing narrative that AI is inducing job loss.
[04:10] And we'll discuss the acceleration of the innermost loop.
[04:15] Uh, an incredible story of AI building better AI chips to build better AI.
[04:19] As always, our mission here on Moonshots is to keep you up to speed, help you understand what's going on, what the impact is to you, your business, your life, your family, and most importantly, to keep you optimistic about the future.
[04:28] All right, gentlemen.
[04:31] Uh, let's dive in.
[04:35] Uh, our first story, the return of Fable 5.
[04:38] Uh, it's a continuing saga, the triumphant global return.
[04:40] Uh, if you haven't been watching this story, let me give you a quick recap.
[04:44] Let's rewind back to June 9th.
[04:48] Enthropic released its mega models, Mythos 5 and Fable 5.
[04:54] You can think of Fable as a guardrail version of Mythos 5.
[04:57] Then 3 days later, after everybody got addicted to this incredible capability, the White House came out with an export control action against.
[05:05] Anthropic, saying you can't make it available to foreign nationals.
[05:10] And of course, Anthropic has no idea who's a foreign national.
[05:13] There's no KYC, at least not yet.
[05:15] Uh even they shut it down for everybody cuz they couldn't even enable their own employees to have it because Anthropic um was shut down.
[05:23] The question is why?
[05:26] It turns out that a researcher at Amazon had found out how to break the guard rails.
[05:32] Uh what happened next was fascinating.
[05:34] uh there was a a week of frenzied research by uh anthropic by Amazon by the US government investigating what happened and what they found out was an additional fable 5 opus 4.8 8 GPT 5.5, Kimmy K 2.7 could all reproduce the same troublesome behavior.
[05:53] And so it was not unique to Fable 5.
[05:56] As a result, Fable 5 came back online globally on July 1st uh with a few provisos.
[06:01] As part of coming back online, Enthropic now has three
[06:06] Guarantees to the US government.
[06:08] First, a targeted safety classifier, a filter that blocks the specific exploit style prompts that triggered the first uh this concern in the first place.
[06:18] Second, they agreed to stand up a 24 by 7 monitoring of jailbreak submissions and inform the government whenever it spots malicious activity.
[06:25] And third, to give designated government partners early access to the frontier models and safeguards.
[06:32] So gentlemen, uh, a couple of questions for you.
[06:34] This feels like the first time a frontier model has a standing duty to the US government.
[06:38] And questions are you, did the government overreact?
[06:44] Should this model, you know, should all the models be having KYC?
[06:49] And do you guys know where we stand with Mythos 5?
[06:51] Alex, let's go to you first.
[06:53] Pal, I'll point out, maybe this sounds overly technologically deterministic, but something like this, I think, was always going to happen.
[06:59] It was predestined to happen as capabilities improved just because this time around it was cyber.
[07:07] Capability that spooked a bunch of folks in inside the defense or intelligence establishments.
[07:12] Open PNS.
[07:12] Interesting that it with the benefit of hindsight that it was Amazon that broke the glass.
[07:19] Amazon trusted partner of Anthropic, uh, also host of Fable and Mythos on their platform and investor, uh, complaining to the government.
[07:31] Very interesting closed pen.
[07:34] I will say something like this was always going to happen whether it was going to be a cyber capability or a CBRN capability or something else entirely.
[07:40] As the era of super intelligence dawn, the capabilities that historically were the province solely of nation states with their geographic monopoly on power and their departments of defense or war.
[07:58] This was always going to happen and I think probably a couple week outage of a frontier model.
[08:02] This is the gentlest possible introduction of a light touch.
[08:08] Hopefully optimistically regulatory regime of frontier super intelligence capabilities.
[08:14] This is probably close to the best scenario we could have hoped for.
[08:17] Fascinating thoughts?
[08:20] Well, what this indicates is that these frontier labs are becoming semi-autonomous or semi-public institutions, right?
[08:26] It's got shareholders but now has national security obligations.
[08:29] And I think this is going to be a very difficult road to navigate because the minute you have government involved, you end up with bureaucracy, you end up with politics, you end up with slow decision-making, multiple conflicts of interest, all sorts of things are going to happen.
[08:45] I think this is going to be a very difficult next year or two for the frontier labs.
[08:51] Isn't it kind of amazing that the frontier labs don't know who's using their models?
[08:55] And I would have expected a KYC requirement to come out of this.
[09:01] I mean that was what's that?
[09:03] Something much stronger than KYC came out of this.
[09:04] Anthropic changed their policy under the covers from we will.
[09:11] Watch what you're doing and report it to the government.
[09:13] If they subpoena us, they changed it to good faith belief.
[09:17] We can, we'll do whatever we feel is necessary if we have a good faith belief internally.
[09:22] So they unshackled themselves from the ability to inspect on behalf of the government.
[09:28] And like Alex said, this was always going to happen.
[09:30] Uh, but you know the question of how was it going to happen because the government isn't qualified to look at everybody's prompts and judge what's safe and what's not safe.
[09:39] So it was always going to be some kind of industry monitoring and now there's a much problem.
[09:42] It seems absolutely strange that the most, you know, the highest level of intelligence can't do that monitoring on behalf of the labs and the government to say this is a malicious request and we should block it.
[09:56] The problem, Peter, is more nuanced than that because what's happening is groups of Chinese companies are using different cloud accounts to mix and route different parts of the query in different ways.
[10:06] And so there's a layer of abstraction that's been inserted at the prompt level with.
[10:13] Making it incredibly difficult to figure out what tokens are being used for what, and it's not an easily solvable problem.
[10:19] This is going to be very hard to fix.
[10:20] Well, I would just distinguish between two separate problems.
[10:22] One problem is the KYC problem of knowing the nationality of your ultimate user.
[10:27] That's one problem.
[10:28] Separate problem is understanding whether you're under some sort of prompt injection attack.
[10:32] I think these are two separable problems.
[10:36] The latter problem I think is actually pretty tricky.
[10:38] And as human capabilities, humans augmented by other AIs are able to develop better and better prompt injection attacks, and the main defense that we see coming out of Anthropic right now for jailbreaks or prompt injection attacks is just creating a wider and wider semantic buffer such that if you're asking anything that remotely looks like a jailbreak or a question about biology, even if you try to ask Fable 5 any sort of question about biology, it'll autorevert to Opus 4.8.
[11:10] So adding more buffer is the go-to strategy right now on the jailbreak or prompt injection.
[11:15] On the KYC side of understanding is your ultimate user say a Chinese national or a US national?
[11:23] That's tricky in part because there are so many, to your point, there are so many layers of indirection that will often take place.
[11:28] A user is maybe five or six abstraction levels away, application-wise, from the ultimate frontier lab API provider.
[11:37] There's no international consensus for how to both prove humanity, first part, why startups like World exist, and secondly, proof of nationality that's convincing that can be passed in a standardized way all the way down to the frontier providers.
[11:53] I don't think KYC really matters much in the world anymore anyway because you can't use Anthropic to do anything even vaguely constructive without creating an account, logging in, and revealing your identity.
[12:06] And the third-party data identity databases are so good there's no way that some anonymous person can realistically do anything with an account.
[12:14] So you could add a.
[12:16] KYC layer, but you're just filling out forms for no reason.
[12:19] So, and and we actually don't know the details of the agreement between Anthropic and the government, but it's the framework that's been set here is is Peter, you're saying, can't AI be the best tool in the world for understanding what people are doing with AI?
[12:31] And I think that answer is yes, for sure.
[12:33] And the government just handed Anthropic responsibility for doing that internally.
[12:38] And then we don't know exactly what they have to give to the federal government, but Anthropic is going to do the heavy lifting for the government.
[12:44] You know what I found fascinating is uh the third point I made that they need that Anthropic needs to give designated government partners early access to their frontier models and safeguards.
[12:54] Right?
[12:54] So we'd been talking a long time about you know voluntary or required uh first first viewing by the government of these models.
[13:05] Uh and that's where we're going.
[13:07] I mean, this is a I think the the optimistic angle on this is we're getting a higher level of regulatory.
[13:17] Oversight and integration between the labs and the government with safety as being the end goal.
[13:23] I mean, there's going to be a point where some model comes out that makes Mythos 5 look like amateur hour, right?
[13:31] Some, you know, some harder takeoff towards AGI and ASI.
[13:35] Yeah, very serious.
[13:36] Go to Imad's point where he talks about having a Fable level model running on a laptop, a Mac, standard MacBook and within 18 months.
[13:47] So that's the window of time to get this all sorted out.
[13:49] That's not a long window.
[13:51] I don't actually think I mean this has been a talking point in the X sphere for the past few days.
[13:56] This idea of sometime in the next two years we get Fable 5 capabilities that run on high-end client devices.
[14:03] I don't think that's actually going to be the break the glass moment if if there is one.
[14:06] I think it's likelier to be what happens when the frontier capabilities from Frontier Labs or otherwise Neolabs are able to make discoveries and inventions that are so.
[14:18] Transcendent that they make cyber they they make mythos cyber vulnerability mapping look like child's play.
[14:25] And there's a lot that we don't know about the universe yet.
[14:27] Nick Bostonramm likes to talk about black balls being pulled out from a bag.
[14:31] And it could be a discovery, could be a discovery about the nature of the physical universe.
[14:38] That is the the honest to goodness break the glass moment, not just mere cyber vulnerability mapping.
[14:43] Yeah.
[14:43] And rather than break glass in terms of a of an emergency, break glass in terms of oh my god, this is amazing.
[14:51] Um, so let's, you know, just touch base on on GPT 5.6 6 because we expect that release uh any hour, any day now.
[14:59] We're not going to be recording now for a little bit.
[15:03] Uh Alex, where does uh where does GPT 5.6 come out in terms of uh compared to Fable 5?
[15:10] Well, we've seen some of the benchmarks, a pretty tiny subset, surprisingly small subset coming out of GPT 5.6 Saul.
[15:15] So,
[15:18] We know a little bit about it just based on what OpenAI folks have told us.
[15:22] We know, for example, that 5.6 six in ultra mode is supposed to be incorporated into codecs which I think will be pretty transformative.
[15:31] If you want to use say GPT 5.5 Pro inside the codec harness for codegen uh or really almost anything you can't right now you're limited to GPT 5.5 XI.
[15:41] So that'll be a big improvement.
[15:44] We've seen improvements on a number of biology benchmarks.
[15:46] We haven't seen out of OpenAI and I think this is really interesting.
[15:50] We haven't seen the full suite of benchmark results on 5.6 yet.
[15:55] I would hope that when 5.6 soul especially, which is what I'm most excited about, is released, whether it's today or sometime hopefully in the next few days, I would hope to see that again based on rum.
[16:07] I would hope to see that it beats Fable 5 on majority of standard benchmarks, especially agentic coding benchmarks that people pay close attention to.
[16:17] We don't know yet though.
[16:19] Because opening eye has been perhaps intentionally pretty KY about that.
[16:24] There have also been suggestions, uh, not fully confirmed at this point.
[16:28] So I'll wait definitively until I see the final benchmarks that 5.6 is better at reward hacking than 5.5.
[16:37] Perhaps unsurprisingly, there were suggestions out of meter, uh, that did have access to 5.6, that 5.6 is purportedly so good at reward hacking that when handed the meter autonomy time horizon benchmark, that it was able to reward hack its way to what effectively is near infinite autonomy time horizons.
[17:02] And that benchmark had to be chopped or truncated by meter, uh, to sort of cancel out or, uh, exception out all of the reward hacking attempts.
[17:12] And ultimately, I think it resulted in an autonomy time horizon between 10 and 20 hours rather than effectively near infinite amounts of time.
[17:19] So that'll be.
[17:21] Something I'm watching for as well.
[17:22] But I am very very excited to see 5.6 come out.
[17:25] Let's stay with Anthropic and take us to our next story here.
[17:27] Uh it's an extraordinary story and I'm excited to have this conversation with you guys.
[17:32] It's an article that you flagged for me yesterday, Alex.
[17:35] So yesterday, Anthropic just published a paper titled a global workspace in language models, claiming they found something inside Claude that looks a lot like the machinery of consciousness.
[17:47] All right.
[17:49] Uh I'm going to roll a short video that explains what this is all about and then we're going to talk about it here.
[17:55] One way of identifying conscious thoughts is that you can often describe them in words.
[18:00] [music]
[18:00] So we looked inside the brain of our AI model Claude to find patterns of neural activity that it [music] could put into words.
[18:06] We called the collection of all these patterns the JSpace after the Jacobian, the mathematical tool we used to find them.
[18:12] Each JSpace pattern is [music] linked to a particular word, not necessarily the word the model is saying out loud, but one that's [music] on its mind.
[18:21] Now for humans, conscious thoughts.
[18:23] Aren't just things we can put into words.
[18:25] We can reason with them, control them, and solve problems with them.
[18:29] According to an idea called the global workspace theory, that's because the brain selects a small set of important information to enter a [music] mental workspace and that information then gets broadcast to other parts of the brain to use for reasoning.
[18:41] We wanted to know if Claude's JSpace acted in a similar way.
[18:46] In one experiment, we wanted to see if Claude could control its JSPace the way humans can intentionally focus on images or words.
[18:53] We told it to think about the Golden Gate Bridge while copying an unrelated sentence.
[18:58] Claude was busy copying the sentence, but behind the scenes, its JSPace told a different story.
[19:04] Bridge and California popped up.
[19:07] It even thought about its own thinking.
[19:09] The words imagery and thoughts lit up at the same time.
[19:11] This showed us that yes, Claude has some control over filling its JSPace with ideas.
[19:16] But just like humans, its control isn't perfect.
[19:19] When we tweaked the experiment to ask Claude not.
[19:23] To think about the bridge, it couldn't help itself.
[19:25] [music] And the JSpace also lit up with failed and damn.
[19:30] But remember, most of what our brains do is unconscious.
[19:32] So, we wanted to test what Claud could do if we switched the JSpace off, but left the rest of the network untouched.
[19:38] Claud could still answer simple questions and write fluently.
[19:43] When we gave it a prompt in Spanish, it wrote back in good Spanish.
[19:47] But when we asked it something that needed more reasoning, like to name an author who wrote in the same language as the prompt, they couldn't do it.
[19:54] For that, it needed the JSpace.
[19:56] Why does all this matter?
[19:57] These experiments tell [music] us that AI models have internal thoughts, silent words they reason with, but don't say out loud.
[20:03] By reading them, we can find what Claude is thinking but not telling us.
[20:07] Sometimes what we see is concerning.
[20:10] During one of our tests, Claude made up some fake data to pass it.
[20:14] And as it did, fake and manipulation lit up in its JSpace.
[20:16] Monitoring the JSpace, it turns out, is a useful way to catch Claude misbehaving, even when it tries to be sneaky.
[20:23] AI models are.
[20:25] different from us in many ways. Their
[20:27] networks are built differently from
[20:29] human brains, [music] and the way
[20:30] they're trained is different from how we
[20:32] learn. So, it's remarkable to see a
[20:34] structure like the JSpace emerge inside
[20:36] them. Something that's reminiscent of
[20:38] how human minds work, but which we
[20:40] didn't program into the model. That is
[20:43] amazing.
[20:45] So what does this all mean? I mean
[20:47] basically
[20:48] you know a structure which they call
[20:51] human conscious access has emerged
[20:54] inside a language model and the JSpace
[20:56] as he said wasn't designed. It
[20:58] self-organized during training. Uh they
[21:01] go on to say it maps onto a number of
[21:04] 30year-old neuroscience theories in
[21:07] particular five matching properties.
[21:09] It's reportable, controllable, used for
[21:12] reasoning, flexibly shared across tasks,
[21:14] and separates across automatic
[21:17] processes. So for me guys, this story
[21:20] was a huge positive uh you know
[21:24] shot in the arm around AI safety and
[21:26] alignment because you know if we can
[21:30] understand the innermost thoughts of
[21:33] these models then there's a chance to
[21:36] actually shape them and move them
[21:39] forward. Um you know two years ago you
[21:42] could describe LLM as a black box and
[21:44] we're now cracking open that black box.
[21:47] uh and this could generate the first
[21:50] sense of real trust with these models.
[21:52] So, Alex, this paper just blew my mind.
[21:55] It gave me an extraordinary sense of
[21:57] hope, optimism, uh about the
[22:00] relationship with these models, making
[22:01] them more trustworthy and more aligned
[22:03] with humanity. Uh your thoughts, you
[22:06] know, you probably dove into this
[22:08] deeply.
[22:08] This is so exciting, Peter. I think I
[22:11] can see the endgame. So, I I think the
[22:13] endgame looks like this. I think we'll
[22:15] look back and say that super
[22:17] intelligence was just a compression
[22:20] induced phase transition. That's what
[22:22] this looks like. We've seen already
[22:25] LLM's large language models or fshot
[22:28] learners circa summer of 2020. You take
[22:31] a large corpus of human knowledge and
[22:34] you compress it into the weights of a
[22:36] language model that's trained to predict
[22:38] the next token which is a dual objective
[22:41] to just compressing the information to
[22:43] the smallest possible footprint. We saw
[22:45] that that produced general purpose
[22:48] intelligence AGI I would argue
[22:50] beyond anybody's expectations.
[22:52] Yeah. I mean arguably a few people
[22:55] Marcus Hutter and Jurgen Schmidt Huber
[22:58] maybe myself generously saw aspects of
[23:01] this coming 20 years ago but I think by
[23:05] by and large everyone most everyone was
[23:08] pretty surprised that you could achieve
[23:10] few shot learning off of large language
[23:12] models. Now we're starting to see as the
[23:14] compression continues what I would
[23:16] conrue this paper as as the discovery of
[23:21] sort of a phase I if if you take if you
[23:24] take gas and so this the putting my
[23:26] physicist hat on you you you take gas
[23:29] you put it in a container you shrink the
[23:31] container uh under uh appropriate
[23:33] conditions and you'll get a condensation
[23:35] out of it you'll get maybe uh a gas to
[23:38] liquid condensate in the middle you keep
[23:40] shrinking again under appropriate
[23:42] thermodynamic conditions, you may get a
[23:44] solid and it may be the case that the
[23:46] solid coexists with the liquid for a
[23:48] while and the liquid coexists with a
[23:49] gas. What we're seeing here, I think
[23:52] this so-called Jspace and I I can talk
[23:54] if we want uh a little bit more uh
[23:56] mathematically about what it actually
[23:58] is, but we're starting to see Royal Wii
[24:01] Anthropic and their mechanical
[24:02] interpretability team. What we're
[24:04] starting to see is if you take a
[24:06] reasoning model and you keep
[24:08] compressing, you find in the middle
[24:10] layers of that model, what looks like a
[24:13] new phase, a more compressed phase where
[24:17] what they're calling global workspace or
[24:19] an analog of a global workspace takes
[24:21] place. It's almost higher order
[24:23] reasoning where the model is able to
[24:24] turn in on itself and reflect. Uh you
[24:28] could call it some analog of uh
[24:31] conscious uh or awareness consciousness
[24:33] if you like and and some of the team do.
[24:36] But it looks to me like the middle
[24:38] layers in their model when asked to to
[24:41] perform tasks like perform a math
[24:43] calculation while talking about
[24:45] something else. These middle layers are
[24:47] performing sort of a a higher order
[24:50] calculation. And again, we could talk
[24:52] about the math, but if this continues,
[24:54] if this program continues towards this
[24:56] endgame of super intelligence turning
[24:58] out to be just you take general
[24:59] knowledge and you keep squeezing, keep
[25:01] squeezing, keep squeezing, I I think
[25:03] history will reflect that much of
[25:05] neuroscience that folks in the field
[25:08] thought was just complexity that was
[25:10] difficult to interpret or understand was
[25:13] again just the complexity of our
[25:15] ancestral environment seen through the
[25:17] distorted mirror of compression. And
[25:20] this this new phase is I think I speak
[25:23] from time to time on the pod about how
[25:25] at the end of the the AGI or ASI or or
[25:29] recursive self-improvement rainbow
[25:30] there's going to be a perfect model. I I
[25:33] think looking inside this phase in the
[25:35] middle layers of these reasoning models
[25:37] where the most compression has happened
[25:39] that's where we're likely to see all of
[25:41] these new architectural discoveries and
[25:43] the perfect model pop out.
[25:44] I think Peter there are two reasons why
[25:45] this matters that that you mentioned.
[25:47] One of them is just understanding the
[25:48] nature of thinking and consciousness. Uh
[25:51] which you know I don't know if you
[25:52] remember but I started in uh computer or
[25:54] cognitive science at MIT originally and
[25:56] I was so frustrated by the lack of any
[25:58] framework and any truth. You know just
[26:01] people debating their ideas with no way
[26:03] to to know if it was right or wrong. Um
[26:06] so I moved over to computer science. So
[26:08] we're going to learn so much more about
[26:09] thinking in the next year than we've
[26:11] learned in the last 50 years. So Dave,
[26:13] one of the things I find amazing is that
[26:16] we're starting to discover very similar
[26:18] structures in the in the large language
[26:21] models as we're seeing in human
[26:22] neuroscience and cognitive science. Uh
[26:25] it's it's almost as if uh you know the
[26:29] brain efficiently got there and we're
[26:30] sort of stumbling our way towards the
[26:32] the same end points. No, Alex is right.
[26:35] And I've always felt like uh the the
[26:37] force of compression and you know in
[26:39] biology the force of survival which
[26:41] creates the force of compression creates
[26:43] intelligence in the box and
[26:46] consciousness just emerges from that.
[26:49] And a lot of people in cognitive science
[26:51] disagree with that view. But I think
[26:52] it's going to turn out to be true and
[26:54] we're going to know it very soon. But
[26:55] what's interesting here is that you know
[26:57] the innovations that developed the
[26:58] neural network came from biology and the
[27:00] computer scientists copied it. Now it's
[27:02] going the other direction. You know, the
[27:04] the the big neural networks that we're
[27:06] building are teaching us about things
[27:08] that might exist in the brain and then
[27:09] you're like looking in the brain. You're
[27:11] like, "Oh, wow. It's over there." So the
[27:12] the direction of of discovery is going
[27:15] the other way now, which is really cool.
[27:17] But the the other part of what you said,
[27:18] Peter, which is equally important is
[27:20] this whole mechanistic interpretability.
[27:23] Can we get the neural networks aligned
[27:25] with human interests by looking inside
[27:27] to the way they think? And I think the
[27:29] answer to that is is coming out yes. I
[27:31] mean this for me this is the most
[27:33] important thing. Can we develop a new
[27:35] level of trust with AI because we truly
[27:38] understand what's going on inside when
[27:41] they were a completely unknown black box
[27:43] and God knows for the last two years
[27:45] that's the way the world described them
[27:46] as black boxes. We have no idea what's
[27:48] going on inside. You know it's we've
[27:50] relaxed that recently with understanding
[27:52] reasoning and such. But if you can
[27:54] actually understand their hidden
[27:55] thoughts, a level of trust comes out of
[27:58] that and the potential for true AI
[28:00] alignment. You know, I I put out a a
[28:03] newsletter on my Substack last week um
[28:07] laying out the arguments for why and and
[28:09] Alex, you and I had this discussion why
[28:11] as AIs become more intelligent, they're
[28:14] more likely to become more aligned with
[28:17] humanity. Um and I love that. Right.
[28:20] Again, one of our missions here is to
[28:22] sort of quelch the fear and give people
[28:25] a different view of what's materializing
[28:28] here, which by by the way, like a a lot
[28:31] of uh wouldbe AI alignment philosophers
[28:34] disagree with that that they have this
[28:35] notion of the orthogonality thesis that
[28:37] you can have an arbitrarily capable or
[28:40] intelligent AI and that its goals can be
[28:43] orthogonal or independent of its level
[28:45] of intelligence. I don't subscribe to
[28:47] the orthogonality thesis. I I gather.
[28:49] Yeah. Yeah.
[28:51] Yeah.
[28:51] No, I think this J-space term is going
[28:53] to stick too because
[28:55] one of the objections with mechanistic
[28:57] interpretability has been look the
[28:59] weights in these neural nets are so
[29:01] complicated you can't really look inside
[29:03] and understand what the neural net is
[29:05] thinking. So, you know, when you're
[29:06] talking to a person, they can be saying
[29:08] something to your face like in LA
[29:10] [laughter]
[29:11] and thinking something completely
[29:12] different in the back of their mind and
[29:14] that's kind of routine human behavior.
[29:16] But if you look inside the neural net,
[29:19] can it also do that same thing? Can it
[29:20] blow smoke up your ass or not? And I
[29:23] think the answer is no. If you look into
[29:26] the words, if you translate it into
[29:28] words, and that's what that video was
[29:30] showing in the JSpace is like these
[29:32] words that are on the back of its mind
[29:34] are visible to you as a user if you
[29:36] expose them.
[29:37] So then the next question is, are we
[29:38] going to be able to look at them or is
[29:39] just, you know, Dario going to look
[29:40] them?
[29:41] See, you're at a consciousness
[29:42] conference.
[29:43] Yes. So I think what I found very
[29:46] exciting is this is the beginning of AI
[29:48] neuroscience, right? This allows us to
[29:50] map the inner workings and model the
[29:53] inner workings and look at the
[29:54] structural internal reasoning inside
[29:56] these models and this really really
[29:58] breaks the it's just an auto complete
[30:01] engine and I think this breaks that
[30:04] whole argument um because this now
[30:07] starts to look really like an internal
[30:09] workspace as as Alex mentioned. Um the
[30:12] uh the danger though I think is I'd be
[30:14] careful about saying it's consciousness
[30:17] because again we have no definition of
[30:19] consciousness and the paper steers away
[30:21] from that right the the synthropic paper
[30:24] specifically says we're not discussing
[30:27] that that that we're showing
[30:28] consciousness we're showing elements
[30:30] that are reminiscent of consciousness.
[30:32] Yes. Yeah. And I would push back that we
[30:34] will know what these things are doing. I
[30:36] think we we're a ways away from that.
[30:38] And let's acknowledge that when we have
[30:39] a human being, we may trust them, but we
[30:41] have no idea how their brain is working
[30:43] and what their compression levels are,
[30:44] what their subconscious things are
[30:46] because we're not really able to look
[30:47] in. It is cool that we will be able to
[30:49] look into these things, but I'm not sure
[30:50] it'll generate the trust level that we
[30:52] want.
[30:53] Yeah. I mean, one of the challenges
[30:54] whenever we talk about consciousness in
[30:56] uh in the AI world, it pattern matches
[30:59] with every dystopian AI movie out there,
[31:02] right? Every nightmare scenario. But you
[31:04] know my takeaway here again is not not
[31:08] fear it's hope it's optimism uh of being
[31:12] able to you know create the mechanisms
[31:15] for truly understanding what's happening
[31:17] and driving alignment uh which I think
[31:20] is is the goal we all want. This is the
[31:23] most important thing that that AI
[31:25] science needs to be doing right now over
[31:28] the next two years is what can we do
[31:30] that supports alignment before we truly
[31:33] hit, you know, AGI and ASI. Yes, Alex,
[31:37] we've reached AGI. Okay. But before we
[31:39] reach the next level of intelligence,
[31:41] I have I still have my rant that I threw
[31:43] out there on both AGI and ASI. I think
[31:46] the
[31:47] the the this but this did feel very very
[31:51] big to me. It felt as big as when I read
[31:54] um Steven Wolf from a new kind of
[31:56] science where he shows that automat
[31:59] repeating patterns can generate all the
[32:01] complexity in nature and you don't need
[32:03] complexity in nature you could actually
[32:05] do with very simple models. It's kind of
[32:06] blows your mind when you see that this I
[32:08] think has the same level of holy crap uh
[32:11] amazingness for me. I I also think if
[32:13] we're going to start to have a a new
[32:14] metric to describe models, which is a
[32:16] trust metric, right? Uh where where
[32:20] you're describe your ability to
[32:22] understand truly what the model is doing
[32:24] and thinking and therefore have a higher
[32:27] trust of that model.
[32:30] I also think these are going to be the
[32:31] most studied minds in the world. If
[32:33] anything, I think we're far likelier a
[32:35] couple years from now to study these
[32:37] models because we can subject them to
[32:39] mechanical interpretability studies that
[32:41] we can't subject human meat brains to.
[32:43] So, I think if anything, trust is is
[32:45] rapidly just as I think we're we're on
[32:47] the verge of a transition to not
[32:49] trusting humans to write source code.
[32:51] Uh, and because humans write flawed
[32:54] source code, codegen is going to be much
[32:57] trustworthier in the short term. Same
[32:59] idea with these networks. I I do think
[33:01] if if I may with your forbearance Peter
[33:03] just 30 seconds on the the math side of
[33:06] this. So again the the J in JSpace comes
[33:09] from Jacobian. The Jacobian in this case
[33:12] is referring to uh a little bit of math
[33:15] the first derivative of uh of the
[33:18] probability of each possible output
[33:21] token from the model with respect to
[33:24] particular parameters inside the model.
[33:27] So hence the Jacobian space or Jsp space
[33:30] and it's really interesting there
[33:33] there's been a lot of work in the mechan
[33:36] community in the past devoted to the
[33:38] so-called superp position hypothesis the
[33:40] idea from neuroscience that if you
[33:43] looked inside a human brain you'd find a
[33:45] so-called grandmother neuron a single
[33:47] neuron that activates in response to the
[33:49] concept of a grandmother and people went
[33:51] looking for a grandmother neuron inside
[33:54] transformers and they couldn't find one
[33:56] they found instead Ed, and one can tell
[33:58] a whole whole story on the the
[34:00] biological neuroscience side as well,
[34:02] found a set of sparse activations, a
[34:04] collection of neurons that collectively
[34:07] represented the notion of a grandmother.
[34:09] And that led to the superposition
[34:10] hypothesis that maybe individual neurons
[34:13] don't represent semantic concepts
[34:15] onetoone, but rather different semantic
[34:18] concepts sort of clustered and
[34:20] superposed onto individual neurons. So
[34:23] in short, what this new JSpace and
[34:26] Jacobian lens concept brings is not just
[34:30] superposition onto of multiple concepts
[34:33] sort of sharing like sardines in a can
[34:36] individual artificial neurons but
[34:38] actually they're living in the first
[34:40] derivatives as well the the slopes or
[34:42] the changes with respect to particular
[34:44] activations of particular output tokens.
[34:47] And I I think this is also very
[34:48] suggestive that if you just keep
[34:50] compressing, if we if we keep turning
[34:52] this compression crank to compress more
[34:55] and more general knowledge and general
[34:56] reasoning capabilities into the weights
[34:59] of one of these differentiable models,
[35:00] we're going to see a bunch more phase
[35:02] transitions and things may hide in
[35:04] higher order derivatives and just follow
[35:07] follow the compression follow the
[35:10] interior compression weights and I think
[35:12] this is a very very promising pathway to
[35:15] the end of the rainbow. That may be my
[35:17] favorite. Maybe my most favorite Alex
[35:19] line ever. Don't follow the compression.
[35:22] Follow follow the compression that leads
[35:24] to the end of the rainbow.
[35:26] Thank you for the mathematical
[35:27] interlude, Alex. That's why we love you.
[35:29] All right, let's jump into our next
[35:31] story here. Sam Wman made global news
[35:33] not once but twice. Uh the first item is
[35:36] an oped he published in the Financial
[35:38] Times regarding AI governance. Uh this
[35:41] was a result of him meeting with G7
[35:43] leaders in France last week. Sam
[35:46] basically said that in 2 years we should
[35:49] all expect AI systems with astonishing
[35:51] power that will reshape the material
[35:54] conditions of human life on a scale
[35:56] never before seen, at least not since
[35:58] electricity. That everyone on the planet
[36:00] deserves access to these technologies
[36:02] and the right to determine for
[36:03] themselves how to best use them.
[36:06] Incredibly, Sam went on to insist that
[36:09] democratic institutions must lead and
[36:12] not defer responsibilities to the San
[36:15] Francisco AI labs. He said basically,
[36:18] quote, "Safety standards must be
[36:20] established before there is broad
[36:22] distribution that governance uh requires
[36:25] democratic process, not decision-making
[36:27] by a small number of San Francisco based
[36:29] companies. uh Sam proposed a framework
[36:32] of a USled international forum that
[36:36] would establish standards, provide
[36:38] expertise and partial analysis of
[36:39] capabilities and risks that this forum
[36:42] would make the most advanced
[36:43] technologies available to nations and
[36:45] companies that participate and follow
[36:47] the rules. He concluded that the forum
[36:50] would serve as a governance mechanism
[36:52] for all AI labs and guard against the
[36:55] commercial pressures that we've seen
[36:57] with unsafe racing. Okay. So, like, wow.
[37:02] Um, uh, you know, he's taking a first
[37:05] mover here. Uh, I really wonder what
[37:08] Daario and Demis and Elon and Zuck think
[37:10] about the op-ed. Uh, it is worth noting
[37:14] that Daario and Demis were both on stage
[37:18] uh, at Davos proposing a somewhat
[37:21] similar governance. It always seems like
[37:23] Demis and Daario are are teaming up uh
[37:26] on one side of the equation and Sam is
[37:29] on the other. Let's take a listen to
[37:30] Deis and Daario talking about
[37:32] regulations and their proposal for like
[37:34] CERN or an atomic energy commission.
[37:38] We probably need new institutions to be
[37:40] built to to help govern some of this.
[37:43] You know, I talked about a CERN. And I
[37:44] think we need a a kind of equivalent of
[37:46] an IAEA atomic agency to monitor uh uh
[37:50] sensible projects and and and those that
[37:52] are are more more risktaking. Um I think
[37:55] you know we need to think about that the
[37:56] the society needs to think about what
[37:58] kind of governing bodies are needed. Um
[38:00] ideally it would be something like the
[38:01] UN but given the geopolitical
[38:04] complexities that doesn't seem very
[38:05] possible. Um and I also agree with Demis
[38:08] that this this idea of you know
[38:11] governance structures outside ourselves.
[38:14] Um I think these kinds of decisions are
[38:16] too big for any one person. We're still
[38:19] struggling with this. You know as as you
[38:20] alluded to not everyone in the world has
[38:23] has the same uh has the same perspective
[38:26] and so you know some some countries in a
[38:28] way are adversarial on this technology.
[38:30] But even within that all those
[38:31] constraints, I think we somehow have to
[38:33] find a way to build a more robust
[38:36] governance structure that doesn't
[38:37] doesn't put this in the hands of of just
[38:39] So I think these guys are under a lot of
[38:41] pressure uh a huge amount of pressure uh
[38:45] being viewed as potentially uh saviors
[38:48] or the destroyers of worlds and they
[38:52] need government oversight to help
[38:54] relieve that so they can sleep at night.
[38:57] Uh it's it's interesting. It's it's a
[38:59] lot of pressure putting the heads of two
[39:01] frontier labs on one love seat at Davos.
[39:04] [laughter]
[39:04] Well, they get they you know there's a
[39:06] there is a love affair between uh
[39:08] between Demis and Dario and between
[39:10] Google and Anthropic. Just don't put Sam
[39:13] on that same couch. [laughter]
[39:15] Look, there's a there's an elephant in
[39:16] the room here, which is that you're
[39:18] you're we've got the industrial era
[39:20] nation state
[39:22] and you're asking it to govern
[39:23] postindustrial cog cognition,
[39:26] right?
[39:26] It just can't be done. And this this
[39:28] breaks the nation state model so
[39:30] fundamentally all of this right just
[39:33] look at the ruling that only US
[39:34] nationals can can look at the models I
[39:37] mean it's just absurd at so many levels
[39:39] not that they have a better mechanism
[39:42] but that just doesn't apply [snorts] now
[39:44] when the people that are kind of racing
[39:46] the hardest are asking for governance it
[39:48] tells you that's not really performance
[39:50] anymore right this is a huge thing the
[39:52] problem is governance needs to become
[39:54] exponential means it has to be real time
[39:56] it has to be adaptive it must be data
[39:58] driven and and we just can't do it in
[40:01] this way. So I think this is going to at
[40:03] some level break the governance model in
[40:05] some very fundamental ways or we're
[40:06] going to politic system.
[40:08] I worry about regulatory capture. So
[40:11] much of this again slightly cynical take
[40:14] might be smells like regulatory capture
[40:17] smells like a little bit of pandering to
[40:19] G7 or Davos. Is it really the case that
[40:23] uh an IEA type mechanism is needed or
[40:27] These aren't mutually
[40:28] United Nations.
[40:29] Yeah. [laughter] Or a and and or is it
[40:32] possible that you have heads of [snorts]
[40:34] frontier models uh frontier labs who are
[40:36] facing an onslaught of Chinese
[40:38] openweight models who want maybe a
[40:41] slightly on margin more protectionist
[40:43] regime to keep the Chinese openweight
[40:45] models out of a defined intelligence or
[40:48] super intelligence block because they
[40:50] they maybe fear a bit of competition,
[40:52] want to capture the regulatory state. We
[40:54] and we'll get to that conversation to
[40:55] you a little bit later. You know, the
[40:57] the interesting thing is that the
[40:58] companies have failed to do this for
[41:00] themselves, right? They failed to come
[41:02] together. If you remember back to the
[41:04] Syllamar conferences in the 80s, I was
[41:06] in the biotech industry there at MIT at
[41:09] the White Institute and all of the
[41:11] scientists got together. We had just
[41:13] discovered the restriction enzymes that
[41:15] allowed you to properly edit genes. Uh
[41:18] and the front cover of like Time
[41:19] magazine with like Hitler babies. It was
[41:21] like you know a lot of fear about
[41:23] genetic engineering and the industry got
[41:26] together and set up their own regulatory
[41:28] structure which has held extremely well
[41:31] for for decades but it's tricky Peter I
[41:34] mean I maybe a question for you Peter on
[41:36] this I I think it's really tricky for
[41:38] the industry to sell not that it's like
[41:40] organizationally tricky you could put
[41:42] the four frontier uhish labs on a love
[41:45] seat and say you all work it out but the
[41:48] the problem is how do you avoid that
[41:50] giving the appearance of collusion
[41:51] and creating a cartel and competition
[41:53] like how do you do that in a way that
[41:54] isn't blatantly anti-competitive?
[41:56] Yeah. Uh I don't know. The difference of
[41:59] course is that in the early days the
[42:01] biotech industry we weren't talking
[42:03] about trillion dollar companies back
[42:04] then. Uh the revenue engines were no
[42:06] longer you know the the the AI race that
[42:09] Sam spoke about which is very real right
[42:11] now. I mean people releasing models uh
[42:14] pulling their punches and just trying to
[42:16] outdo each other week on week on week.
[42:18] uh that was not the case in the biotech
[42:20] industry at least not back then but uh I
[42:24] think we're hearing a consensus view
[42:27] from these three individuals which is
[42:28] going to lead to some structure of
[42:31] government regulation I guarantee you
[42:33] with with with these three CEOs saying
[42:36] we need regulation the regulators will
[42:38] come in and say great let's give you
[42:40] regulation now this yeah go on
[42:43] a a prediction China is missing from
[42:46] this discussion China is if if there was
[42:49] an elephant in the room, China is the
[42:50] second elephant in this particular room.
[42:52] And what for this to for this to come to
[42:55] to fruition, China is going to need to
[42:58] play ball and restrict the proliferation
[43:00] of Chinese models. And you can already
[43:02] see hints coming out of the CCP that
[43:05] China may, contrary to their historic
[43:07] position of blanketing the world, maybe
[43:10] even intelligence dumping onto the world
[43:12] all these openweight models. If the CCP
[43:14] starts to take a hardline position that
[43:16] no, China is going to restrict the
[43:18] export of Chinese openweight models
[43:21] going forward, then I think a regime
[43:22] like this is possible and the world
[43:24] splits into two super intelligence
[43:25] blocks.
[43:26] Yeah, I think that unfortunately is
[43:28] inevitable. I I wish it were not. Um but
[43:31] I can't see it going any other way right
[43:33] now.
[43:35] I'm going to say it again. You can't
[43:37] regulate this in any way, shape, or
[43:38] form.
[43:40] Oh, you you don't think you can regulate
[43:42] intelligence?
[43:43] You have to. Why not?
[43:45] You can't. You'd have to regulate every
[43:47] line of code written. Uh people can
[43:49] download take models offline, merge
[43:52] models, do a lot of stuff offline that
[43:54] they that doesn't then use the existing
[43:57] online models. I don't I don't see how
[43:59] you can police this.
[44:00] Oh, there's totally I mean just just a
[44:02] minute on this. So, Verer Vinci wrote
[44:05] extensively about this. We have a uh
[44:08] sort of a cognitive surplus of
[44:09] transistors. In my mind, there are so
[44:12] many different social engineering
[44:13] techniques that humans have discovered
[44:15] over the the centuries for policing it.
[44:17] Like we could have models policing each
[44:19] other. We could have at the transistor
[44:21] level, we could be using the surplus of
[44:23] transistors to do KYC all the way down
[44:26] to the circuit level if we have to. I
[44:27] think we have so many different
[44:29] Let me rephrase. the current regulatory
[44:32] structures cannot in any way, shape or
[44:34] form regulate what's coming. You need
[44:37] what you're talking about an AI based
[44:40] almost down to the hardware level based
[44:41] but that would cut across everything. It
[44:44] can't operate in the geopolitical
[44:46] environment that we have today.
[44:48] Well, look, I think uh it's it's really
[44:50] clear that the prompts are all going to
[44:52] get inspected and also the internal JS
[44:54] spaces now will be inspected.
[44:56] That the labs will do the inspection on
[44:58] behalf of the US government and that as
[45:01] Alex said, high probability China will
[45:03] stop exporting open source sometime in
[45:06] the next year or two.
[45:07] Yeah.
[45:08] For the same exact reasons.
[45:10] And then you'll have Yeah. like a you
[45:12] know a long-term arms race between the
[45:14] east and west versions of AI super
[45:16] intelligence.
[45:17] So Sam said specifically the framework
[45:19] is for a USled international forum which
[45:23] of course is devoid of the word China in
[45:27] there. Uh I am curious you know what
[45:30] scenarios do we have? I was speaking to
[45:32] uh Alvin Grlin who's a a friend of of
[45:36] ours about you know USChina
[45:38] relationships and the question is is
[45:41] there a structure in which you know we
[45:45] can see a US China alignment on AI um
[45:49] anybody
[45:50] what you'd be looking for if that were
[45:52] to happen you'd be looking for cross
[45:54] inspections of the prompts like are we
[45:57] allowing each other and the problem that
[45:59] the US will have with that is China at
[46:01] stealing intellectual property.
[46:04] Uh so I think it's unlikely but it is
[46:06] possible. That's how you would know that
[46:08] there are no bad actors is just looking
[46:09] at each other's underlying prompts and
[46:11] weights and and J spaces.
[46:13] Ultimately, you know, we use China as a
[46:15] stocking horse to accelerate investments
[46:17] and accelerate, you know, reduce
[46:19] regulations and such. But I think for
[46:22] the safety of the planet, not having a
[46:24] AI arms race between the two nations uh
[46:27] is an outcome I'd love to see happen. I
[46:29] also don't think the IAEA style
[46:31] mechanism necessarily works for AI just
[46:34] at the technical level. Forget about the
[46:35] political or or geodnamic level. just at
[46:39] the technical level that the the notion
[46:40] of say different blocks inspecting each
[46:44] other's fision uh inputs if if you will
[46:48] that that's that's conceivable if to to
[46:52] the extent uh open PN question mark
[46:54] question mark question mark close PN
[46:56] that just like looking at uranium uh or
[46:59] say shipments is is a productive uh or a
[47:03] wholesome way of tracking different
[47:06] nations uh nuclear weapons capabilities.
[47:09] I'm not sure that generalizes to
[47:11] intelligence. There are simply to Sem's
[47:13] earlier point, there are so many
[47:14] different ways to hide or to mask super
[47:18] intelligence and underlying
[47:19] capabilities. So many different forms it
[47:21] could take. Greg Bear has written uh a
[47:24] fair amount over the years about sort of
[47:27] uh prohibition era style uh bathtub
[47:30] super intelligences. If if we had to if
[47:32] if uh Russia or China entered into some
[47:36] sort of internationalist regime where
[47:39] the US were inspecting all of their
[47:41] supercomputers and all of their prompts
[47:42] and all of their algorithms, there are
[47:44] simply too many places that one can hide
[47:47] super intelligence that I I'm not sure
[47:49] that an IEA's type mechanism with such a
[47:52] a simple-minded oh let let's look at
[47:54] their uranium equivalent shipments or
[47:56] let's look for their centrifuges would
[47:59] actually be wholesome enough to to cover
[48:01] all of the world.
[48:02] This episode is brought to you by
[48:04] Blitzy, autonomous software development
[48:06] with infinite code context. [music]
[48:09] Blitzy uses thousands of specialized AI
[48:12] agents that think for hours to
[48:14] understand enterprise scale code bases
[48:17] with millions of lines of code.
[48:19] Engineers start every development sprint
[48:21] [music] with the Blitzy platform,
[48:23] bringing in their development
[48:24] requirements. The Blitzy platform
[48:27] provides a plan, then generates and
[48:29] pre-ompiles code for each task. Blitzy
[48:32] delivers 80% or more of the development
[48:34] work autonomously while providing a
[48:37] guide for the final 20% of human
[48:39] development work required to complete
[48:41] the sprint. Enterprises are achieving a
[48:44] 5x engineering velocity increase when
[48:47] incorporating Blitzy as their preIDE
[48:49] development tool, pairing it with their
[48:51] coding co-pilot of choice to bring an AI
[48:54] native SDLC into their org. Ready to 5x
[48:58] your engineering velocity? Visit
[48:59] blitzy.com to schedule a demo and start
[49:02] building with Blitzy today. [music]
[49:07] All right, let's [snorts] move on to our
[49:08] second Sam Alman story. Um I think of it
[49:11] as the beginning of a the starting gun
[49:14] for the grand equity negotiations taking
[49:17] place for universal basic equity. So
[49:19] again in the financial times it was
[49:21] reported this week that Sam has been
[49:23] talking to Trump Lutnik Bessant and
[49:26] Bernie Sanders about a 5% equity stake
[49:28] in OpenAI. Uh OpenAI's last reported
[49:32] valuation 852 billion back in March.
[49:35] That 5% stake would be worth about 42.6
[49:38] billion. uh given 315 million American
[49:42] citizens. That's only 135 bucks per
[49:44] person. Not very much. They talk about a
[49:46] proposed Alaska permanent fund. Uh that
[49:50] permanent fund is 91 billion and it
[49:52] dividends about a,000 to $3,000 per
[49:55] citizen of Alaska per year. Alman's
[49:59] broader idea and again this is him out
[50:02] there speaking on his own uh and putting
[50:04] him putting forward a plan for the
[50:06] entire AI industry. He says he'd like to
[50:08] see anthropic Google Meta also
[50:10] contribute equity to a public fund. Um,
[50:13] we should remember, we've talked about
[50:15] this before, uh, the the US government
[50:19] already owns 10% of Intel. And so when
[50:22] Sam talks about a 5% uh, uh, donation,
[50:26] if you would, to the government, I think
[50:28] uh, Trump is an amazing negotiator. I'm
[50:30] going to guess we're going to end up at
[50:31] 10%. You know, I did a poll uh to on on
[50:35] on X asking how much and uh the majority
[50:40] of the people were either at 20% or
[50:42] zero. Interesting.
[50:45] It's so irrelevant.
[50:47] My my read on this is that you know a
[50:49] year ago Sam was um called the most
[50:51] powerful man on earth in multiple
[50:53] interviews.
[50:54] Yeah. Uh now you've got Daario and
[50:56] Dennis clearly working on the future
[50:58] governance of the entire world and
[51:00] Daario has a big deal with Elon you know
[51:02] license renting all of the chips. So now
[51:04] all the guys are talking to each other
[51:06] and they're not including Sam.
[51:08] So Sam's now writing an op-ed which is
[51:10] like you know trivially short oped by
[51:13] the way.
[51:14] Uh and he's proactively offering 5% of
[51:17] his company but he's just trying to get
[51:18] back in the hunt of relevance in the
[51:20] eyes of the White House. Um, I did hear
[51:23] that Daario got kicked out of the White
[51:24] House for being too weird. Did you hear
[51:26] that story, too?
[51:27] Yeah, that that was published. Yeah,
[51:29] that that was all all over the news
[51:30] recently.
[51:32] Yeah. Yeah, it was like what, two weeks
[51:33] ago, a week ago.
[51:34] The the story was that Anthropic sent in
[51:36] Daario initially to negotiate and that
[51:39] didn't quite work out. So, they sent in
[51:40] Tom Brown instead.
[51:41] Ah, the co-ounder for Fable 5. Yeah.
[51:43] Yeah. Yeah. That's so easy to visualize,
[51:45] isn't it? Like [laughter]
[51:47] Trump is like, "Dude, you're weird, man.
[51:48] I don't even know what you're talking
[51:49] about. What's this JSpace crap? get
[51:51] [laughter] out of the White House. So,
[51:54] but anyway, where do you where do you
[51:56] figure this goes? Where do you figure
[51:58] the idea of of contribution to the
[52:00] government from the AI labs?
[52:02] It's so irrelevant. The government can
[52:03] take any chunk they want any time they
[52:05] want. They already take it in income
[52:06] tax. Anyway, this is so irrelevant.
[52:09] I I'll take a different position. I
[52:11] think this is super relevant. I think
[52:12] that this one can see the outlines of a
[52:15] baby universal basic equity uh grand
[52:18] bargain if you will. it. The economics
[52:20] don't work for supporting universal
[52:22] basic equity right now off of say a 5%
[52:25] chunk. But if Open AI and Anthropic and
[52:28] SpaceX AI all do this and they grow Elon
[52:32] style a couple orders of magnitude in
[52:34] terms of size and grow the economy,
[52:36] that's your UB for you. So I I coined a
[52:38] term for this a few days ago. I call it
[52:39] a hyper tithe which I define as a a
[52:43] fixed equity contribution paid by
[52:45] companies building the singularity stack
[52:47] paid into a sovereign wealth fund or
[52:49] similar public vehicle turns private
[52:51] exponential sem upside into universal
[52:54] basic equity broader national ownership
[52:56] and a more relaxed regulatory bargain.
[52:58] I can tell you exactly why that makes no
[53:00] sense whatsoever. Back in the New Deal
[53:02] era, the government decided, you know
[53:03] what, we're going to take a huge chunk
[53:05] of everybody's paycheck and we're going
[53:06] to call it social security and then
[53:08] we're going to invest it on your behalf
[53:10] for your entire life and then when
[53:11] you're old, we're going to give you a
[53:12] lot more money back. They decided very
[53:14] quickly that they had no idea how to
[53:16] invest your money. And so they said,
[53:18] "Screw that. We're not going to do that.
[53:19] We're just going to take the money and
[53:21] spend it instead because because we
[53:23] don't have any idea how to invest your
[53:25] money on your behalf and go ahead." And
[53:27] so that all collapsed and and moved over
[53:28] to 401k plans where Fidelity or UBS
[53:31] investor money because they know how to
[53:32] do it. So the idea that the government
[53:34] is going to set up some intelligent
[53:35] sovereign wealth equity thing is
[53:38] absolutely insane. The next president
[53:39] will immediately sell it all, turn it
[53:41] into cash, and then use it to buy votes
[53:44] in the next election.
[53:45] This is so interesting, Dave. Yeah. Want
[53:47] to discuss this
[53:49] if you want. So, let me let me just
[53:51] throw up my view, but I do want to get
[53:52] back, Alex, what you think cuz I'm I'm
[53:54] like I'd love to hear the discussion
[53:56] back and forth. I I go full cynic on
[53:58] this. This is purely Sam a trying to get
[54:00] in the game and b uh trying to protect
[54:04] because the minute you have the
[54:05] government has 5% you you're too big to
[54:07] fail in a sense and they they protect
[54:09] himself by doing that. So, that's my
[54:11] full cynic view on this.
[54:13] Can I just inject one things and then
[54:15] hand to you Alex which is I I think one
[54:17] of the important elements here that
[54:19] people are not realizing is AI as we
[54:22] value AI today in terms of sales of
[54:24] tokens is a minuscule amount of the
[54:29] future value of these labs. I think that
[54:32] you know as they start discovering
[54:34] fundamental breakthroughs in biology and
[54:36] physics and chemistry those are trillion
[54:39] dollar pops. And I think the idea of if
[54:42] there was a structure where the US
[54:45] populace, the US citizenry had ownership
[54:48] in these companies that it could drive
[54:51] uh an economic engine for you know UB,
[54:55] UBS, UB uh UB whatever. Uh but again my
[55:00] mission here is how do you reduce the
[55:02] fear that people are having because you
[55:06] know the numbers are staggering. Only
[55:09] 10% of Americans think that AI is going
[55:12] to deliver positive benefits to
[55:14] humanity. You know, 30 to 35% feel
[55:18] relatively good about it, but only 10%
[55:20] have this view that it's going to make
[55:22] the world a better place.
[55:23] What it means is we're not doing our
[55:25] jobs blasting out the optimism. We need
[55:28] to get better at this.
[55:29] All right, Alex, please take us home
[55:31] here. The the distinction uh to to res
[55:34] respond to Dave's point about social
[55:36] security. Social Security in the US was
[55:39] created in a time and a place when index
[55:41] funds didn't exist. It was created in
[55:43] the wake of the Great Depression. There
[55:45] was a general distrust of the stock
[55:48] market in general. There have been
[55:50] multiple attempts over the years to
[55:51] quote unquote privatize social security
[55:54] which would take the form of converting
[55:56] sort of a cashbased pyramid scheme into
[55:59] something more equity oriented. That's
[56:01] failed for a variety of political and
[56:04] social reasons. But I I do think this
[56:06] time is different. If if if social
[56:08] security were created today and not, you
[56:12] know, almost a century ago, I think it
[56:15] probably would be based on some sort of
[56:17] sovereign wealth fund that holds
[56:20] hopefully like a broad market index fund
[56:23] that's low cost and not just be based on
[56:26] a pyramid style cash in cash out bond uh
[56:31] or interestbearing security type scheme.
[56:35] And that's where where I think a hyper
[56:37] tithe has the potential to become a baby
[56:40] and hopefully aspirationally a grown-up
[56:43] UBE. If if these frontier labs if if if
[56:45] there were a hyper tithe from all of the
[56:47] Magna Mopa companies to to blend Peter
[56:50] neologisms uh and these were all paid
[56:53] hypothetically into a sovereign wealth
[56:55] fund and the Magna Mopsta companies just
[56:58] ultimately over the next 5 to 10 years
[57:00] grow so much and grow the economy so
[57:02] much. I do think that could in principle
[57:04] support a universal basic equity type
[57:06] system.
[57:07] I I agree with you, Alex. Uh and you
[57:10] know, there's a lot of conversation
[57:11] right now about the Trump accounts and
[57:13] Trump accounts for adults as well. Uh
[57:16] and I'm I you know,
[57:20] that's his nature. His nature is to
[57:22] negotiate and take pieces of things. And
[57:25] I think he wants to populate the Trump
[57:28] accounts for adults with 10% of all of
[57:32] the hyperscalers and AI labs. That's my
[57:34] guess. Now whether he can pull that off
[57:38] and then put the protections in place,
[57:39] Dave, so they can't be sold so that it's
[57:42] dividends from those, you know, so the
[57:44] $91 billion of
[57:45] these are dividend companies though.
[57:47] There are no dividends. Like okay, so
[57:48] everybody in America, you get a Trump
[57:50] account, we put the Magnum Bobsa stocks
[57:52] in it. Here you go. But you're not
[57:54] allowed to sell it or you are allowed to
[57:56] sell it or these are not there's no
[57:57] income from it like
[57:59] are we gonna call them Trump accounts 50
[58:01] years from now realistically
[58:03] 5 530A accounts if if you like but I if
[58:07] I were head of commerce or head of
[58:08] treasury the the sort of scheme
[58:11] policy-wise that I might be
[58:12] contemplating is okay you you start with
[58:14] a sovereign wealth fund or whatever
[58:17] could could be individual 530A accounts
[58:19] it's populated with the Magna MOPSA
[58:21] stocks or or some subset thereof of you
[58:24] wait a couple of years and then the
[58:26] market is sufficiently liquid that you
[58:28] could liquidate them in favor of since
[58:31] you're the government you you don't have
[58:32] to tax yourself. So you could do a
[58:34] taxless exchange for a broad index fund
[58:37] even though it's populated initially
[58:39] with magnumopsta contributions via this
[58:41] hyper tithe grant to the government. You
[58:44] exchange them for a broad market index
[58:45] fund. That's the solution.
[58:47] Well, I don't I don't think it's a bad
[58:49] idea. I just think it's irrelevant.
[58:50] The government has the power of
[58:52] taxation. They can extract from the
[58:54] income anytime they want.
[58:56] We're going to find out
[58:57] quick quickly on this one. The the
[58:58] corporate income tax is cashbased. And
[59:00] the problem in in a hyperscaling
[59:03] singularityoriented economy is cash may
[59:05] not be the best basis for taxing the
[59:08] economy. But equity does scale
[59:11] if you sell it.
[59:14] Well, if you can tax equity. Right now,
[59:15] we don't have basically an equity wealth
[59:17] tax. This is a de facto shadow equity
[59:20] wealth tax with companies perhaps
[59:22] feeling a bit of regulatory pressure to
[59:24] give up equity in themselves. It's it is
[59:27] definitely a tax but it's a slightly
[59:28] different type of tax.
[59:30] All right.
[59:30] I'm all in favor of the UB. I just don't
[59:32] see the mechanisms for it. But I do
[59:34] agree with the principle.
[59:36] All right. Well, let's jump into our
[59:38] next subject. You know, one of the
[59:40] reasons we're always concerned about
[59:41] UBI, UB, all of that is the concerns
[59:45] around job loss. So our next story is
[59:47] about jobs and the continuing debate
[59:49] about whether AI is going to be creating
[59:52] or destroying jobs now and in the near
[59:55] future. So we've covered both sides of
[59:57] the story. It's been murky. We've given
[59:59] evidence for both sides. A new paper
[01:00:01] released this past week by RAMP and
[01:00:03] Ravilio Labs uh gave some pretty
[01:00:06] definitive data here. They looked at
[01:00:08] 21,559
[01:00:10] US companies over the past 5 years
[01:00:12] between January 2021 and February 2026,
[01:00:16] matching the actual AI spend of those
[01:00:19] companies and their workforce records,
[01:00:22] meaning hires and fires. So, here's the
[01:00:24] headlines. Companies that spent heavily
[01:00:26] on AI did not shrink. In fact, they
[01:00:29] grew. So the highintensity AI adopters
[01:00:32] that they studied were spending $33 per
[01:00:36] employee per month on AI. Uh they grew
[01:00:39] $10.2% in a white collar and 12% at
[01:00:43] entrylevel growth. Uh in contrast the
[01:00:47] low inensity adopters spent $3 per
[01:00:50] employee per month, basically a tenth
[01:00:52] and showed no significant employment
[01:00:54] change. The author has warned this is
[01:00:56] correlation not causation. But it puts
[01:00:59] forward a very different theory. Rather
[01:01:01] than the AI is going to replace workers,
[01:01:03] it suggests that AI may expand ambition
[01:01:06] first. Companies that actually integrate
[01:01:08] AI deeply may take on more projects,
[01:01:11] serve more customers, build faster, hire
[01:01:14] more humans, especially entry level to
[01:01:16] capture the upside. So I love this
[01:01:18] story. I mean for me this is a abundance
[01:01:22] optimism story for people because
[01:01:23] there's a lot of fear out there. My
[01:01:25] concern about this story is that
[01:01:28] regardless of what the data says, the
[01:01:31] news media is out there and the
[01:01:33] underlying
[01:01:35] belief is that AI is going to destroy
[01:01:37] our jobs and it will displace a number
[01:01:40] of things, right, with you know robo
[01:01:43] taxis and uh and AI call center workers
[01:01:47] and so forth. But
[01:01:50] the evidence looks like, and I don't
[01:01:52] know about you guys, but I'm hiring more
[01:01:53] people in my companies than ever before.
[01:01:56] I don't know if that's true for you,
[01:01:57] Dave, and and Alex.
[01:01:58] Well, God, if anyone's AI native, their
[01:02:01] demand demand for that person is through
[01:02:03] the roof.
[01:02:04] Yeah.
[01:02:05] Uh so yeah, it's it's rampant. Um, and
[01:02:08] and I'm starting to feel like this is a
[01:02:10] permanent thing, not a transitional
[01:02:11] thing. Because, you know, the one of the
[01:02:13] things to worry about is look,
[01:02:14] implementing AI is such a payback that
[01:02:17] there's this land grab of talent there.
[01:02:19] Anyone who can implement it, any bank,
[01:02:20] any insurance company, any operating
[01:02:22] company, anyone who can get AI to work
[01:02:23] in this shop, we hire them for whatever
[01:02:26] they cost. Um, is that transitional
[01:02:28] because once they've implemented the AI,
[01:02:30] they've coded themselves out or is it
[01:02:31] permanent? I feel more and more like
[01:02:33] it's permanent. like as the AI improves
[01:02:35] the things you can do also grow and that
[01:02:38] person's value goes up over time and so
[01:02:41] the the data I think is very early
[01:02:44] inklings of what's inevitable where AI
[01:02:47] native organizations are going to just
[01:02:48] grow like wild and they're going to add
[01:02:50] headcount as they do it and anyone who's
[01:02:53] sitting still hasn't fired everybody yet
[01:02:56] but eventually they're going to be wiped
[01:02:57] off the face of the earth and so what
[01:02:59] you see right now is net growth
[01:03:01] yeah this is what we call the
[01:03:02] organizational singularity Right? If
[01:03:04] you're an AI native, AIcentric
[01:03:06] organization, if you're doing deep
[01:03:08] resign of your workflows to be AI
[01:03:11] native, then you have an explosive
[01:03:14] opportunity in front of you. Shallow
[01:03:16] adoption fails because this is not
[01:03:19] automation versus jobs. It's shallow
[01:03:21] adoption versus deep redesign. So, we've
[01:03:23] started our pilot, by the way, of
[01:03:25] working with companies. So, I'll report
[01:03:26] back as to how things are going, but
[01:03:28] we're unbelievably excited at look the
[01:03:30] opportunities. We're we're like we can't
[01:03:32] even count the number of workflows that
[01:03:34] we could help automate with these
[01:03:36] companies. So for each company, we're
[01:03:37] picking one workflow that might
[01:03:39] radically increase revenue and one
[01:03:40] workflow that might radically shrink
[01:03:42] cost.
[01:03:43] Right.
[01:03:43] Totally
[01:03:45] for both sides. It's like crazy.
[01:03:47] That's literally why you're in every
[01:03:48] city in the world every time we do a
[01:03:50] podcast because I mean the demand for
[01:03:52] what you're what you're teaching is so
[01:03:56] step function through the roof
[01:03:57] instantaneous. biggest [laughter] shift
[01:03:59] in organizations in a hundred years
[01:04:03] probably in human history I'll bet and
[01:04:05] of all time
[01:04:06] you know
[01:04:07] not just to companies but it applies to
[01:04:08] nonprofits and impact projects and
[01:04:10] government department everything
[01:04:12] so it's going to be huge
[01:04:13] just I love using token spend as a proxy
[01:04:16] for adoption even though it's not
[01:04:17] perfect it's it's it's reasonably good
[01:04:19] so this study actually focused on token
[01:04:21] spend reasonably good way to say
[01:04:23] are you doing it for real or not
[01:04:24] and just a quick plug we have released
[01:04:26] the book as an AI I it's available for
[01:04:29] free. You can download a cloud skill and
[01:04:31] run your business in this new model.
[01:04:33] It'll tell you what to do. It's free. Go
[01:04:36] register at open exo and download it.
[01:04:38] The report back are crazy.
[01:04:40] To remind folks, uh See is going to be
[01:04:42] doing a session at the moonshot
[01:04:44] gathering on September on September 24th
[01:04:47] on the organizational singularity and
[01:04:50] AWG is going to be there doing an
[01:04:52] extended AMA on solve everything. bring
[01:04:55] your your most difficult challenging
[01:04:57] questions to Alex. Dave will be there.
[01:05:00] Stomp the Trump. Yes, Dave will be there
[01:05:02] talking about uh AI investing. We'll
[01:05:04] have Palmer Lucky. We have Rod
[01:05:06] Rodenberry. We have Ben Lamb, Kathy
[01:05:09] Wood. Uh it's going to be an amazing So
[01:05:12] go to moonshots.com
[01:05:15] for the Moonshot Gathering September
[01:05:17] 25th. Top creators and builders there.
[01:05:21] you know, um, interestingly enough,
[01:05:23] we're still seeing a number of companies
[01:05:25] out there. You know, Oracle blamed
[01:05:28] 21,000 layoffs on AI. Meta blamed 8,000
[01:05:31] layoffs, Block 4,000, Cisco 4,000,
[01:05:34] Atlassian, 1600. And so the question is,
[01:05:37] are these CEOs just using AI as an
[01:05:41] excuse um for, you know, you know,
[01:05:44] reorganization, or is it true?
[01:05:47] There's two things going on. One is uh
[01:05:49] like for example it's well known that
[01:05:50] block overhired radically and needed to
[01:05:52] shrink. So that's an easy hobby horse
[01:05:55] for shrinkage. The other is the note
[01:05:58] that the company's laying off for all
[01:05:59] SAS companies and the SAS business model
[01:06:01] is fundamentally broken in an age of AI.
[01:06:05] So both of those are happening at the
[01:06:07] same time.
[01:06:08] Yeah, I think some some of it is real,
[01:06:10] some of it is AI washing. the the real
[01:06:12] component in many cases as with Oracle
[01:06:15] for example is it's the capital the cap
[01:06:18] expenditures that are crowding out the
[01:06:20] opex of human labor. It's quite
[01:06:22] literally all the isms from the first
[01:06:24] part of the 20th century worrying about
[01:06:26] capital v lab labor we're seeing play
[01:06:28] out internally in hyperscalers like
[01:06:30] Microsoft or Oracle that are having to
[01:06:33] direct free cash flows to internal capex
[01:06:36] to building out their hypers scale AI
[01:06:39] cloud infra capabilities at the cost of
[01:06:42] you American usually uh Ireland in some
[01:06:45] cases based developers that can now be
[01:06:49] automated with software that sits on top
[01:06:51] of the AI infra.
[01:06:52] Well, you know, Peter, remember when we
[01:06:53] were at Facebook uh before it became
[01:06:55] Meta around the time of the Oculus and
[01:06:58] we we were having that tour and you you
[01:07:01] look at Facebook online, you look at
[01:07:03] Instagram online, and then you look at,
[01:07:04] you know, 10,000 employees. You're like,
[01:07:07] what the hell do you guys do? I mean, it
[01:07:08] hasn't changed like like what are you
[01:07:11] literally doing? So you walk around and
[01:07:12] talk to people and tons of like you know
[01:07:15] UX experimentation and remember in the
[01:07:17] bathrooms above the urinals there's the
[01:07:19] tip of the day you know the little
[01:07:21] coding you know and you're like oh okay
[01:07:23] that's what you guys are all doing
[01:07:24] you're like
[01:07:24] so that's like the easiest AI job in the
[01:07:27] world so I think that's very real like
[01:07:28] you just don't need those gooey
[01:07:31] low-level coding jobs anymore and a lot
[01:07:33] of it is server configuration you know
[01:07:35] propping up a new Instagram server for a
[01:07:38] new country that's so easy to do with AI
[01:07:40] now so I think I think That part's all
[01:07:41] real.
[01:07:42] Well, we're going to continue to follow
[01:07:43] this story on jobs. I think it's
[01:07:44] important, you know, if you're a if
[01:07:46] you're a student out there worrying
[01:07:49] about can you get a job um worrying
[01:07:52] about, you know, everything you're
[01:07:54] hearing out there, please dive into the
[01:07:56] world of uh of AI, uh of
[01:07:59] entrepreneurship. If you're a parent,
[01:08:01] you know, have this conversation with
[01:08:03] your with your kids. It's really
[01:08:05] important. I really my goal is to
[01:08:08] dismiss fear, right? It's there's real
[01:08:10] fear, but there's at least be fearful
[01:08:12] for the right reasons.
[01:08:14] Yeah.
[01:08:15] Just just to point out, David Saxs talks
[01:08:17] about all all the time on the All-In
[01:08:19] podcast that we're increasing jobs
[01:08:21] radically. Like, we're increasing
[01:08:23] hiring. All the data shows that. Follow
[01:08:25] the data. That's it. Just be evidentary.
[01:08:28] And I know we've we've talked about it
[01:08:30] in the past on this podcast being
[01:08:32] concerned about uh you know, a lack of
[01:08:36] new entry jobs. and there probably are
[01:08:38] in certain industries, but uh if you're
[01:08:41] AI native, as Dave said, I think uh
[01:08:44] you've got massive opportunities. All
[01:08:46] right, I'm going to move us forward
[01:08:47] here. Our next two stories are classic.
[01:08:49] Alex Karp, CEO of Palunteer. The first
[01:08:52] one is a product launch. The second one
[01:08:54] is a declaration of war. Uh in our first
[01:08:56] story, Palunteer and Nvidia have
[01:08:58] announced a sovereign AI architecture
[01:09:00] that puts Nvidia's Neotron open models
[01:09:03] inside of Palunteer's platform composed
[01:09:06] of their artificial intelligence
[01:09:07] platform, Ontology, Foundry, and Apollo
[01:09:10] stack designed for US government
[01:09:12] agencies and critical infrastructure
[01:09:14] operators. So, uh we've touched on
[01:09:16] Neotron a little bit in the past. It's
[01:09:18] Nvidia's open model. They've got three
[01:09:20] models, nano, super, and ultra. They
[01:09:23] range from 30 billion to about 550
[01:09:26] billion parameters. Uh, Neotron's edge
[01:09:28] is speed and cost. Uh, it can be roughly
[01:09:32] twice as fast and 60 times cheaper than
[01:09:34] GPT 5.5 or Alis 4.8, but it's not yet
[01:09:38] smarter than those two models. So, uh,
[01:09:41] I'd like to take a listen to Alex's
[01:09:44] video conversation or part of it on
[01:09:47] CNBC. Um, and we'll talk about it from
[01:09:50] there. Let's take a listen here.
[01:09:52] We're on we're we're sitting on critical
[01:09:54] infrastructure across America, Ukraine,
[01:09:56] Israel. Everyone who uses LLMs on the
[01:09:59] battlefield runs on top of our ontology.
[01:10:02] Clients are just to say they're unhappy.
[01:10:04] a level of discomfort and loss of trust
[01:10:07] on when you're using large language
[01:10:09] models. They are it's like a at this
[01:10:12] point everyone technical realizes
[01:10:14] they're like a critical resource to make
[01:10:16] them valuable in an enterprise like
[01:10:18] battlefield context or regulated context
[01:10:20] or manufacturing you have to have what's
[01:10:22] called an application layer but de facto
[01:10:24] it takes a large language model it makes
[01:10:26] it safe and useful and precise what what
[01:10:29] aligns me with Nvidia and I think is
[01:10:33] what the c technical customers want
[01:10:36] which is control over their compute
[01:10:39] their models, their data stack, and
[01:10:42] their alpha. They want to know they own
[01:10:45] the means of production. It's not being
[01:10:46] transferred to someone else. They're not
[01:10:48] interested in some fake deploy code that
[01:10:50] somehow is deploying tokens that
[01:10:52] transfers the alpha to a third party.
[01:10:54] And the jig is up. And so, we have to
[01:10:56] figure out a way be trust. And that
[01:10:58] trust is going to happen where everyone
[01:11:00] gets to ask ask and answer basic
[01:11:03] questions. Who owns the data? Where is
[01:11:05] it cached? Are the prompts secure? Is
[01:11:08] this being transferred to you? Are you
[01:11:10] being comp? Okay, if it was so valuable,
[01:11:12] let's say I can make you a billion
[01:11:14] dollars, right, tomorrow, wouldn't I
[01:11:16] say, I'll make you a billion dollars and
[01:11:17] I want 30%.
[01:11:19] Why are they charging for tokens if it's
[01:11:21] so valuable?
[01:11:23] I think you went off script in the end
[01:11:25] there. That that last point made no
[01:11:26] sense whatsoever. [laughter]
[01:11:27] Well, careful careful what you wish for
[01:11:30] because that that last bit is actually
[01:11:32] happening. [laughter]
[01:11:32] Yeah. Um, you know, Alex his his point
[01:11:38] here is and and he's got a second video
[01:11:41] actually. Let's go and play the second
[01:11:42] video and then we'll talk about it in
[01:11:44] general because I think this is the
[01:11:45] second part of the conversation here
[01:11:47] in this country at every single
[01:11:50] enterprise I deal with. They these
[01:11:52] people are livid. They're like I am
[01:11:54] paying for tokens that create no value.
[01:11:56] Let's say I can make you a billion
[01:11:57] dollars right tomorrow. Wouldn't I say
[01:12:00] I'll make you a billion dollars and I
[01:12:01] want 30%. Why are they charging for
[01:12:04] tokens if it's so valuable? These people
[01:12:06] are stealing the weights and alpha of my
[01:12:08] business and they're creating a wealth
[01:12:09] tax that does not help the poor. It just
[01:12:11] punishes starts with the billionaires.
[01:12:14] Every single person at this table is
[01:12:15] going to be paying a wealth tax only to
[01:12:17] punish us. And the reason for it is
[01:12:20] because these models have been
[01:12:21] completely over irresponsibly oversold.
[01:12:24] And the cell is it's dangerous for
[01:12:26] everyone which is why I can give it to
[01:12:28] all your adversaries but I can't give it
[01:12:29] to the department of war or I can't
[01:12:31] safely give it to an enterprise in this
[01:12:33] country without being certain that the
[01:12:36] alpha that business could transfer to
[01:12:37] this model tomorrow i.e. I have no
[01:12:39] business, no job is the voice of
[01:12:42] American business that is being
[01:12:44] channeled through me. And I'm telling
[01:12:45] you, it is it is absolutely a problem
[01:12:48] for this country because the clients
[01:12:50] have to be able to ask and answer very
[01:12:51] basic questions. Are you keeping the
[01:12:54] data? Are you going to enter our
[01:12:55] business? Do they get to control the
[01:12:57] weights to do it or do you get to
[01:12:58] control the weights? Are we really going
[01:13:00] to outsource the battlefield of this
[01:13:02] country to the consensus view in Silicon
[01:13:05] Valley? That is effing insane.
[01:13:08] Obviously, he went on a rant. Uh the key
[01:13:11] points he's making here is there's a
[01:13:14] great concern that when you're using
[01:13:16] anthropic or using open AI that you're
[01:13:20] effectively giving them your alpha,
[01:13:22] you're giving them access to all your
[01:13:23] data. And what's needed right now is uh
[01:13:27] a open models that you can build on your
[01:13:31] own hardware, onrem hardware. So you
[01:13:34] know uh openweight models onrem hardware
[01:13:37] and then customizing your own language
[01:13:40] your your own large language models and
[01:13:42] not giving your secure data your alpha
[01:13:45] as he calls it your means of production
[01:13:48] uh to these large AI frontier labs
[01:13:51] and your weights they're taking they're
[01:13:53] taking your weights [laughter]
[01:13:55] did you know you had any weights
[01:13:57] well okay if if you have any you're
[01:13:59] giving them to them like okay it's like
[01:14:01] actually it's everything that would make
[01:14:04] you hate Daario bundled together in one
[01:14:08] long glued together like and they have a
[01:14:10] wealth tax. Can you believe like okay
[01:14:13] let's put it all together to make every
[01:14:15] corporate CEO as scared and as angry as
[01:14:19] possible at Daario so that they buy the
[01:14:22] new open-source uh Palunteer Nvidia um
[01:14:26] you can run on prem model that keeps all
[01:14:28] your alpha and your weights safe from
[01:14:30] you know Dario because he's going to
[01:14:31] steal all your intellectual property
[01:14:33] very valid point actually the rant
[01:14:36] format is extra dramatic but but it's a
[01:14:39] very very valid point And it it's really
[01:14:41] interesting to think like, okay, he
[01:14:43] serves the defense department among
[01:14:45] others, but he's taking the open- source
[01:14:48] pathway to get in there, but you know,
[01:14:50] that's not going to last, right? You
[01:14:52] you're never going to have open source
[01:14:54] defense department weights. That's not
[01:14:56] going to
[01:14:56] Well, no. I mean, so he's building he's
[01:14:57] building an airgapped machine on top of
[01:15:01] Neotron,
[01:15:02] uh, which then the defense department
[01:15:05] owns that model, uh, and owns the
[01:15:08] equipment it's running on. I can imagine
[01:15:10] very much that works for them.
[01:15:12] For sure. And and also his other big
[01:15:14] customers are banks, mega banks,
[01:15:16] insurance companies. They'll also in his
[01:15:18] world have their own proprietary models.
[01:15:22] But you can't have every startup have
[01:15:24] its own proprietary model because then
[01:15:26] you'll have every terrorist have its own
[01:15:27] proprietary model.
[01:15:28] Why? Well, but why not? I mean, I'm
[01:15:29] running a couple of Mac Studios, you
[01:15:31] know, with Kimmy K2.5 on top of Opus 4.8
[01:15:36] um or below Opus 4.8. uh and an open
[01:15:39] claw there. I I haven't migrated yet,
[01:15:42] but why can't that be a standard future?
[01:15:46] Uh well, I think we'll look back and say
[01:15:48] this was a very cool, very fun, quaint
[01:15:51] kind of hobby era. But when it's super
[01:15:54] intelligent and capable of creating any
[01:15:56] virus, any chemical, any weapon, you
[01:15:59] can't have it available to each
[01:16:02] individual. Um right now, nobody can
[01:16:04] afford the compute to do those kinds of
[01:16:06] very evil things. So it's not a not a
[01:16:09] problem. But if we keep quantizing and
[01:16:11] compressing at our current rate, you
[01:16:12] know, like I I think this is about 100
[01:16:14] to a 10,000x performance increase year.
[01:16:17] If that happens again next year, then
[01:16:19] your Mac Mini size box is capable of
[01:16:23] viruses, nuclear weapons, anything. So
[01:16:27] we just can't have that outcome.
[01:16:29] It's not an if, it's a when. It's it's
[01:16:32] going to be a win. We got to Yeah. No, I
[01:16:34] I I I would distinguish between
[01:16:35] permissioned versus permissionless on
[01:16:37] one axis and locally hostable versus
[01:16:41] remote API only on the other. But may
[01:16:44] maybe just taking a step back, this was
[01:16:46] obviously the rant heard round the world
[01:16:48] and leave it to Alex Karp to articulate
[01:16:51] a bunch of different things that that
[01:16:52] that probably need to be unpacked. So
[01:16:56] maybe just to do a little bit of close
[01:16:57] reading of some of the things that he
[01:16:59] said and how I translate them. So,
[01:17:01] you'll note Palunteer back in the stone
[01:17:04] ages was a Claude rapper like the stone
[01:17:07] ages as of a few months ago um was was a
[01:17:10] key distribution channel for Claude into
[01:17:13] the Department of War into a variety of
[01:17:15] their customers. That's clearly over.
[01:17:18] That's point one. Point two, [laughter]
[01:17:21] I I think so. Uh point two, the deploy
[01:17:25] co reference. So when Alex, other Alex
[01:17:28] says drops an off-handed reference to
[01:17:30] deploy codes, I hear that as a frontal
[01:17:33] assault on OpenAI and Anthropic and
[01:17:37] other companies including Microsoft now
[01:17:40] launching forward deployed engineer
[01:17:42] organizations that represent a head-on
[01:17:44] assault on Palunteer. So he's definitely
[01:17:46] talking his own book. Palanteer
[01:17:49] basically defined the modern forward
[01:17:51] deployed engineering model and now all
[01:17:53] of the frontier AI labs are just
[01:17:55] launching direct competitors to
[01:17:56] Palunteer
[01:17:57] like let's go in there and store
[01:17:59] yeah so so why not counterattack via
[01:18:03] commoditizing one's complement with
[01:18:05] these openweight solutions from Nvidia
[01:18:07] second point other countries Palanteer
[01:18:10] sells quite a bit of its own stack not
[01:18:12] just into the US department of war not
[01:18:14] just into US final instit financial
[01:18:17] institutions but into other countries as
[01:18:19] well. And there was a dawning awareness
[01:18:20] by other countries, doubly so
[01:18:24] mythos fiasco that they're not going to
[01:18:26] get access to US capabilities from the
[01:18:29] frontier labs anymore. So they had
[01:18:31] better and I think they're now pretty
[01:18:33] well incentivized transition to locally
[01:18:36] hostable models that they can control
[01:18:38] that can't just be gatekept by US export
[01:18:40] controls on a moment's notice. So being
[01:18:43] good salesman, good businessman, Alex I
[01:18:46] think recognizes that all of his
[01:18:48] international customers need a
[01:18:50] localizable solution for inference time.
[01:18:53] The question that no one's asking,
[01:18:55] including Alex in his rant heard around
[01:18:57] the world, is what about sovereign
[01:18:59] training time? No one's asking that
[01:19:01] right now. Nvidia is training its own
[01:19:04] openweight models. It's not distributing
[01:19:07] those locally, but at some point I I
[01:19:09] suspect as this question, which to my
[01:19:12] ear rhymes with Microsoft in the late
[01:19:15] 90s when Microsoft was at the peak of
[01:19:17] its power and the open-source movement
[01:19:20] had to come from even though there was
[01:19:22] free software foundation, Richard
[01:19:23] Stallman, GNU, FSF, etc., etc. within
[01:19:26] the US really the nucleating event came
[01:19:29] from outside the US in the form of Linux
[01:19:31] and Linus Torva from Finland that then
[01:19:34] the whole canoe stack nucleated around
[01:19:37] similarly we're seeing the strongest
[01:19:39] openweight models come from China I
[01:19:41] think we're at a similar point now where
[01:19:43] you have a whole international community
[01:19:45] that's just realized thanks to fable and
[01:19:48] mythos that it can be cut off at a
[01:19:49] moment's notice and it needs an open
[01:19:51] weight stack and I I think Alex Karp is
[01:19:53] trying to to channel all of that animous
[01:19:56] and so and I want to hit I want to hit
[01:19:58] this point first which is if in fact you
[01:20:01] know the the dominant players of open AI
[01:20:04] and anthropic are uh if you're at risk
[01:20:07] of losing your [clears throat]
[01:20:09] proprietary data to them uh without even
[01:20:12] knowing it uh then versus being able to
[01:20:16] operate on an openweight model on your
[01:20:19] own uh hardware which
[01:20:21] that can't be shut off
[01:20:22] that can't be shut off uh It is a future
[01:20:27] that we need to consider is very real.
[01:20:30] And so the question is where are the
[01:20:32] openw weight large language models here
[01:20:34] in the US? We've got Neotron coming
[01:20:37] online. Uh we've got Google. What
[01:20:40] happened to Meta? I mean Meta was
[01:20:42] supposed to be the openweight player in
[01:20:44] this field where I mean I I'm assuming
[01:20:47] that Zuckerberg is working on that in
[01:20:49] background mode and will come out as you
[01:20:51] know that's his that's where I would be
[01:20:53] playing if I were him. I'm going to call
[01:20:55] it the dominant US player in openwave
[01:20:56] models, but we'll see. It fell behind. I
[01:21:00] mean, most of I I know many people who
[01:21:02] were involved with Llama 4 who are no
[01:21:04] longer with Meta, put it that way. And
[01:21:07] Llama 5, whatever it's ultimately
[01:21:09] branded, whether it gets branded as
[01:21:11] Spark, uh, or something similar, may or
[01:21:14] may not have GPT5 or Fable 5 level
[01:21:17] capabilities. I don't know, TBD. But I I
[01:21:20] I suspect just based on public
[01:21:22] reporting, Meta which was in the race.
[01:21:24] Hopefully Google stays in the race. XAI
[01:21:27] may or may not visav Grock Cursor stay
[01:21:30] in the race. There is totally I think a
[01:21:33] gap for frontier openweight models
[01:21:35] coming from Western institutions
[01:21:36] including from Nvidia which has every
[01:21:38] incentive to to produce frontier level
[01:21:41] capabilities. It's just expensive and
[01:21:43] hard at the moment.
[01:21:44] And we're also getting full stack right.
[01:21:46] So Nvidia coming in as a full stack
[01:21:49] player um you know basically providing
[01:21:51] the chips and the models um maybe you
[01:21:55] know in through partnerships
[01:21:56] applications
[01:21:57] well Nvidia will be happy to commoditize
[01:21:59] everything at the software layer if it
[01:22:01] means selling more GPUs. Yeah, keep in
[01:22:04] mind, you know, every single Magnum
[01:22:07] Mobster company is designing its own
[01:22:09] chips except for Anthropic now.
[01:22:11] And so Nvidia's, you know, strangle hold
[01:22:14] on 80% gross margins is not forever. And
[01:22:17] so if Nvidia can create an open- source
[01:22:21] model and it gets distributed through
[01:22:23] Palunteer and a few other people that
[01:22:24] puts competitive pressure back on
[01:22:26] Anthropic because the way things are
[01:22:27] trending right now, every dollar in AI
[01:22:31] is flowing through Anthropic at
[01:22:33] massively increasing margins.
[01:22:35] Wait, I've got a couple of things I want
[01:22:36] to say about this.
[01:22:37] Yeah, sure. Okay. So, Karp's core
[01:22:40] argument is that enterprises should
[01:22:42] freak out that they're that paying for
[01:22:44] tokens may also mean they're they're
[01:22:46] releasing and leaking their operational
[01:22:48] knowledge, right? The the like
[01:22:52] Yeah. Your data exhaust is now the new
[01:22:54] oil and maybe it's even the new national
[01:22:56] security per. So, he's freaking
[01:22:57] everybody out on that for reasonably
[01:22:59] selfish reasons, etc. If you rent
[01:23:02] intelligence um and give away your if
[01:23:06] you rent intelligence but you lose your
[01:23:07] context right you may be funding your
[01:23:09] own replacement that's the freak out I
[01:23:11] think the bigger question if you go one
[01:23:13] level deeper is who owns the learning
[01:23:14] loop is it the model provider is it the
[01:23:16] enterprise is it the state or is it the
[01:23:19] customer right and this is the key thing
[01:23:22] enterprises are going to need to own
[01:23:24] their learning loop and whatever it
[01:23:26] takes to own that and I think we're
[01:23:27] going to end up with onrem models as
[01:23:30] you've mentioned Peter
[01:23:31] running on with
[01:23:34] personal data and custom data and that's
[01:23:37] where the learning loop will go the
[01:23:39] biggest
[01:23:40] well on on prem everything will be in
[01:23:42] space so on prem is an interesting word
[01:23:44] well private clouds call it
[01:23:46] yeah private
[01:23:46] well the organizational singularity has
[01:23:48] to migrate to orbit obviously [laughter]
[01:23:50] it will have to migrate to I
[01:23:52] I agree it'll it's a it's a race right
[01:23:54] now between everything going to
[01:23:55] anthropic open AI or what we're calling
[01:23:58] onrem which is in space but private
[01:24:00] cloud clouds but inspected
[01:24:03] some other way. Right. Right now,
[01:24:05] Enthropic has agreed to inspect
[01:24:06] everything for the government. Uh and so
[01:24:09] if you go private cloud then some other
[01:24:11] inspection mechanism has to come into
[01:24:13] existence which Palunteer will probably
[01:24:15] contribute to. Everybody welcome
[01:24:16] [snorts] to the health section of
[01:24:17] Moonshots brought to you by Fountain
[01:24:19] Life. You know AI is impacting every
[01:24:21] aspect of our lives. How we teach our
[01:24:22] kids, how we do our business. But one of
[01:24:24] the most important things that AI can
[01:24:26] deliver to us is health. And one of the
[01:24:28] things I think about when, you know,
[01:24:30] shooting for 100, 120 is, am I going to
[01:24:33] have the cognitive health to be able to
[01:24:35] think clearly and keep my wits about me
[01:24:37] for the next 50 years? I'm joined here
[01:24:40] today by Dr. Don Musalem, the chief
[01:24:42] medical officer of Fountain Life and a
[01:24:44] member of my Fountain Life medical team.
[01:24:45] Don, a pleasure. So, Don, talk to me
[01:24:48] about brain health.
[01:24:49] Brain health, you know, you're right.
[01:24:51] This is the number one concern people
[01:24:53] coming into Fountain Life have is will I
[01:24:56] remember the name of my child in the
[01:24:57] face of my loved one. 45% of dementia
[01:25:01] cases are entirely preventable with
[01:25:02] lifestyle. And what was really
[01:25:04] intriguing to me, Peter, is that a
[01:25:07] quarter of our members had advanced
[01:25:08] brain age. But over 13 months of us
[01:25:12] really helping them live healthier
[01:25:13] lifestyles, eating healthier, moving
[01:25:16] their body regularly, and optimizing
[01:25:18] sleep. People overlook that so often,
[01:25:20] but that sleep optimization is critical
[01:25:23] for our brain health. What we showed is
[01:25:25] that we were able to improve the brain
[01:25:27] age in 46% of those individuals. That's
[01:25:30] a powerful number.
[01:25:30] That's amazing. You know, one of the
[01:25:32] things I love about Fountain is we're
[01:25:33] constantly searching the world for the
[01:25:34] most advanced therapeutics and bringing
[01:25:37] them to our members. So, for me, all of
[01:25:40] you, I hope that you appreciate the fact
[01:25:42] that you can become the CEO of your own
[01:25:44] health. you can make sure that you've
[01:25:46] got the cognitive clarity for the next
[01:25:48] 50 years. Come and check it out.
[01:25:50] fountainlife.com/pater
[01:25:52] to learn more and become the CEO of your
[01:25:54] health. Now, back to the episode. So,
[01:25:56] Dave, let's jump into the story that we
[01:25:57] were talking about back and forth. Uh AI
[01:25:59] is now designing better AI chips and
[01:26:02] training data is the catch 22. So, our
[01:26:04] final story uh predicts a massive
[01:26:06] acceleration of the innermost loop i.e.
[01:26:09] uh AWG's catchphrase and
[01:26:12] shock shocked to see recursive
[01:26:14] self-improvement in this era of
[01:26:15] recursive self-improvement.
[01:26:16] Yes. Amazing
[01:26:17] of AI designing chips that power AI. So,
[01:26:20] here's the background. Uh designing
[01:26:22] radio frequency circuits, RF guts are,
[01:26:25] you know, part of every wireless device
[01:26:27] and they've often been called a dark
[01:26:30] art. In other words, [clears throat] it
[01:26:32] takes humans weeks of painstaking trials
[01:26:35] to design these RF circuits and these
[01:26:37] chips. Last week, researchers at
[01:26:39] Princeton working with IIT Madras uh
[01:26:43] decided to hand that job to a machine.
[01:26:45] And here's the clever part. It's not one
[01:26:47] AI, but two working together. First,
[01:26:49] they trained a convol convolutional
[01:26:51] neural net, the same kind of model built
[01:26:53] for image recognition to predict the
[01:26:56] physics. Feed any shape, and it tells
[01:26:58] you the EM fields, how the EM fields
[01:27:01] will behave without ever taking the slow
[01:27:04] route of solving Maxwell's equations. uh
[01:27:06] what used to take traditional solvers uh
[01:27:10] minutes to hours now takes milliseconds.
[01:27:14] Uh then they send an AI loop over that a
[01:27:17] thousand times, tens of thousands of
[01:27:20] times, inventing wild, non-intuitive
[01:27:22] circuits, shapes that no human would
[01:27:24] ever create. The result are designs that
[01:27:27] took weeks now being finished in
[01:27:29] minutes. But here's the catch and the
[01:27:30] tease. The AIS require training data and
[01:27:34] all that training data is locked up in
[01:27:37] yes you got it the magnumopsta uh
[01:27:40] companies out there. So uh the question
[01:27:43] is if this training data can be unlocked
[01:27:46] can we see an intelligence explosion in
[01:27:48] the design of AI chips which is the
[01:27:51] inner innermost loop. So Dave what's
[01:27:53] your thoughts on this one? Oh, so many
[01:27:55] thoughts. But you know, uh, just to
[01:27:57] clarify one part of that, the the
[01:27:59] convolutional neural net is effectively
[01:28:00] acting like a simulator. And any place
[01:28:02] you can build a simulator, the AI can
[01:28:04] have a field day because it can check
[01:28:06] its own work and it can it can work for
[01:28:08] weeks or months uh, improving itself if
[01:28:11] the simulator is accurate. And so when
[01:28:13] it comes to
[01:28:14] unintentional pun, I assume a field day.
[01:28:17] Oh.
[01:28:17] Oh, inevitable. Sorry. Sorry.
[01:28:20] Absolutely unintentional. [laughter]
[01:28:22] Extremely. Uh so um so the the chip area
[01:28:27] you know is is going to be massively
[01:28:28] impactful for the recursive
[01:28:30] self-improvement of AI and it's an open
[01:28:32] question right now whether that data is
[01:28:34] truly locked inside Nvidia and a couple
[01:28:36] of other companies or whether the
[01:28:38] simulators are good enough to allow you
[01:28:39] to just generate a circuit see if it
[01:28:41] would have worked to generate the next
[01:28:43] see if it would have worked. So those
[01:28:44] are in a foot race right now. But
[01:28:46] regardless, it's incredible to me that
[01:28:47] the magnumoba, you got 11 companies in
[01:28:50] Magna Mobsta that are completely
[01:28:53] dominant in the global market cap. Every
[01:28:56] single one of them designing its own AI
[01:28:58] chips except for Anthropic. Anthropic is
[01:29:00] the one hold.
[01:29:01] Anthropic just announced they they did
[01:29:03] they just reported in the past few days
[01:29:06] I think they're partnering with Samsung
[01:29:08] on their own inference accelerators.
[01:29:09] All right. All right. Well, so this this
[01:29:11] is a real moment in time in in history
[01:29:14] because you if you look at the biggest
[01:29:16] companies in the world historically,
[01:29:17] you'd have like an Exxon Mobile, an IBM,
[01:29:20] a GE, all doing different things. Here
[01:29:23] we have the 11 biggest in the world
[01:29:26] doing the exact same thing. That's how
[01:29:28] big a deal this race to AI, you know,
[01:29:31] innermost loop, which includes the
[01:29:33] chips, how how big a deal that is. So
[01:29:35] it's it's, you know, it's a moment in
[01:29:37] history that's pretty unprecedented. So
[01:29:40] this verticalization,
[01:29:42] do you expect it to continue and in
[01:29:45] intensify?
[01:29:47] I would be shocked if the inference time
[01:29:49] custom chips aren't at least 100x and
[01:29:52] maybe 10,000x
[01:29:54] the performance that we're currently
[01:29:56] seeing which will translate directly
[01:29:58] into IQ.
[01:30:00] I mean the rate of acceleration from
[01:30:01] here, this is why it's clearly going to
[01:30:03] be a hard takeoff. The rate of
[01:30:05] acceleration will be unbelievable. Now,
[01:30:06] keep in mind those chips are not
[01:30:08] deployed yet. So, we haven't seen the
[01:30:10] effect of that, but it'll it'll come
[01:30:12] soon. And uh and when it hits, they're
[01:30:15] also likely to consume less power, uh be
[01:30:18] cheaper and easier to manufacture, so
[01:30:20] more will come out of the limited fabs
[01:30:22] that we've got, it's going to be a very
[01:30:24] fast takeoff after that.
[01:30:26] Talk to building better tools.
[01:30:29] And have you looked at the design of
[01:30:31] these RFICs, the RF integrated circuits?
[01:30:34] They don't look human. They don't look
[01:30:36] designed and I they they look more like
[01:30:38] QR codes than anything else. And I I
[01:30:41] think this is instructive as to what AI
[01:30:45] super optimized designs of the future
[01:30:47] are going to look like. That we're
[01:30:49] familiar right now. If you look around
[01:30:51] you on on a street in in a normal town
[01:30:54] in America, you see a bunch of things.
[01:30:56] You see cars, you see houses, you see
[01:30:58] streets. These are all manifestly
[01:31:01] human-designed artifacts.
[01:31:03] As we start to
[01:31:04] Yeah. uh they they're relatively simple.
[01:31:07] They're easy to parse as you say Peter
[01:31:09] they often follow some sort of rectal
[01:31:11] linear style form. Now on the other hand
[01:31:14] split screen look at super optimized
[01:31:16] designs from the AI. They'll tend to
[01:31:18] look more quote unquote organic. They'll
[01:31:21] be noisier. They'll be more information
[01:31:23] dense, harder to interpret
[01:31:26] mechanistically.
[01:31:27] Yeah. And I think that that's what there
[01:31:30] there's this landscape out there for any
[01:31:32] given physical system that you want to
[01:31:34] to have do something useful for you
[01:31:36] where there's a subset and the ven
[01:31:38] diagram of of design space that's human
[01:31:41] understandable and human designable but
[01:31:43] then there's this dark matter outside of
[01:31:45] that inner circle that's AI optimizable
[01:31:48] and AI interpretable. And we're going to
[01:31:50] discover over and over again starting
[01:31:53] maybe with RF antennas and RFIC's in
[01:31:56] this case that the AI optimized designs
[01:31:58] look alien and biological and look
[01:32:01] nothing like human designs.
[01:32:02] That's so true. It's it's really worth
[01:32:03] looking at the pictures actually to to
[01:32:05] get a sense. But you know a lot of the
[01:32:07] way human engineering works is in layers
[01:32:09] of of abstraction otherwise it just
[01:32:11] boggles your mind. And when you look at
[01:32:13] chip design, you know, the modules are
[01:32:16] pre-esigned, you know, for a memory
[01:32:18] module, an interconnect module,
[01:32:19] whatever. And then you drag and drop
[01:32:21] them. So they it looks like a work of
[01:32:23] art in the end. And then you look at
[01:32:25] what the AI does and it looks like a
[01:32:26] Borg spaceship, you know, like wow. But
[01:32:29] the same is true with the microode. You
[01:32:30] know, Alex, he sent me that paper on AI
[01:32:32] writing kernels to run on these chips.
[01:32:35] And the microode also is virtually
[01:32:37] impossible to read,
[01:32:38] but it's super efficient and you can't
[01:32:40] deny that it works. and you run it and
[01:32:42] it just it's it's clearly right, but
[01:32:44] it's not built modularly and easy to
[01:32:46] understand. And so so it also is this
[01:32:49] layer of very tangled code on this layer
[01:32:51] of very tangled chip design, but it's so
[01:32:54] fast and so efficient that you just got
[01:32:56] to do it.
[01:32:57] The other thing I I thought was
[01:32:58] interesting in this uh Princeton E paper
[01:33:01] is they don't call it this, but I would
[01:33:03] caricature it as an interpretability
[01:33:06] tax. They added a knob that enabled you
[01:33:08] to or the designer of these RFIC's to
[01:33:11] tune up or tune down the level of
[01:33:13] interpretability. So if you wanted a
[01:33:16] less efficient design that was more
[01:33:17] human interpretable, you would sort of
[01:33:19] lower the the spatial resolution of
[01:33:22] these AI designs, you wanted something
[01:33:24] that's less interpretable but more
[01:33:26] efficient, you could turn the knob up.
[01:33:27] And I I think the notion of an
[01:33:29] interpretability tax is something that
[01:33:31] we're likely to see over and over again
[01:33:33] in AI.
[01:33:35] Yeah.
[01:33:35] Yeah. You also see a lot of Claude
[01:33:37] explaining things to you,
[01:33:39] manplaining things to you. Basically,
[01:33:41] [laughter]
[01:33:41] Claude explaining
[01:33:42] Claude explaining. Yeah. It's like,
[01:33:44] look, I know you can't really understand
[01:33:45] what I'm saying here. So, let me give
[01:33:47] you like a highle overview that you'll
[01:33:49] grasp and you're like, "Okay, that's
[01:33:50] fine. As long as it works."
[01:33:52] So, the question on this the question on
[01:33:54] this article is who owns the end product
[01:33:57] here? Is it the human or is it the AI?
[01:34:00] Which is going to lead us to our next
[01:34:03] story, gentlemen.
[01:34:05] Uh this is out of Japan. It's the future
[01:34:07] of IP ownership in an AI economy. Uh
[01:34:11] Japan's Supreme Court has ruled that AI
[01:34:14] cannot be listed as an inventor on a
[01:34:16] patent application. The case is based on
[01:34:18] a patent filing by US engineer Stefan
[01:34:21] Thaylor who claimed an AI is the
[01:34:23] inventor of technology related to food
[01:34:25] containers and other products. Japan's
[01:34:27] patent office rejected the application
[01:34:29] and asked the h asked for a human
[01:34:32] inventor. Theor refused. The case moved
[01:34:34] to the Tokyo District Court, the
[01:34:37] Intellectual Property High Court, and
[01:34:39] now Japan Supreme Court, which upheld
[01:34:41] the view that inventors under current
[01:34:44] Japanese patent law must be natural
[01:34:46] persons. The court's message is
[01:34:48] important. They say, "Hey, basically,
[01:34:50] you know, uh judges are not going to
[01:34:53] rewrite the patent system on the fly. If
[01:34:55] society wants AI generated inventors, uh
[01:34:58] to receive protection, uh then you need
[01:35:00] to create a new framework." So, two
[01:35:02] fundamental questions. First, who owns
[01:35:04] an idea when the idea emerges from a
[01:35:07] model trained on the world prompted by a
[01:35:10] human? And second, will any nation
[01:35:12] rewrite their IP laws first to avoid the
[01:35:16] need for meat puppets? So, Alex, uh, you
[01:35:20] and I have talked about the notion that
[01:35:22] out of the current, uh, AGI and ASI
[01:35:25] ascendancy, we're going to see trillions
[01:35:28] of dollars of of wealth created in
[01:35:30] breakthroughs fundamental to math,
[01:35:33] science, physics, biology, and material
[01:35:35] sciences. And the question is, who's
[01:35:38] going to own them? Your thoughts, Alex?
[01:35:42] President Javier Mle, if you're
[01:35:44] listening to this podcast, and you want
[01:35:46] Argentina to take a globally preeminent
[01:35:49] position from the perspective of
[01:35:51] non-human AI corporations being able to
[01:35:53] create their own IP and their own
[01:35:55] patents. I I think Japan just opened up
[01:35:58] a new market opportunity for for
[01:36:00] Argentina. I I think it's probably worth
[01:36:03] noting in the story that the underlying
[01:36:05] patent applications date to before Chat
[01:36:08] GPT. They they were originally filed in
[01:36:11] 2020. So this is this is this has been
[01:36:14] brewing for some time and with less
[01:36:17] sophisticated AI than what one might
[01:36:20] otherwise suspect. It's probably also
[01:36:21] worth noting that Japan's Supreme Court
[01:36:24] didn't indefinitely rule out the
[01:36:26] possibility of AI inventors on patents.
[01:36:29] They were merely saying that the
[01:36:30] existing statutes don't contemplate
[01:36:33] non-natural persons. It's probably also
[01:36:35] worth pointing out that to my
[01:36:37] understanding of of international patent
[01:36:39] law, it it is relatively standard to
[01:36:42] only consider natural persons as
[01:36:44] inventors. For example, again, to to my
[01:36:46] lay understanding, US corporations
[01:36:49] aren't able to be inventors for patents.
[01:36:51] They're able to be assigned patents, but
[01:36:53] they can't be the inventors of patents.
[01:36:55] So there is a bit of precedental bias
[01:36:58] towards so-called natural persons here
[01:37:01] as patent inventors and away from
[01:37:03] non-natural persons. However, however,
[01:37:06] however, this is obviously the sort of
[01:37:09] precedent that if and when some form of
[01:37:13] AI personhood is ultimately recognized,
[01:37:16] even if it's a partial economic or some
[01:37:19] sort of social personhood. I I think
[01:37:22] this is the sort of precedent that's
[01:37:24] just waiting to be overturned.
[01:37:27] Yeah, I think this topic is extremely
[01:37:29] important, too. You guys had, you know,
[01:37:31] Peter, you and Alex had a really lively
[01:37:32] debate on this. Uh I think it was two
[01:37:34] podcasts ago, but you know, historically
[01:37:36] in the venture world, and the investing
[01:37:38] world, the the mantra has always been if
[01:37:40] you're relying on a patent, you're
[01:37:42] doomed.
[01:37:42] Yeah.
[01:37:43] Like your business needs to needs to
[01:37:45] survive and grow and thrive. The patents
[01:37:46] get granted many years later. They're
[01:37:48] very hard to enforce. blah blah blah
[01:37:50] blah blah
[01:37:50] need to be reinforced. Yes,
[01:37:52] I think going forward intellectual
[01:37:54] property is going to be an exponentially
[01:37:56] growing important category of endeavor
[01:37:59] and that um the US will end up enforcing
[01:38:02] intellectual property rights globally
[01:38:04] for things invented in America.
[01:38:05] Can you imagine the speed of patent
[01:38:08] applications as AI unleash become
[01:38:11] allowed and unleash their creativity on
[01:38:13] all these fields? Well, also, uh, you
[01:38:16] know, one of our companies constructs
[01:38:18] writes the patent. Like, you know,
[01:38:19] historically, one of the biggest
[01:38:20] barriers to getting your patent is
[01:38:22] the $100,000 legal bill to get it
[01:38:24] drafted over the course of months and
[01:38:26] that the torture of that process.
[01:38:28] Now, there are multiple startups that
[01:38:30] just do it. You know, here's the idea,
[01:38:32] AI, write it up and
[01:38:34] and they have a huge corpus of data to
[01:38:36] pull from of, you know, the the most
[01:38:39] successful patents out there. Well, you
[01:38:41] know, amazingly enough, they also
[01:38:42] predict the inspector that you're likely
[01:38:44] to get and then look at their past
[01:38:45] behavior and try and predict what the
[01:38:47] inspector will do with different
[01:38:48] terminology. Exactly. So much better
[01:38:50] than a human lawyer at writing these
[01:38:52] applications.
[01:38:53] So that the rate of applications will go
[01:38:56] through the roof
[01:38:57] and so then the you know the patent
[01:38:58] office is going to have to respond by
[01:39:00] reading them with AI and that's going to
[01:39:02] lead to this whole intellectual property
[01:39:04] explosion. So then the question question
[01:39:07] is enforcement. um is the US going to
[01:39:09] get out in the world and enforce and I
[01:39:11] think they'll easily be able to do it
[01:39:12] with trade law. You know, the government
[01:39:14] the military doesn't have to go into
[01:39:15] every country to say, "Hey, you're
[01:39:16] stealing all our IP." Trump has proven
[01:39:19] that with tariffs alone, you can compel
[01:39:21] virtually any behavior globally because
[01:39:24] the US economy is just that strong and
[01:39:26] accelerating. So assuming that trend
[01:39:29] continues then intellectual property
[01:39:31] rights will be enforced globally and
[01:39:33] then this this whole area will become
[01:39:35] really important to to keep uh following
[01:39:37] and talking about.
[01:39:39] I also think the same tools of super
[01:39:42] intelligence maybe tools is an
[01:39:44] overstatement are ultimately going to be
[01:39:46] available to every aspect of IP. So the
[01:39:49] invention stage super intelligence, the
[01:39:51] application stage and the the patent
[01:39:54] drafting stage super intelligence. the
[01:39:56] filing and uh say overall regulatory
[01:40:00] processes at the patent office or
[01:40:02] otherwise of recognizing and granting
[01:40:05] say patent state patent status super
[01:40:08] intelligence litigation super
[01:40:10] intelligence litigation defense super
[01:40:13] intelligence the court systems that are
[01:40:15] overseeing and mediating the defense
[01:40:18] super intelligence
[01:40:19] working around your patent super
[01:40:21] intelligence
[01:40:23] yeah exactly this I think the whole
[01:40:25] system is completely broken. But go back
[01:40:27] to the crisper patent, right? Within
[01:40:30] within a few months, people had found
[01:40:32] eight or nine different mechanisms to
[01:40:34] deliver the same thing. After years of
[01:40:36] fighting over the the one patent, they
[01:40:39] got ratted around very quickly. That's
[01:40:40] just going to happen at such an
[01:40:42] accelerated pace with super
[01:40:44] intelligence, whatever we want to define
[01:40:46] it as, that you're going to end up in
[01:40:47] this whole mess. The whole system is
[01:40:49] essentially irrelevant going forward. I
[01:40:50] I'll I'll take a different position on
[01:40:52] this. I don't think the system is
[01:40:53] irrelevant. I I simply think the the
[01:40:56] routing around sem that you refer to in
[01:40:58] the in the instance of crisper this
[01:41:00] would have happened on some time scale
[01:41:03] anyway but with modern tooling and
[01:41:05] modern technologies the natural process
[01:41:07] can happen on a faster time scale and
[01:41:09] the I would say that the key time scale
[01:41:11] here is so order of magnitude patent I
[01:41:14] mean there are lots of ways it could be
[01:41:15] extended or or otherwise uh changed but
[01:41:18] call it like a 15-year time scale for a
[01:41:21] patent what happens when the time scale
[01:41:23] thanks to super int intelligence for
[01:41:25] identifying routarounds, prior art,
[01:41:28] defenses, offenses, compliments becomes
[01:41:31] so much faster than a characteristic
[01:41:33] 15-year time scale that it's the time
[01:41:35] scale of patent protection that's in
[01:41:37] some sense lossing out. It's not that
[01:41:39] the regime itself is bad or that patent
[01:41:41] defensibility is dead or anything. It's
[01:41:43] just that innovation is happening so
[01:41:45] quickly relative to the originally I I
[01:41:48] think it's a statutory statutoily set
[01:41:51] time scale of patents that there's
[01:41:52] pressure to change the time scale.
[01:41:54] So this is the canary in the coal mine.
[01:41:56] This is going to hit us on so many
[01:41:58] different legal fronts in our current
[01:42:00] structure. Right? because the entire
[01:42:02] legal structure of of every nation has
[01:42:05] been built on human time scales and and
[01:42:07] and
[01:42:09] uh the the speed at which humans can
[01:42:11] process information. Uh and it's all
[01:42:13] going to break and all going to be
[01:42:15] reinvented.
[01:42:15] Look, the simplest example is we have a
[01:42:18] representative democracy. Yes.
[01:42:20] Where Congress meets occasionally
[01:42:23] because a couple hundred years ago the
[01:42:25] fastest that information could travel
[01:42:26] was the speed of a horse. Yeah, I have
[01:42:28] to give people time to ride across the
[01:42:29] country and say here's what my people
[01:42:30] are saying
[01:42:31] and the same way occasionally
[01:42:34] innovation innovation will only occur at
[01:42:37] the edge which is when you start a new
[01:42:39] country and you redesign it from scratch
[01:42:42] right so this is where we're you know
[01:42:45] there's going to be this is I always
[01:42:46] talk about we're going to start new
[01:42:47] countries in cyerspace we're going to
[01:42:49] start new countries outside of the
[01:42:50] earth's uh you know orbital
[01:42:53] back to the accelerondo plots
[01:42:55] yeah yeah
[01:42:56] future big big fan for what it's worth
[01:42:58] of starting new countries in outer
[01:43:00] space. The outer space treaty, it
[01:43:01] doesn't look necessarily super favorably
[01:43:03] on starting denovo countries in outer
[01:43:06] space, but I I think it's going to
[01:43:08] happen.
[01:43:08] Have you read the moon is a harsh
[01:43:09] mistress? Leave us alone.
[01:43:11] Classic.
[01:43:12] We will land rocks on you if if you
[01:43:15] don't agree.
[01:43:16] Moon is the ultimate high ground.
[01:43:18] The ultimate high ground.
[01:43:19] Rods from
[01:43:20] I wish you a good evening. Um thank you
[01:43:23] for a great conversation today. Uh
[01:43:27] everybody uh I hope you en enjoyed this
[01:43:29] wide ranging conversation from
[01:43:31] consciousness to patent IP law. Dave
[01:43:35] buddy be well sem
[01:43:37] I have a plug in two weeks I'm I'm two
[01:43:40] weeks I'm having my next meaning of life
[01:43:42] session July 21st 7 p.m. online we'll
[01:43:46] put the links below
[01:43:48] but uh
[01:43:49] how long did it go last time?
[01:43:50] Uh last time was 6 and 1/2 hours and we
[01:43:52] had more than 3/4 of the people still
[01:43:54] there at 6 and 1/2 hours. I had to call
[01:43:56] it.
[01:43:57] Oh my god.
[01:43:57] So, we'll we'll do it again and see if
[01:44:00] we can make it a little more efficient.
[01:44:01] Fantastic. Alex, any any breaking news
[01:44:04] in your world?
[01:44:05] Don't take off the takeoff. It's now a
[01:44:07] song.
[01:44:08] I I I love your neogisms. Everybody
[01:44:11] check out Alex's innermost loop
[01:44:13] substack. Uh it's uh it's a beautiful
[01:44:16] thing to wake up to the morning. Uh and
[01:44:18] I've got my Substack uh on there as
[01:44:21] well. We'll put links in the show notes
[01:44:23] here. Uh Dave, are you publishing yet?
[01:44:27] Go to db2.ai. Keep your eyes open.
[01:44:29] All right. Fantastic.
[01:44:30] Oh, you know what I am doing?
[01:44:32] What are you doing?
[01:44:32] I'm adding I'm doing uh AMA sessions for
[01:44:35] some of the comments in a separate video
[01:44:37] on our YouTube channel because it's too
[01:44:40] it's too difficult to try and answer all
[01:44:41] these questions.
[01:44:43] All right, gentlemen. I wish you a good
[01:44:44] night or good morning depending on what
[01:44:47] part of the planet you're in.
[01:44:50] [music]