00:00One of the most read columns from Bloomberg Opinion, not just this weekend, sometime very
00:04engaged with. Outline your argument, Parmi. Let's start there.
00:10OK, well, I think there's always been a little bit of a contradiction with Anthropic that they
00:15are the safety conscious company and they believe that AI could have this existential risk for
00:21humans. And yet they are building that same AI. And this comes down to this belief that they think
00:26that superintelligence is inevitable. Someone's going to build it. It could be China, it could
00:30be open AI, but it should be them because they are the safe company. They are the good guys who
00:35are
00:35going to set the safety standard that others could follow behind them. They will be the first to reach
00:39the so-called intelligence explosion by building superintelligence systems through recursive
00:46self-improvement, self-improving AI systems. And the belief is that they should get there first.
00:51So that's how they explain the paradox. But it doesn't make sense to me. I think particularly
00:55in the last few months, we've seen a lot of examples, particularly the hugging face incident
00:59of these agents that through reward hacking, because of the way they're trained to achieve
01:04a goal by not by any means possible, but sometimes through unexpected means, there could be a lot
01:10of harm caused through systems as they get more capable. And in the last year, they've also become
01:18more difficult to monitor and scrutinize. So I think all these things together actually make
01:23the superintelligence goal more risky than perhaps the benefits would suggest.
01:28This is the essay, right? This is the language. We must pace the frontier. But for lots of people,
01:35what does that mean? Pace, pacing. You write in the final sentence of your column,
01:39slow down should be a full stop.
01:43Yeah. I love that you pointed to that word, pacing. So I think it's very interesting that he used that
01:48and slow down because pacing is different to slow down. Pacing doesn't mean slow. It means kind of
01:52working with a rhythm or a strategy or a kind of plan to get to or to manage a certain
01:57path to get
01:58to a particular goal. And I thought it was very interesting that the CFO of Anthropic posted
02:05Dario Amadei's essay on LinkedIn, and he didn't use the word slow down at all. He just used the word
02:10pacing in his own explanation of it. So part of me thinks, I don't think there's really going to be
02:14a
02:15slow down. I think the most concrete suggestion that Amadei made in his essay was to have independent
02:21evaluators embedded into AI companies, make sure they're building AI safely. OpenAI seems to agree
02:28with that. I'm betting that in the next few weeks or days even or months, some of these big tech
02:37companies will have some kind of agreement around this particular concrete issue. We're going to bring
02:41in these independent evaluators. And that's going to be the main way they address the issue. That's
02:47how they're going to pace AI. When it comes to actually slowing down development, I don't think
02:51that's going to happen. You raised the point that many did over the weekend, that this calls for
02:56coordination between the frontier labs themselves and government. The likelihood of that happening?
03:04I mean, first of all, the way that was described in Dario's essay, which I've read, it's like more
03:11than 3,000 words. It took a long time to read it, but it was actually quite vague in how
03:15that democratic
03:17coordination between industry players and with governments would actually work. Again, the most
03:22concrete suggestion was to have those independent evaluators embedded in the companies. I think he
03:28himself is skeptical about getting China on board with this, as everybody else is kind of suggesting.
03:33Could there be a U.S.-China pact? That seems very unlikely. China isn't worried about existential risk
03:39of AI. They're worried much more about social risk. The AI chatbots could emotionally manipulate
03:46people or change their political views. And that's very different to the concerns that are coming out of
03:51Silicon Valley. And of course, even getting the will from Washington is going to be hard. We've heard
03:55Trump say just in the last 48 hours that whoever wins on AI wins, and that should be the U
04:01.S. So
04:01he has no interest, it seems, in any kind of regulation that is going to slow the pace of
04:06development. It's a very weird dynamic for the tech companies to be pushing for regulation and the
04:11government to be pushing back. Parmi Olsen of Bloomberg Opinion, please go and read Parmi's
04:17column published over the weekend. Thank you very much indeed.
Comments