- 3 minutes ago
WIRED Senior Correspondent Lauren Goode speaks with AI researcher Timnit Gebru about the existential threat of artificial intelligence. Lauren and Timnit also talk about deep-state regulatory capture, the reality of the ‘mind behind the machine’ and the role of human agency in a world rapidly shifting towards autonomous systems.
Category
🤖
TechTranscript
00:00Timney, thank you so much for being here and joining me on the big interview this week.
00:03Thank you for having me.
00:05A few weeks ago, a young AI researcher, Edanthropic, basically stepped down and in his resignation post on X indicated
00:13that a lot of people within the company feel that we are facing this major existential risk with AI.
00:21And then a few people responded and there was this whole pile on and discourse about the real existential threats
00:27of AI.
00:27And when you and I spoke in the immediate aftermath of that, you said that you felt that particular narrative
00:35was a distraction from the real problems that we're facing with AI.
00:40You know, I would even go further than that and say that it's dangerous.
00:43It's not just a distraction, but it's harmful discourse.
00:47And so the reason being, before I answer your question, you know, I try to do this, I want to
00:53give a little brief history of how this narrative has been going on for a long time.
00:58Because I wrote a Wired op-ed in 2022.
01:02We wrote a statement in 2023 when this happened.
01:05Elon Musk and Peter Thiel have been saying this since 2013.
01:08Prominent people have been saying that AI is an existential risk to humanity.
01:13And the way it goes is usually, and this is exactly the same rhetoric every three years.
01:18So I can regurgitate whatever I wrote 10 years ago to respond to these things.
01:24For example, the Future of Life Institute, which now is advising.
01:27And what is that?
01:28Yeah, the Future of Life Institute is an institute that was founded by Max Tegmark and Jan Talen, a billionaire
01:35who led Anthropics Series A funding.
01:39And he was a Skype co-founder, correct?
01:41Skype co-founder Jan Talen also funds Meter, which is an auditor, so-called third party, whose report went viral
01:49about these rogue agents hacking, hugging face.
01:52So to the public, it might seem that there are so many different entities coming together saying the same thing.
01:59But just like the tobacco industry and fossil fuel industry playbook, there's a playbook here where the same billionaires who
02:07founded and funded Anthropic stand to benefit so much more from its IPO more than anybody else.
02:14It's also founded and funded the institution's warning about existential risk, quotation marks of AI.
02:22Also founded and funded the third parties that are being cited right now as if they are like an independent
02:27entity.
02:28So that I want to ground the public in that fact.
02:31And I can say more about that.
02:32Tell us why you thought the machine god narrative is a distraction.
02:37It's even more than a distraction.
02:38It's harmful.
02:39For example, the reason I was telling you all these facts is that these people, the funders, the founders, the
02:45investors who stand to benefit and profit the most from these companies IPO-ing are saying this.
02:53That they have been the ones seeding this narrative going back multiple decades, actually.
02:57People should ask why.
02:59If I stand to get lots of money from a particular company, why do I seem to be the person
03:04saying this particular company might build something that kills us all, right?
03:08It sounds counterintuitive.
03:10But the first thing is that if you convince the public that you're building something so powerful that it's beyond
03:20even your control, it's beyond something that we've seen before.
03:24It's beyond something that we don't have existing regulation for.
03:30And this is why I have a lot of respect for Lina Khan because one of the things she said
03:35is that there's no exception to the current laws that we have for AI companies and that they talk as
03:40if there is an exception.
03:42You're saying that whatever you're building is so powerful that it's beyond anything we've seen before.
03:48Right.
03:48It's self-aggrandizing.
03:49It's saying we're the future makers here, but the future is going to be really scary.
03:54They don't always say the same people.
03:56You have to pay attention to what they say in the news cycles.
03:59When they say that AI can cause an existential risk, they also say that AI can stop climate change, stop
04:07world, bring us world peace and eradicate poverty.
04:10AI can do all of these things because it's powerful.
04:12It can bring us utopia, but also if the wrong people do it and we don't have guardrails or whatever,
04:18it can also just kill us all.
04:20So the same people believe these things.
04:23There's only a few people who might be in one of these camps.
04:26When you're telling people that you have the super powerful machines, what you're speaking to, first of all, is to
04:32your investors.
04:32You're saying that you want to get your hands on these super powerful machines.
04:37You're talking to governments saying you don't want your adversaries to get their hands on these super powerful machines.
04:42You want to get your hands on these super powerful machines.
04:45And you're also telling any regulating bodies that whatever regulation they're thinking about should be about these fictional super powerful
04:54things.
04:54And imagine if I'm talking about potentially eradicating all of humanity, things like pollution, data centers, they sound kind of
05:04small potatoes, right?
05:06You're talking about copyright protection for artists, actual losses that are going on right now.
05:12Government should not waste its time thinking about this stuff.
05:14If you waste your time regulating us on these small topics, you risk China getting its hand on these super
05:22powerful machines.
05:24And you don't want that because at least us, trust us, we can build the machine God that can help
05:30us.
05:30And we're telling you we want to be regulated.
05:33But you don't want China to be the one.
05:35That's the first one.
05:35The first one.
05:36The second one, second issue is you can always abdicate responsibility.
05:42In our prior conversations, we've talked about the usage of certain language or phrases to describe this modern era of
05:49AI and how you take issue with some of them.
05:51So, for example, in our previous conversation, you said something about autonomous weapons.
05:55After that was published, you told me you later regretted saying that because you would have phrased it differently.
06:00We've talked about PDOOM.
06:01We've talked about how when you say that agents have gone rogue, such as OpenAI, you're essentially removing culpability from
06:07the engineers who built those tools.
06:09I can't help but wonder if maybe there are groups that are, we've even talked about the word alignment.
06:14So I was just going to say I wonder if there are groups that are actually more aligned than they
06:19realize around safety and transparency, but who are deploying different phrases to describe what is essentially the same thing.
06:28Some of the reasons that I regretted saying these words is not that the term autonomous weapons has, I have
06:35any issue with that.
06:36It's a legal term that has, even landmines are considered autonomous weapons legally.
06:41It's just that the people in the so-called existential risk camp have co-opted it to mean something different.
06:48When I said the term, I knew I'm like, huh, I wonder if this is going to be taken in
06:52that way.
06:52And it was because the people talking about existential risks of AI took it to mean what they're saying, right,
07:00which is like some kind of superhuman, superintelligent machine doing stuff on its own or whatever.
07:05Just to bring it back to the latest conversation about this, I was at the U.N. General Assembly this
07:09week.
07:10I was a part of some side panels, so I wasn't actually in the U.N. GA nor the Security
07:14Council.
07:15But there were some prominent tech folks there, including Sam Altman from OpenAI and Dario Amadai from Anthropic.
07:21Dario reportedly said, if managed poorly, I even believe AI could be a risk to humanity as a whole.
07:28His competitor, Sam Altman, said we could lose control of the future to AI.
07:33So explain how it is that these folks who once again have these companies that are valued at nearly a
07:43trillion dollars or more are now here on the world stage at the U.N. saying, I don't know, there
07:48could be some risks here, right?
07:51And is this part of regulatory capture?
07:55Are they going to be looking for ways to self-govern before there's a regulatory crackdown on this?
08:01Absolutely. It's part of regulatory capture.
08:03And in the same paper where I introduced a test reel bundle, I talk exactly about this regulatory capture and
08:09especially what has happened at the U.N.
08:12Briefly after, you know, Sam Bankman-Fried, as you know, was an effective altruist.
08:15And briefly after he went to prison, the effective altruists were briefly scrutinized by journalists.
08:21And journalists were talking about, it was a political article, for example, about the regulatory capture.
08:25Let's talk about right now. What can be done? What kind of governance do you think is needed for AI?
08:31So, I mean, we've been talking about regulatory capture has been happening for a long time.
08:36In 2023, same thing. Sam Altman said the same thing.
08:39We need the world cooperation, et cetera, et cetera.
08:42The EU AI Act regulated and then he threatened to pull out of the EU.
08:46So you just have to see what they've actually done.
08:49When there is actual regulation that holds them liable, they threaten to pull out or they lobby super hard to
08:56water down that regulation.
08:58On the other hand, they're going around telling these multilateral bodies that there has to be world cooperation, et cetera,
09:05et cetera.
09:05And then again, what is this doing?
09:07It is A, selling themselves as organizations that are creating super intelligence, right?
09:13And so that's already marketing and B, they're making everybody scared of anybody else who might be creating such things.
09:22So they don't want open rate models from China.
09:24They don't want this competition.
09:25I am not following Chinese regulation that closely, but they're not talking about existential risk.
09:31They're talking about deep fakes and they're talking about like real things that need to be regulated.
09:35So the first one is marketing yourself as creating some super powerful, unprecedented things for which we don't have existing
09:45regulation, which is not true.
09:46We have existing regulation.
09:48But the second one is what they're doing is when someone tries to enforce the existing regulation, they either threaten
09:55to pull out or they lobby hard so that they don't have this existing regulation.
09:58And so the reason I keep on going back to 10 years ago, five years ago, three years ago, is
10:04that the same in my book, I say the same movie on repeat with different heroes.
10:09So what is the solution for governance right now if you had to propose it?
10:13Very, very simple things.
10:15First of all, Lina Khan outlined five things.
10:19One is deceptive marketing practices.
10:21There's a law for deceptive marketing practices and you can go after companies for that.
10:26Two is transparency, documentation.
10:29Before you put something out there, you should be able to tell us where all the data came from and
10:35actually document this simple thing they don't do and they will never do.
10:39I'm going to tell you that they will fight tooth and nail to do the simple thing of documenting data
10:44and labor exploitation of data workers.
10:47That's another one that they don't want to talk about.
10:49We are here in the world, in the clouds, talking about superintelligence, where you have hundreds of millions of people
10:56around the world painstakingly labeling data, even pretending to be chatbots.
11:01There are data workers called AI impersonators, right?
11:05We now know that OpenAI hires people to look at your conversations with their chatbots.
11:11So people should be careful.
11:12Don't believe that, you know, you have privacy.
11:14Right, 404 Media just recently reported that the new MetaMuse chatbot is actually a person on the other end who's
11:21responding.
11:22Things like this, very simple, data transparency, labor exploitation.
11:26You should not be able to steal data from people.
11:29Even the first three things I talked about, data transparency, documentation, labor exploitation.
11:33If they had to abide by laws like that, and they were not allowed to steal data and not documented,
11:42the market calculation, right now the market calculation is not working.
11:46But with these additional measures, it just would not work whatsoever, right?
11:50So you would automatically slow them down and have to make them accountable for something.
11:57Back in 2021, you co-authored a paper that was titled On the Dangers of Stochastic Parrots.
12:02Google had approved it initially.
12:04Then it was being reviewed.
12:06There were parts of it that were in dispute.
12:08You said, look, if you want me to remove my name from this, I'm not going to be a part
12:13of this.
12:13You ended up decamping from Google.
12:16That paper was later presented at the 2021 ACM Conference on Fairness, Accountability, and Transparency, and people still reference it.
12:23Can you briefly explain for someone who's never heard of stochastic parrots before what it has to do with the
12:27AI we're using today, what it means?
12:29Stochastic parrots is a metaphor to help people understand what large language models do.
12:34And large language models are trained on vast amounts of textual data on the internet, trained to output the most
12:40likely sequences of text, given their training data.
12:43They power most of the chatbots that we see today, whether it's cloud, whether it is ChatGPT.
12:49But when I wrote this paper, ChatGPT hadn't come out yet, but we saw the race to build larger and
12:54larger language models.
12:55And so that's the danger of building larger and larger language models that we were describing in this paper.
13:01That they would essentially parrot people?
13:04To parrot is to repeat back without understanding, right?
13:07So there was this whole existential risk narrative was happening back then, too, if you can believe it.
13:14And so there was all this conversation about how OpenAI had claimed that GPT-2, the precursor to GPT-3,
13:21that powers ChatGPT was too dangerous and too powerful to release.
13:25There was this whole conversation about whether GPTs can be ethical or are they creative and all this stuff.
13:31And so we really wanted to ground the conversation in the real issues.
13:35One of them, one of these issues is the environmental catastrophe, which a lot of people are now seeing, but
13:41we discussed it back then.
13:43And that was one of the main sections that Google people were unhappy with, the environmental cost.
13:48The other one is not documenting your data because you say you have too much data to document.
13:53The other one is deceiving people into believing that there is a mind behind the textual outputs that they're interacting
14:01with.
14:02And so there, that's where we really wanted to explain that these systems are parroting the patterns of their training
14:11data.
14:11And it's very dangerous when you're outputting text like that because when you have plausible-sounding text or very fluent
14:19text, there are so many different kinds of issues that can occur besides you believing that there is a mind
14:25behind a machine.
14:26I gave, in that paper, we gave an example of this Palestinian man writing good morning, which was translated to
14:32attack them.
14:33And because of that grammatical correctness, and there were no cues that the translation could be wrong, and people believed
14:43the translation.
14:44So the other issue of believing there is a mind behind the machine is what we call automation bias.
14:49You over-trust automated systems, and if you believe that this thing is an all-knowing machine, then you're going
14:56to over-trust the errors that you get, right?
14:59So we're seeing this with medical scribes, where I was just reading an article, another article, where medical scribes that
15:08were summarized said this woman was microdosing on mushrooms, since a woman has never heard, never discussed mushrooms, never done
15:17mushrooms.
15:17She saw it on the notes.
15:19So none of the doctors, nobody, nobody checked, because again, if you believe, this is why I believe the superintelligence
15:26existential risk discussion is not just distracting, but dangerous.
15:30Because with automation bias, with over-trusting these machines already, if you believe that they are nearly superintelligent, instead of
15:39error-prone, large language models parroting things, then you're going to be less likely to check.
15:45You're going to be less likely to put checks and balances and regulation, and you're seeing things like misdiagnosis based
15:52on medical errors and things like that, which are serious.
15:55It's funny.
15:56I must be too much of an elder millennial because you say that there's too much automation trust, and I'm
16:01like, I'm so distrustful.
16:03I've called the bank, and it's a robot.
16:05I'm like, nope, nope, I have to.
16:06I don't want to talk to it.
16:07Yeah, exactly.
16:08So one of Anthropics co-founders, Jack Clark, recently posted something on X.
16:13I know, you know what I'm going to say.
16:16Yeah, I do.
16:17He basically, he put stochastic parrot in quotes and said he, it was a mimically fit cognitive virus that spread
16:24from 2021, when your paper was out, to 2025.
16:28It temporarily blinded many gifted people to the nature of AI progress, burned up crucial years of research.
16:34He later says, the use of this frame causes people to materially underestimate what AI systems can and can't do.
16:42When you saw Jack's post on X, what was your initial response to that?
16:47I was not surprised, by the way, let me just tell you, because the effective altruists have been, I was
16:54telling people that this is a talking point that they're telling lawmakers now,
16:57because every time I said something, they were like, oh, you should not, you should not take seriously someone who
17:03still takes the stochastic parrots thing seriously in 2026.
17:07So this has been a talking point of theirs for a while.
17:09Right.
17:10It's that the idea is that the research is outdated, right?
17:13Yeah.
17:13And it's not.
17:15And that's, that's what I want to say.
17:16It's so ludicrous that we're even saying this, because these are definitions of what large language models are.
17:22What large language models are has not changed, will never change, like large language models are large language models.
17:28Now, you might have large language models in a separate system that's trained in a, you know, which, you know,
17:35they have now reinforcement learning agents.
17:37But our paper was about large language models, and that's never changed.
17:40And these chatbots still have large language models as a basis.
17:45And we are seeing, it's so ridiculous that Jack Clark is talking about overestimating systems, because what I'm seeing is
17:53the examples that I just told you.
17:55It's over trusting these systems, not having checks and balances, and ending up misdiagnosing someone's breast cancer to the wrong
18:04side.
18:04So there's a direct line between LLMs being stochastic parrots and misdiagnosing, giving a medical misdiagnosis using an AI tool,
18:15because why?
18:16How does that actually, how does one lead to the other?
18:20Because large language models are stochastic parrots.
18:24The stochastic parrots, they don't understand what's inside the text.
18:30So you cannot expect them to be factual.
18:32Even if you see something like the AI overview, I had another example where I was calling an oncologist friend
18:39of mine to ask about a specific medication and whether it was appropriate for a specific use.
18:45Because I read the academic paper saying that it was not, and I wanted confirmation.
18:50And my oncologist friend was like, oh, look at the Google overview.
18:53It says that it's appropriate.
18:54But I read back, but it turned out not being.
18:58Why?
18:58I counter that all the time with Google AI overviews.
19:00Because they are stochastic parrots trained to give you the most likely sequences of text based on their training data.
19:08So this is not grounded in a world models.
19:12Emily M. Bender has been trying to say this in so many languages for a long time.
19:16Also one of the co-authors on the paper, correct?
19:19Co-first author with me.
19:20To say that the research is outdated is ludicrous.
19:24Because right now, as we are hyping up these super intelligence and all that, there are news stories that are
19:30going unnoticed, which is about the complete opposite scenario that is actually happening in the real world.
19:35My understanding is that some of the biggest critiques people have had about focusing so much on research from that
19:42era may not be incorporating the thinking or the reasoning or even the recursive intelligence.
19:50There is no recursive intelligence.
19:51That is happening.
19:52Is there none?
19:53There is no thinking.
19:53How is there none?
19:54Again, this is something, by the way, that at the Berkeley AI Conference earlier this summer, I heard someone from
19:59Google talking about recursive intelligence.
20:01Of course they are.
20:03The New York Times just said a big story about how the scientists and researchers right now, that's what they're
20:08looking towards, recursive intelligence, the AI learning from other AI.
20:12Is that a reality?
20:14It sounds like you're saying that's not a reality.
20:16No, because there's one problem with AI researchers, which is aspirational naming and aspirational stuff.
20:22So machine learning, that's aspirational naming.
20:25But machine learning is real.
20:26But the naming is aspirational.
20:29The machine not necessarily learning.
20:30So this happens all the time, right?
20:32So just because there's curriculum learning in AI, it's a field.
20:36But let me give you a paper from my former manager, Sammy Banjo.
20:40And I asked him, aren't you tired of your whole life being like debunking the whole reasoning thing, every single
20:46paper you write?
20:47He quit after I got fired from Google, and he's now head of machine learning research at Apple.
20:52And if you look at almost every single paper that they have, it's showing how if you change the benchmarks
20:59on the reasoning slightly, the whole thing breaks down.
21:02It's not reasoning.
21:04Just because you're looking at chain of aspirational naming.
21:07Let me give you another example.
21:09Just because your models are large, the stochastic patterns that you trained to print out certain tokens, you call them
21:19chain of thought reasoning.
21:21You didn't know that they were thinking.
21:23You don't know it's a chain.
21:24You just know that these are tokens that are being printed out.
21:27But you call them chain of thought reasoning.
21:29Now you're saying that they're reasoning already.
21:30One of the biggest crises that we have right now is actually sound scientific research.
21:36So if you look at my work, if I ever have access to the data, the code, the training data,
21:44and the evaluation data, which none of these companies give you those things.
21:48You don't even know whether they ingested that benchmark during training or not.
21:52If you ingest a benchmark during training, it's like studying to the test.
21:56It's like me coming to an exam, knowing what the answers to those 10 questions are already, studying that, and
22:03writing it down, right?
22:04So every time I have had access to these things, I have shown how their claims are not correct.
22:11But then with Sammy Benjo and his team's paper, what they showed is, and this is from 2025, what they've
22:17shown is that you just change the benchmarks a little bit, tweak it a little bit, and it all breaks
22:22down, showing that you were just sort of relying on the patterns.
22:26Now, some people, when I give them this example, they're like, well, we're talking about 2026 models.
22:31Guess what?
22:32Real evaluation in science takes time.
22:34We should not be going from press releases to lawmakers parroting those press releases and journalists repeating those claims.
22:41If you want to do real research and evaluation, ask for the training data, the evaluation data, the methodology, so
22:49we can all reproduce it.
22:51And so you are ascribing the thinking, reasoning, recursive intelligence.
22:59It sounds like you still see that as something that is purely human.
23:02It's ascribed to us humans.
23:04It's the way our neural processes work, but the AI doesn't work that way yet.
23:08That's what you believe.
23:09No, and I don't know if it'll ever, like intelligence, that's aspirational naming.
23:13And sometimes I play just to take people back to these hype cycles.
23:16I tell them something, a claim that was made, and I say, is this 1954 or 2024?
23:21What would you say is the biggest part of your own thinking, your own research that has evolved since the
23:28early days, 2020s, of AI till now that has surprised you the most?
23:33I have to be actively making space for the kinds of models that I think should be built and building
23:40them and sometimes ignore the noise.
23:42That's the conclusion I'm getting to over time.
23:44That's your biggest learning, your biggest takeaway.
23:46Yeah.
23:47There is so much noise online right now.
23:49It's hard to get any deep work done if you're paying attention to whatever the AI guys say.
23:54And even if your research is all about debunking what they're saying, it gets tiresome.
23:59It's not fun to do that.
24:00It's more fun to think about the future you want to have, the technological you want to have, and work
24:06on that.
24:06Before I let you go, what would you say gives you the most hope right now for the future of
24:12AI?
24:13I want to tell you that we have a list called the AI resist list where we talk about people
24:20resisting in the ways that they should resist in many ways, which is media capture, narrative, data centers, funding, etc.
24:29And so we list the ways in which they're resisting.
24:32And also people are creating alternative tech futures that don't kill our environment, that don't kill, exploit labor or steal
24:40data and instead are actually actively helping their communities.
24:45And these ideologies are spreading, right?
24:47You see one small organization somewhere doing something, you get inspired by them, you do something different.
24:52So I have hope that there are a lot of people tired of what they're seeing and they're actually in
24:59small circles doing something different.
25:01I believe in human agency and collective power to imagine a better future and stop bad things from happening and
25:11ban things if they are bad, right?
25:13So my belief in human agency, I think, is what gives me hope.
25:17Thank you so much, Timmy.
25:18I really appreciate you joining me today for the big interview and for sharing your insights.
25:22And congratulations on your upcoming book.
25:25I look forward to reading it when it comes out early next year.
25:28And I'm sure we'll chat again soon.
25:30Thank you so much.