Who decides when AI is too dangerous?

Support for the show comes from ServiceNow. AI is moving fast across the enterprise, but without visibility, it's just chaos. Different tools, different models, different teams using AI in completely different ways. ServiceNow turns that chaos into control. With the AI control tower, you see all your AI across the business in one place. What it's doing, what it's done, and what it's about to do. So you stay in control. To put AI to work for people, visit servicenow.com. Support for Decoder comes from Uber Advertising. Whether it's ordering dinner, booking a ride, or shopping for groceries, we live in a world where we make decisions the moment a need arises. But these moments aren't just when people choose Uber. They're when people choose brands. Uber Advertising helps brands understand the context behind those decisions. The moment, the need, and the mindset across mobility, delivery, and commerce. So brands can show up with greater relevance when people are ready to act. Uber Advertising, where life's movement becomes your brand's momentum. Learn more at uber.com slash advertising. Support for this show comes from BetterHelp. Summer can feel like a sprint. Kids home, trips to plan, routines flipped upside down. It's easy to slip into survival mode just trying to get through it. Then suddenly it's over, and you're wishing you enjoyed the days just a little bit more. Therapy can help you slow down and actually be present for the moments that matter. With BetterHelp, you can connect with a licensed therapist from anywhere on your schedule. Don't just survive this summer, thrive. Visit BetterHelp.com slash VOXpods. Hello and welcome to Decoder. I'm Nilay Patel, editor-in-chief of The Verge, and Decoder is my show about big ideas and other problems. My guest today is Hayden Field, the senior AI reporter here at The Verge. Often when Hayden shows up on Decoder, it's because something has gone wrong in the world of AI. And last weekend, that something was a pretty intense mix of Anthropic, the Trump administration, and Anthropic's new model, Claude Fable 5. On Friday, not even a week after Anthropic released Fable 5 to the public, the United States government said it was imposing export controls on that new model, as well as the underlying mythos model that Fable was based on. Those controls restricted foreign nationals, even those working for Anthropic in the United States, from accessing these models. Anthropic then took Fable and Mythos offline for everyone because the company said it was worried it would not be able to restrict access and reasonably comply with that order otherwise. As you might imagine, this is all a giant mess. Hayden actually just published a fantastic play-by-play on the Verge about how this all went down last Friday and the scramble through the weekend from both sides to figure out exactly what happened and how it might all get resolved. So I wanted her to come on the show and walk me through the timeline and what it all means. The situation is ongoing. As of Tuesday when we're recording this episode, Fable is still offline. In fact, if you boot up Claude, it tells you right above the chatbox window that Fable 5 is currently unavailable. Yet, as you'll hear Hayden explain, whether Fable comes back online this week or not, the ripple effects of the government's feud with Anthropic have far-reaching consequences for the tech industry and the United States AI regulatory regime. There's also a big irony here, and you'll hear Hayden and I talk about it quite a bit. Anthropic has spent years arguing that AI might soon be powerful enough to be dangerous and that the government needed to get serious about regulating AI sooner rather than later. Well, now we're here, and Anthropic doesn't love the way it's playing out. And now everyone, but maybe especially the Chinese government, is watching to see whether the United States AI regulatory approach takes the shape of a serious safety framework or whether it's just another weapon for the White House to use against companies and people that don't bend the knee to the Trump administration. Like I said, it's a real mess. Okay, Verge senior AI reporter Hayden Field on the Claude Fable ban and the new AI regulatory landscape. Here we go. Hayden Field, you're the senior AI reporter here at The Verge. Welcome back to Decoder. Thanks. Great to be here. It's always chaos when you're here, Hayden. That's what I've come to discover. It really is. Also, my rule is still in effect in that it's always a Friday. Stuff always hits the fan on a Friday, and this was no different. Particularly when it comes to the government regulating AI, there's a real what if it all went crazy on Friday, so we all had to think about it over the weekend. And then I think this has been basically true throughout the Trump administration. The scramble to put it all back together on Monday is immediate. Exactly. It's crazy. And the fact that all these talks were happening all over the weekend, and then Monday, still no resolution. I'm surprised that things aren't resolved yet, but yeah, lots of drama. Well, this podcast is going to come out on a Thursday. We're recording on a Tuesday. We'll see what happens over the next two days. But I think the big picture here of how the United States government should regulate AI and what that means for American AI companies on the world stage is open. I don't think it's going to be resolved with whatever happens with Fable. So let's try to understand the whole story and then pull those themes out of it, because they're just going to keep coming back around again and again and again. I think with every new model released now. So let's start at the start. What is Claude Mythos? What is Fable? How do they relate to each other? Great question. And I asked them the same thing when they came out with this like a week ago, or by the time we publish this, it'll be like a week and a half ago, because yeah, AI companies are really bad at naming things, and it's always very confusing. But the situation is Mythos is the underlying model that is powering both Mythos 5 and Fable 5. Fable is a new thing. They haven't had Fable 1, 2, 3, or 4. They just immediately started with Fable 5. Very confusing. But Fable is the watered down or safeguarded version of Mythos 5. So when they came out with Mythos preview in April, Anthropic kept saying, you know, they were really hyping it up. They were saying this is a, you know, potential like cyber weapon. It's got to be never released to the public, at least until safeguards catch up, et cetera, et cetera. We're only gonna release it to enterprise and governments, aka cyber defenders is what they called them. You know, places that need to patch up all their frameworks before, you know, bad actors come in and exploit those vulnerabilities. Now, fast forward to last week, and they released not only Mythos 5, which is the first like official Mythos. Before it was Mythos preview, so we skipped straight to five. Then they also released on the same day Fable 5, which was the very first like public version of Mythos class models, aka same underlying framework, but now tons of safeguards built on top to hopefully stop the public from accessing this super dangerous, by Anthropic's own admission, model that could, you know, mess up everything. So yeah, that's what they released, and they spent a long time hyping this up and talking about how dangerous it was. And then it kind of came back to haunt them later. So Fable is Mythos with the security guardrails. And when it first came out, there was a wave of controversy, right? AI researchers, security researchers found those guardrails were actually preventing them from doing research and seeing what Fable was capable of. Because if they tried to push the boundary, it would downgrade them to Anthropic's previous models. Yeah, it became kind of like a meme. Like people were making fun of how strict the guardrails were when it came out. So that's the thing that's confusing me, where there's a controversy because the guardrails are so strict, and then something happened that made the government think Fable was so dangerous that it needed to be pulled. What happened? I talked to some independent red teamers about this, and before the Amazon white paper was circulating, before this allegation of a jailbreak was public, I talked to some red teamers about how safe Fable 5 was, and they were all pretty impressed with how the guardrails held up. Like they were, I was expecting it to be, you know, a little bit easier to jailbreak. They were saying they had tried a lot of stuff and nothing was working. So that's not my experience when I usually talk to independent red teamers. They're like, yeah, I got it to break here. I got it to break there. In this case, they were saying it held up pretty well. Now, what happened this time is, according to a source familiar with the negotiations who was directly involved with the negotiations that I spoke with yesterday, this is kind of the timeline of last week. Mid last week, Anthropic was made aware of some research that Amazon researchers were doing, and that they had, you know, uncovered a potential jailbreak, something they were pretty worried about. They sent it over to Anthropic. Anthropic was kind of going back and forth with the Amazon researchers talking about it. And they were discussing like whether it was really a jailbreak or whether it was like not that big of a deal, essentially. Apparently, reports say that Amazon CEO Andy Jassy then got worried about it and called maybe Scott Bessent, some member of the Trump administration on Friday and talked about his concerns. Trump administration freaked out immediately. They immediately call Anthropic. They're like, they send them a message, I'm not saying it's not... Like, any jailbreak of this is obviously, you know, serious and worth considering, but I think that it's the type of thing, when you don't have direct knowledge of how these systems work, you're going to take a really black and white, immediate approach. Like, I think that these guys weren't experts. Andy Jassy hears about this from his own researchers. His own researchers, by the way, who had been going back and forth with Anthropic for a couple days on this. So they weren't, you know, calling the president about it. They're like, okay, let's discuss this and kind of, you know, figure out what we can do here. The researchers allegedly didn't seem to think it was, you know, the end of the world. They're just like, oh, we uncovered this, you know, let's discuss. Anthropic was working on it. And then Andy Jassy hears about it. He makes this call. Everything hits the fan, which makes sense because if you are not an AI expert, and even if you are an AI expert, maybe you're not an expert on cybersecurity and how exactly these systems work, of course. So you're going to freak out. I mean, it's kind of understandable, but what is really strange is what happened after that. So, you know, making this phone call, the administration becoming aware, okay. But what I think is really weird is the 90-minute ultimatum, them not giving them additional details when they said, hey, let's talk about it. Are you talking about the vulnerability we're already aware of or are you talking about something else and can you just tell us about it so then we can talk about it? They said, no, 90 minutes, do it or die. And then making it so that no foreign national can access the system ever. That is what's really strange. Just kind of the way all these cards fell is everyone in the tech industry thinks it's just really strange. And it's kind of indicative of, like you mentioned, the way that the Trump administration has been operating with tech. A lot of times it seems to be President Trump has one conversation and he's kind of swayed from that most recent conversation. So, you know, he's been very anti-regulation, very pro-American AI advancing, exporting American AI, which is something that the Commerce Department is requesting. It has an RFI out on right now. They're all about exporting American AI and like, you know, advancing and beating China. But then they do this thing that kind of kneecaps everyone with no notice and could very well impact not just Anthropic, but also OpenAI, Google, every competitor in the space that also has what they call a semi-equivalent Mythos product. So, yeah, there's just, it's really, it seems really disorganized and it's very interesting that, you know, this often ends up in a scramble with all these CEOs kind of trying to race to kowtow to the Trump administration without really being grounded a lot of times in science or evidence that really, like, matches what's going on. I feel like the level of freakout is not usually matching, you know, the evidence there, but. Amazon is an investor in Anthropic. They're a major partner. Why did Andy Jassy kneecap his own investment in this way? Well, Amazon typically, like, does a lot of red teaming, just normally. You know, it's kind of just part of their process. So it makes sense that this would have been tested and then unearthed. But what's interesting is why Andy Jassy called the Trump administration about it. I'm not sure. Maybe it was a typical call that was already happening and he just happened to mention something offhand. Maybe he was calling them and saying, yeah, good thing you did the executive order because XYZ, don't know. But what is interesting, and I'm glad you asked this because a lot of the sources I spoke with said that the stuff in the alleged jailbreak and in the Amazon white paper that's been circulating, all of it could be achieved by OpenAI's GPT-5.5 and some other models. So that's the allegation that this is not, these skills are not unique to Fable 5. The skills in question as part of the jailbreak. Mythos is obviously more powerful than Fable because it doesn't have safeguards. But when it comes to Fable and the problems that were, you know, brought to light by Amazon, it seems like those problems are not unique to Fable and that, you know, GPT-5.5 can also achieve those. So it's like, this is something that a lot of people in the industry are asking is why Anthropic? So a real element of this is that it's always Anthropic. Anthropic has the most contentious relationship with the Trump administration. They were designated a supply chain risk by the Department of War, but the posture here is different than the usual Anthropic drama. Historically, Anthropic has said, we don't want you to use our tools for X, Y, and Z. And the Trump administration has said, no, we get to do whatever we want with the AI tools. That's why they were designated a supply chain risk by the Department of War. They said, we have some red lines about how we want to be used in the military context. And Pete Exed said, nope, I'm going to do whatever I want. You have to allow all lawful purposes. That's the line. And because you won't, you're a supply chain risk. There's a lawsuit that's being argued about. Here, it's the opposite. The Trump administration is worried that the model is too powerful and could be used for too many things. And Anthropic seems to think that that belief is overblown, that the administration is overplaying a thing that they can fix. They just need the time to do it and, you know, some rigor and conversation about how effective their fix is. How do you put that together? That on the one hand, the Trump administration is anti-AI regulatory efforts. On the one hand, they say, we get to do whatever we want with the models, make them as powerful as you can. And then here, the model is too powerful and has to be pulled before any foreign national can even look at it. The administration is trusting these companies to kind of tell it what it should be doing. We've seen that a bunch with, you know, any regulation that has happened or like it has become voluntary. You know, lots of drafts have been adjusted after the Trump administration talks to AI companies. They already don't like Anthropic. You know, there's already a big trust breakdown between the two from the supply chain risk thing, from the lawsuit, from drawing red lines. And so I can't imagine that this exact situation would have played out if this had been a competitor. I mean, OpenAI has a Mythos semi-equivalent. Again, like, you know, no company is really saying theirs is as good, but they say it's, you know, pretty good. Microsoft, Google. I can't, I think that the freakout would have still happened, but I cannot imagine the rest of it would have played out the way it was. I think they would have given them more time. I think they would have had more of a conversation. And I don't think they would have made this foreign nationals rule that has like, you know, sent shockwaves through the whole industry and the world. One of the sources I spoke with said he already knew that backup contracts were being signed with non-US companies right now. Open weight models were being deployed on alternative hardware arrangements because now companies everywhere are saying, oh wow, we got to make political risk part of our business plan now. We don't really know what's going to happen with American AI. Let's, you know, make some backup plans here, which is not good for the industry. And so I think it's just interesting that the Trump administration is so afraid of China overtaking us. Yet, you know, they're kind of willing to kneecap one of their champions as it were, in the industry. But I also think it's interesting that the reasoning from the Trump administration that's been coming out day by day seems to differ. So every day they seem to have kind of a new excuse or part of the conversation that's coming to light. So first they say, oh, well, what if China had access? A source I spoke with said that's a rumor from a few weeks ago, from way before Fable 5 or Mythos 5 came out. And that it was referencing like a Chinese telecom company that the administration had previously approved to have access to Mythos preview. And that when the administration said, oh, actually, you know what? We changed our mind. Let's take them off. Anthropic immediately took them off and revoked that access. So I think that's an old rumor. Then we're also seeing the executive order thing where, you know, rumors have come to light that, oh, Anthropic wasn't complying with the Trump administration's executive order. Well, that executive order, it hasn't gone into effect yet. They only signed it on June 2nd. It's like, you know, two weeks ago. It takes 60 days for them to figure out which models are going to be covered under it. There was a voluntary, you know, partnership of like a 30-day pre-release review. It's voluntary. So I just think, you know, I'm not, it's funny because I'm not like, oh, Anthropic is totally in the right here at all. But it's just interesting because in this situation, there's just, there's so much like misdirection and scrambling and just weirdness that it's rare that you see me feeling kind of bad for an AI company. It really is. Like, I am not like, oh yeah, like they're doing great. All AI companies have their own issues, but in this one case, I'm like, wow, like, I really don't know if this I see it as, and they're like, oh, okay, let's roll it back a little bit. So who knows, honestly, but, you know, the reason I became an AI reporter is because I felt like there was no real regulation in the space, and this was even before ChatGPT, and, you know, journalism is a way to kind of shine a light on what's going on in the industry. And I think, you know, still six years later, we're not seeing a lot of regulation, and any that has happened got rolled back. So, you know, maybe we'll go back in that direction. I do see us going somewhere there, but maybe not even as much as that first rung. It almost feels like the industry needs to push for a more considered regulatory approach, right? That they need to say, this is chaos. This can't be how we operate. Every new model can't be forced to contend with panic phone calls on the weekend. What kind of system would you build, right? If you wanted a more rigorous review process here, is it they just submit their models to the government and the government says what's dangerous and not? Is it more industry-based? What are the solutions being proposed? There haven't been a lot of really robust solutions proposed recently because of this current administration. They've had a really light touch approach to regulation, so, you know, no one's really surfacing those. Now that that might be changing, I think we'll see more robust frameworks proposed. And I just hope that they're not voluntary. AI companies already do a lot of voluntary stuff. If you're going to regulate, do it, you know? There's not, I don't think you need to, there's a lot of different opinions on how light touch or heavy touch regulation should be, but I do think that voluntary frameworks are pretty watered down, you know what I mean? It's like, we already have those, so what's next, you know? There's a big, I think it's important to remember how, like, the general public feels because this is who AI is really affecting at the end of the day. And yeah, the rise in AI populism and the fact that people are upset, there's data center protests going on everywhere. People are even in extreme cases, like, attacking AI CEOs' homes or attempting to. There is a lot of disquiet here and people feel like they're getting replaced. They feel like their jobs are getting replaced. You know, I was listening to a Seattle city council hearing recently and a lot of AI engineers were testifying saying that they got laid off because of AI and they were engineers. So I think both AI leaders and the government often kind of are pretty distanced from what the general public is feeling about this technology and it's important to note. So I think, you know, we'll see how that impacts, if at all, regulation that comes out. I don't have high hopes for a lot of regulation under this administration because they can be really sway-y, you know? I think, like, you know, they have dinner one time and then things might shift a little bit. And to be fair, like, many politicians are that way, but, you know, we'll see. Maybe after this executive order, you know, the 60 days are up, maybe we see some changes that surprise me. I don't know. One of the weirder aspects of this is that Anthropic itself is the company that most pushes for regulation, that most sounds the alarm, is the most doomerish of all the companies. Jack Clark, one of the co-founders of Anthropic, recently put out research and his conclusion was that, quote, we believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology. Anthropic is way on the front line of saying, oh, we might kill everyone. Boy, we should stop this. But they're also on the front line of model capability, right? By all accounts, Mythos is the most powerful model that exists today. Is Anthropic just getting itself in trouble? Is this just marketing strategy for them that's backfiring? I think it's a combination. I think that the marketing speak is definitely backfiring. You know, if you say something's that dangerous and then a researcher uncovers something that maybe wasn't quite meant to be possible. Yeah, people are going to freak out. And so that's why I'm kind of like, you know, I can see the first thing happening to anyone. I can't see the rest of it the way that the cards fell. But yeah, I think it's partly there. You know, it came back to haunt them. You know, it was a little bit on them in terms of the hype. But I also think that they are trying to be honest in some ways about the risk that this technology can pose. And, you know, it would be really bad if a, you know, system that can flag high level vulnerabilities in our global infrastructure that we use, like your bank system or, you know, a government website, could just be used by bad actors if it fell into the wrong hands. Yeah, that's pretty bad. And so, you know, I think it comes down to communication. Like, yep, it's important to talk about the risks and not downplay them. But also, you know, if you do that and then you make a mistake or something unexpected happens, you got to deal with the consequences. So, yeah, there's a lot to unpack there for sure. And I also think that, you know, a lot of companies do downplay risks of different systems. Like, you know, we've seen a bunch of AI CEOs, including Anthropic, I believe, talk about how, you know, jobs aren't all going to be replaced. It's okay. You know, there's, I always take all this stuff with a huge grain of salt because at the end of the day, whatever AI company you're talking about, whether it's Anthropic or not, they all have the incentives to turn a profit and make money at all costs. And especially if they become a public company, which they are about to. So people like to paint Anthropic one way and other competitors another, but they're all, they're all pursuing the same goal. They just are doing it a little bit differently. Is there a version of this where we just treat AI like military technology from the jump and the public. Gets watered down version of the military technology. Cause that seems to be the risk profile that the Trump administration is operating within, right? This is fundamentally offensive technology. We can deploy it against our enemies. We can use it to attack the banks. Everyone is worried about the security implications here. Is there a world in which we just start there and then say to deploy to the public, to consumers, you got to cut this way down. We're not going to sell F-22s to the public, but we can sell Ford Raptors to the public. Like there's some version of this, right? Where it's like the technology gets cut way down for the public with the, the bleeding edges just, just fundamentally for the military. Yeah. I mean, that's like what we've seen a lot for decades. And I think it's starting to happen now with the mythos stuff. You know, fable is the watered down version and, you know, enterprise and governments have access to the more powerful version. There's a lot of different opinions on that. Like OpenAI, one of its biggest marketing pushes right now is, oh, Anthropic wants, you know, the most powerful AI to only be in the hands of the elites and we want to bring it to the public. So there's a bunch of different marketing pushes right now. Everyone's just kind of using everything to their advantage or to paint their competitors badly. Anthropic included. So it's like, you know, I think the race is just heating up so intensely that, you know, the AI companies at the forefront are doing anything they can to win at all costs. Some, some because they think that they are the best and most moral winner, which I think can always be a big issue as we've seen in like every, every sci-fi or, you know, fantasy movie. The for the greater good thing is, is typically used by, you know, the, the villain in the story. So I think it's, I don't know, it's, there's a big race happening right now and I totally agree with Jack Clark that, you know, it might be good to slow things down a bit. If everyone could agree, if everyone could agree, which is very, you know, a huge long shot and, you know, especially it'd be very difficult to get other countries to agree as well. But we are definitely building the technology much faster than we can think about the implications of it. And we have been for a long time. I think this is the first episode of Decoder where Dario Amadei was directly compared to Thanos. So that's great. To be fair, I was thinking about the guy in Harry Potter when I said that. What was it? Grindelwald. Grindelwald. Oh my gosh. All right, well, Dario, if you want to come on and address these allegations, you're more than welcome. I also got to say, it's not just Dario. All the AI CEOs are saying that. All of them. It's like they're all, yeah, it's, it's crazy. Where does this end? What are the next steps here? There's meetings happening this week. People are going to be hearing this on Thursday. There's a chance Anthropic is just back on the market. There's a chance it's not. But after this, it seems like everybody wants a process. Where do you think this ends? Honestly, it's very difficult to predict how this administration is going to act. As we saw with the supply chain risk thing, even though a lot of