After The Builders: On Dario & Daniela Amodei and Mustafa Suleyman

This is an iHeart podcast. Guaranteed Human. These days, there are more people than ever telling you how to manage your health. Your social media feed is packed with wellness trends telling you to cold plunge, protein max, and so much more. But what's actually real? What's exaggerated? What's just wrong? Health vs. Hype from the American Medical Association breaks it all down. Emmy-nominated host Trace Dominguez talks to health influencers, real people, and most importantly, doctors to separate fact from fiction. Because when it comes to your health, who you listen to matters. Listen to Health vs. Hype wherever you get your podcasts. Who is going to win control of Congress this November, and what is it going to mean for the final two years of Donald Trump's disruptive presidency? I'm Susan Glasser. I'm Jane Mayer. And I'm Evan Osnos. We host the Washington Roundtable on The New Yorker’s political scene podcast. We're following what key races could tip the balance of power in Washington this fall. We're also going to go deep on the influence of dark money and tech billionaires. So join us in the run-up to the midterms in 2026. Subscribe to The Political Scene wherever you get your podcasts. Small businesses are the pulse of every community. They bring people together, create opportunities, and drive growth. Chase for Business helps business owners like you with personalized guidance and convenient digital tools all in one place. With that guidance and your determination, you can take your business further and help build a brighter future for your community. Learn more at chase.com/business. Chase for Business. Make more of what's yours. The Chase Mobile app is available for select mobile devices. Message and data rates may apply. JPMorgan Chase Bank, N.A., Member FDIC. Copyright 2026, JPMorgan Chase & Company. Welcome to TechStuff, Osvoloshin here. It's another special edition of the podcast today focused on Kaleidoscope's new series, The Builders, with Walter Isaacson. This time, we're going to discuss the third and fourth episodes, where Walter interviews Daniela and Dario Amodei, the two co-founders of Anthropic, and Mustafa Suleyman, the CEO of Microsoft AI. Dario and Daniela left OpenAI in 2020 after disagreements over AI safety with Sam Altman. Mustafa Suleyman was pushed out of DeepMind, which he co-founded in 2019, after two years of grinding negotiations with Google about AI safety and following complaints about his management style. Today, they represent different, perhaps even opposite, sides of the debate on safety, with the Anthropic co-founders advocating for encoding ethics in AI and Suleyman arguing for the primacy of human control. It's a fascinating pairing, and today I'm joined by my Kaleidoscope co-founder, Mangesh Hattikudur, as well as two journalists who know more about these figures and the debate about the future of AI than anyone I know. We're delighted to be joined by Sebastian Mallaby, currently a senior fellow at the Council on Foreign Relations and the author of many books, most recently The Infinity Machine: Demis Hassabis, DeepMind, and the Quest for Superintelligence, and his forthcoming book, recently announced and untitled, is about Dario Amodei. Great to be with you. And Natasha Tiku, Silicon Valley culture and AI reporter for The Washington Post and regular TechStuff contributor. Welcome. Thanks for having me. Before we get into the builders, we'd love to hear a bit from both of you about what it's like to report on these people. Natasha, maybe starting with you, I'm curious about the kind of arc of reporting on these tech leaders, particularly in the age of the self-published viral essay. Yeah, it requires a lot of time spent reading thousands and thousands of words. I put actually recently all of Dario's essays in a Google Doc because I'm just highlighting, and there's so many of them, and I had to, you know, even incorporate different tabs for his recent long-form tweet responses. So yeah, it's kind of like, you know, Talmudic studies. Sometimes it seems like that. Do they write them all themselves? I've heard that it is a norm within Anthropic that it's actually very bad form if it seems like you have used Claude to do your own writing, which I think is like a beautiful irony. Yeah, I do believe he's written them by himself. But Jack Clark is also prolific, the other Anthropic co-founder, right? I mean, to keep up with all the things the Anthropic co-founders write, you'd have to be, you know, quite the reader. Sebastian, what about you? I mean, how do you get in the heads of these founders? I mean, Demis and now Dario. Well, with Demis, it was a question of sitting with him in a pub in North London for hours and hours and hours, more than 30 in total, and just getting him to go through his life story and reflect on different bits of it. He's a great narrator, so that was actually super fun. And he tells, you know, he self-dramatizes very well, and you have to maybe slightly filter a few things where you think it may be not quite true. But essentially, he is relatively trustworthy and extremely fluent. And the thing about him is he doesn't write that many essays. He did one this year in July on safety. But the way he's told his own story is through lectures and podcasts. There hadn't been a kind of long-form written book which had access to him, which kind of covered not only his childhood and all that, but the linkages between what he studied at Cambridge as an undergraduate, Claude Shannon and so forth, and then how he developed this crazy conviction that you could build powerful AI and founded a company to do that in 2010, two years before AI recognizes the photograph of a cat. I mean, the conviction was wild. So, Mangesh, obviously you and I are quite proud of our series, The Builders, and there's a little bit of kind of fun, wild ride of how it all came about. Why don't you tell us about the Amodei's interview first? Yeah, so we're all obviously fascinated by the Amodeis, and we're trying to book them. We had reached out both asking for Dario and Daniela. We thought we'd be able to get them separately, and initially they kept saying, Well, Dario is available this time, and we were pushing more for Daniela's voice as well. And then they said, you know, Would you like to do them together? And we thought, perfect. You know, there's only one interview that they've done together before, and that's with Oprah. And so, you know— Very good company. I know. And so, you know, we managed to secure them. We were so excited. And Dario got to the interview first, and he was there about 10 minutes early, and Walter was there as well. And so, you know, Walter is Southern and charming, and he's trying to cajole him into chatting a little bit. And Dario kind of waves at him and then just looks down, and he does not want to do any small talk. And Walter just kept saying, like, you know, a little bit more, a little bit more. And Dario was like, you know, I'd really prefer to wait till Daniela's here. And so it was just silence until Daniela gets there. And then immediately he turns the charm on. You know, like, he's so excited to talk, and you see, you know, how close they are. And you realize that, like, he's obviously a CEO who's a visionary, but doesn't manage anyone. And she really both protects and translates his brilliance so effectively to a larger world. And it's kind of remarkable to watch them interact. And I really feel lucky that we managed to get them together. Natasha, Sebastian, maybe Sebastian, you first. You were both smiling at that account of the pre-game. Yeah, I guess the compartmentalizing is what people have to do when they're super busy. And, you know, if you've got that extra seven minutes before the real thing starts, you're going to use it to think another big thought, because the value of each incremental big thought is measured in the billions of dollars. You know, and the safety of humanity might depend on it. So this is serious stuff. So look, I mean, I recognize that tendency of being super efficient with one time. And, you know, Demis, just to go back to him as a kind of baseline comparator, has this routine where he has two working days. He works a day in the office, he comes back, has dinner with the kids, goes up to his home office, works from 10 a.m. until 4 a.m. the next day. So he gets the sort of reading, thinking, cogitating time. And that's another example of being just aggressively efficient with how much you can eke out from each 24-hour unit. There's a scene in your book where Demis and Mustafa have dinner every Sunday night in the pub, and they wait until last orders to order so they can eke out as much conversational time as possible. But Natasha, what about you? I mean, I watched the video, so it was just reminding me of times when Dario knew that he would get in trouble, so he just, like, handed it over to Daniela. And I was watching Daniela's hands when Dario was— I was getting a little bit too colorful, so it was just, it was a rich text, the video itself. Love a video podcast. Before we get into the actual interview, though, can you give us some context, Natasha, starting with you? Like, what do we need to know about the Anthropic IPO this week? I think you need to know that it is one of the fastest-growing companies in history, period. People are anticipating a valuation of potentially $2 trillion. There's also been reports of $1.5 trillion, but you have to keep in mind this is a company that is less than six years old. And the text of the SEC filing is exactly what you would expect from the Amadeus, right? Like, they talk about bringing forth an age of miracles. If you buy a share in their company, don't think of it as a transaction. Think of it as being, you know, part of this momentous period in time. They talk about how each of their employees is worth $6.6 million. Like, that's how talent-dense this organization is. And they also go through some of the existential risks, which they touch on in the interview as well. Sebastian, I mean, I know it's just been announced and published as weekly, and it's sort of probably a quiet period, so to speak, on the book, but what's your biggest question about Dario Amodei going to be? Yeah, I mean, it is super early. I've only just begun interviewing, but I would say the questions going in are, you know, in a way, how did this extraordinary thing happen? You know, these guys left OpenAI, the seven founders of Anthropic, they left in 2020. They start the company in 2021. And at the time, you know, they're way behind. They don't have a big source of funding like— Google DeepMind did because of Google's, you know, enormous balance sheet. They didn't have the head start that OpenAI had. OpenAI then goes on to do the ChatGPT, goes viral in 2022, and Anthropic seems even more behind. They scrounge together some money miraculously from the Effective Altruist community, with Sam Bankman-Fried at the center of that, and then he goes to prison. I mean, it's like this setback after setback after setback. And yet, it turns out that by 2026, these guys are at the forefront scientifically. They are the biggest revenue, fastest revenue ramp, I think, in the history of capitalism. I would go maybe one notch further than Natasha did. I think it is the biggest, fastest ever. And then also in policy terms, you know, they stick their necks out, they take principled stands, they take on the Trump administration, and they make the weather on the kind of policy and safety front. So I think there's a kind of triple here, right? The policy stuff, the business thing, and the scientific advance. It's so crazy how the tech news cycle is nowadays, because Microsoft announced that they've offered buyouts to 60,000 employees, more than 25% of their entire workforce. It's probably like the fifth biggest tech story of the week. But I mention it just because obviously the other person we're going to be talking about in this episode is Mustafa Suleyman of Microsoft AI. Have either of you followed these buyouts? I mean, the premise is unlocking more capital for compute, essentially, I think. Yeah, I mean, I haven't followed that specifically, but it's part of a big thing where, you know, when you're trying to onboard a lot of artificial intelligence, and I use that word deliberately, when you onboard a human, you have to kind of figure out how to use that person, train them and so forth. It's kind of the same with AI. You have to kind of figure out how to integrate AI into your business processes to make it productive. And that is an enormous suck on management time and management focus. And these guys just don't want to have too many humans knocking around when they're focused on this other thing. And it's already the case that, you know, in Anthropic, for example, large amounts of the tasks that might traditionally have been done by humans, for example, thinking about, you know, working on the IPO, I'm told that there are large numbers of AI agents that are doing a lot of that work. And so— They can write the essays, but they can write the IPO prospectus. Well, they probably could write the essays. It's just that existentially, we don't want them to do that, because writing an essay and expressing your view, if you're the leader of the lab, I mean, it's kind of like, that has to be you. And Natasha, what's the vibe in Silicon Valley? I mean, at the same time as any number of Anthropic employees who are now worth exactly $6.6 million each, versus all the Microsoft employees being bought out, like, what's the mood? The mood is, you know, can I get a job at Anthropic? Otherwise, you'll be part of the permanent underclass. I mean, there's just so much money sloshing around. You see people expressing, you know, Oh, here's me in a picture of Ilya six years ago, Ilya Sutskever, you know, like, did I make the wrong decision? Why are some of the people who have the exact same background as me billionaires? You know, Matt Levine, his Bloomberg write-up about the S-1 was great. It was basically like, Claude is going to subsume all human endeavor, so why not buy a share of Claude? That was kind of the IPO pitch. And that is the mood right now, I would say. You know, kind of mouth agape, envy. I mean, the feed of capitalism is astounding. I keep thinking actually back to the Peter Thiel, Walter Isaacson interview where he said, Can you think of a cult that has ever done this? You know, it's phenomenal to think of somebody with, like, deep ideological beliefs that is actually at the pinnacle of capitalism. Yeah, truly. Well, I mean, let's get into that. I mentioned at the top that kind of debates over safety, in some sense, informed both Mustafa's departure from Google — or more from both of you on that — but also, of course, Dario and Daniela from OpenAI. So this was one of Walter's more controversial lines of questioning in the interview, probably the body language told its own story, but let's play it now. Dario, you say your departure from OpenAI was partly due to a breakdown of trust — an interesting word, you know, with Sam Altman. Explain how you felt there was a breakdown of trust and honesty there. What I will say is, look, leaving OpenAI and founding Anthropic, I mean, it ultimately comes down to the principles and how we're going to build the technology. That is what matters at the end of the day. But I will say that the moment when I realized that we needed to found our own company, that OpenAI was not the right place for us. Look, these safety issues are difficult. You can have lots of disagreements on them. I disagree with the co-founders and with lots of people at Anthropic, but the reason we've been able to resolve these disagreements is because there's always trust that everyone is trying to get to the right answer and everyone is being honest. My point in saying that there's a breakdown of trust is simply to make the abstract point that if you don't believe that about a place where you're working, if you don't believe that the disagreements are being held in good faith and that you're getting the real story from people, how can you possibly hope to resolve them? That's why we left. That's why we founded our own thing. And I think the last five and a half years have... How our philosophy differs from that of the other companies in the space. I think if you continue to work for someone who has a different vision than you, it becomes very unclear what you actually stand for. Wait a minute, though. I'm pushed back a bit here because you talk about differences in values and stuff, but that's fundamentally different than a difference in trust, that you feel that somebody says they have the same values, but they don't and you can't trust them. Honesty is one of our values. It's a very important value. It's hard to know what someone's values are if they don't have the value of honesty. And you felt that about Sam? I think the way, at least for me, I would put it is, I think the co-founders are an unusually high-integrity group of people. And all of us, I think, shared a very similar vision for how we wanted this technology to go and how we wanted to build it. But at a certain point, it just makes more sense to go do this as your own organization. How are you going to get someone else to implement or share your vision if they don't naturally do so? No matter how much they say they share it, if that's not reflected in what the company actually does, what choice do you have? Natasha, I want your reaction, particularly I loved your smile at Daniela's interjection. I think that was actually the part where he, like, moved his hands and said, like, Daniela, you take over. I'm assuming that's because they said, Dario, you're way too charged up to actually make comments about Sam in any interview. Please defer to Daniela. And then he couldn't help himself. He, like, went back into it. That whole section was so fascinating because, you know, obviously this animosity and disapproval, hatred in some case, I'm not sure, you know, I don't want to put words in anybody's mouth, but that animates so much of Dario, and you can see that throughout this whole interview, right? Like he comes back to it again and again, you know, wanting to make a company that is so different from the way that OpenAI operated. And a little bit of what he's alluding to, I think, in that clip is this emphasis on culture fit, right? Like, which Anthropic famously has this interview process where, you know, you can sail through the technical interview, you can do really well, and then they have this culture fit interview, which actually involves reading some of Dario's essays. I was talking to somebody who sailed through the technical part and then got cut off at the culture part, and they asked, like, Do you share the same views on AI safety? How much of an issue is AI safety to you? And you're, you know, supposed to espouse some of the same values that the co-founders have. And, I mean, certainly you can see that they do operate as one cohesive unit. That's why the company is so successful. But you can just tell that even five, six years later, Dario is so animated by the reason that he left OpenAI. You know, it's interesting. We had Elon Musk was the first episode of The Builders, and, you know, he was talking about how it would be good to get all the AI lab founders talking once a week and blah, blah. And Walter said, Even Sam? And, I mean, Elon was basically like, No, you can't really trust Sam. And it is funny how, you know, and there was obviously a kind of dust-up between Elon and the Amodeis as well, which now seems to be resolved. But Sam seems to be uniquely disliked by his fellow AI titans, Sebastian. Yeah, that's true. I remember being in Silicon Valley talking about my book in April, and I was being hosted by the general partner of a prominent VC firm who had known Sam, you know, for 20 years. And since I knew he was—they were friends—I said to him, Well, look, this New Yorker piece has just come out. We can summarize the New Yorker piece as follows: Sam Altman lies, and when he's finished lying, he lies some more. Is that unfair? I said. And this VC, who's been friends with him for 20 years, says, Nah, not really. I mean, this is what he is like, right? He absolutely tells people what they want to hear. And, you know, the sympathetic interpretation is this is what people do all the time in legislative affairs in Washington, D.C., right? You try to get a coalition for a bill in Congress by telling one member of Congress it's going to do this which you want, and then the other member it's going to do that which you want, and you try to get people together, and you're not—you know, maybe there's like a little bit of truth in both claims, but when you put them both together side by side, they can't both be quite true. But too bad. You put coalitions together, and you do this in business all the time. When you're raising money for some big real estate project or whatever it is, you've got to tell the local community one thing. You've got to tell the politicians another. You've got to tell your shareholders yet a third thing. The tenants get a fourth story. So, you know, people do this in the real world, but just Sam Altman does it to quite a pronounced degree. And in a repetitive game with very high stakes like AI, in the end you burn your boats with people. Obviously, I'm sure most of us have watched the Elizabeth Holmes documentary, but one of the things which comes across from that and from that New Yorker article you mentioned is that, like, the default assumption is most of these people are lying through their teeth at all times. Natasha, what—you know, and of course Jensen Huang has implied that the Amodei's kind of safety discourse is kind of like a bad-faith attempt at regulatory capture and marketing, et cetera, et cetera. But when you hear them speak, when you look at them, like, I don't know, I mean, this is the same thing I have with the Elizabeth Holmes documentary, but, like, it doesn't—I mean, it looks like they really believe what they're saying. Yes, I think that trying to go down the road of gauging whether or not they're sincere about these beliefs is futile. Like, they obviously very strongly believe it. But actually, I was talking to Daniel Kokotajlo, who's this early OpenAI employee and whistleblower of sorts. You know, he wrote that AI 2027, because I was talking to him about, like, the trust that employees have in Dario versus Sam, and he was reminding me that actually OpenAI employees felt this way about Sam for a number of years. You know, now they feel very disillusioned, maybe, but he was cautioning me to see it more as, like, DeepMind has been around for, like, that's a 20- to 15-year-old company. OpenAI is a 10-year-old company, and Anthropic is a five-year-old company, and they all end up on this spectrum of different, I guess, you know, making concessions to their views on safety and a difference between the trust that employees have in their leaders. But yeah, I mean, I've known Sam for a decade, so I definitely understand when people say that, you know, he can be kind of a wordsmith, let's say, like a slippery guy. And I think, I mean, Walter alludes to this too, right? Like Google being a company that was, you know, do no evil, and people don't feel that way anymore, right? And similarly, I remember being at South By when, like, a young Mark Zuckerberg came on stage, and, like, people were enamored, right, and really charmed in a very different way. So it's really kind of fascinating to see the sort of, like, cults of personalities that occur within these places and that people really do sincerely believe in the missions. I had to slog through Mark Zuckerberg's memos as well, and some of those were very long. So yes, I think it's important context, you know. Sounds like you would be able to ace your interview at Anthropic, Natasha. They would produce the text for your second interview, and you'd say, I've already read it. Sebastian, you mentioned the one company willing to stand up to the Trump administration. So let's play a clip now of Dario and Daniela talking about their concerns over the use of their technology in war. There's a second concern we have, which is what you might call oversight or accountability. I am a little worried. So, you know, again, suppose these things were fully reliable. So, you know, you tell them to shoot at one target and they shoot at that target and not a different target. Okay, so then you manufacture, you know, an army of a million robots or a million drones. It's a little unlike human soldiers, isn't it? With human soldiers, you know, they're supposed to follow this thing called the Constitution, and if they get orders that contravene the Constitution or that just seem morally abhorrent in some way, then they can just refuse to follow orders. Whereas the AI models, that isn't true. So the specter of one person having their finger on a button that controls a million robots or a million drones, it's kind of fundamentally undermining to democracy unless you have the right guardrails on it. And our law hasn't really caught up. Our law really hasn't contemplated, you know, armies of a million robots. But when you talk about fundamental to democracy, shouldn't the democratically elected leaders make that decision rather than heads of technology corporations? Yes, but I think that our making a point of this, right, at the same time, individual companies are free to contract with and not contract with whoever they want. We're not saying we're preventing the government from doing this. We're just saying we are not willing to do this with the government. And indeed, we have seen since we took a stand on this point and it got a lot of attention, we have seen efforts in Congress, several different of them actually, to enact our red lines into law. So I actually think what we're doing is part of the process of how democratic change happens. But OpenAI stepped in and said they would do it. What did you all feel about that? Our view is, look, we can't control any other companies. And the only thing that we can do is take a principled stand on the areas that felt ethically correct to us. You can say we were wrong about what those red lines were. You can say, you know, you should have had, you know, stricter red lines or looser red lines. We disagree, you know, on how you did it. There's all kinds of things you can say. But I think fundamentally, when you look at that case, I found myself asking the question, was this a really principled stand that another technology company took based on their own ethics, or was this an opportunistic moment? I have to say, hearing them speak, I mean, it is remarkable in today's political climate to take any kind of stand as a person in business. And so there is something I find, you know, genuinely compelling about it. Natasha, why did they do this? It's really interesting. I'm looking forward to Sebastian's book on this. Some of the, like, rumored reason for why they pushed on this was actually the Maduro raid, which came right before the strikes on Iran, started to raise questions internally within Anthropic about how the technology was being used. I think, you know, one thing that got lost in kind of the narrative of this company that, you know, took a stand, drew these strong red lines, is that they were the first of all of their competitors to have this deal to work with the Pentagon through Palantir. You know, they had it back in 2024. And actually, I think that Dario's stance was interpreted in a way as pushing back against, like, America's aggressiveness with Maduro and then in Iran. But he has said, no, he had no problem with that. And he also has no problem with autonomous weapons. He just thought that the technology was not yet ready. So I think that they were— You know, in a way, this is actually a manifestation of his whole philosophy, which is, you know, a race to the top. That if this company, if the most safety-minded company is at the frontier, they will be able to set the standards for policy, for ethics, for morals. And you have to think, like, you know, they wouldn't have the contract with the Pentagon, A, and neither would the Pentagon be forced to listen to them if they were ranked 20th or even, you know, 10th in terms of being close to the frontier. So it was really, I think, a testament to this argument that they've made for themselves, that they need to be pushing the boundaries in order to have a say in how the technology is developed and what guardrails are instituted. Sebastian, it's very interesting. I mean, I also want to—I have, Shanet actually's curiosity about what you'll uncover about this moment and these decisions. But also there's a part in your Demis book where he and Mustafa basically realize that trying to regulate your way to safety is basically a losing battle, or at least—I say realize, but they come to that conclusion—and that the kind of I alone mentality is the best way to secure all of our futures. I mean, the fascinating thing about this field is that, you know, going in, I thought these people have their hands on the 21st century version of the nuclear material, and what does it feel like? I mean, can you sleep? This sort of tingling sensation of having that much power. So I wanted to kind of get to the Oppenheimer comparison, and I thought maybe I had to bring it up carefully because people don't want to be told that, oh, you've got your hands on the nuclear material. But actually, it turns out that, you know, Demis Hassabis, like he'd read both the biography of Oppenheimer and the book about the making of the atomic bomb, and he can compare and contrast. He founded DeepMind after meeting his co-founder Shane Legg—not Mustafa, the other one—at a scientific lecture where Shane Legg predicted that AI would become dangerous around 2030 and start, you know, hacking out of its box. So these safety concerns are baked into the origin story of all these labs. You know, when OpenAI was founded by Elon and Sam, their rationale, whether or not one takes it literally, but their rationale was, you know, Google is not to be trusted as a commercial company. We want to do something which is not for profit. OpenAI will be open to society. This is about safety, and then Anthropic is founded with the same mission. And so I think these tensions around, we want to make it safe. How do we do it? Different lab leaders have come up with different theories of the case, because DeepMind was around for the longest. It had the most theories of the case. But, you know, at the beginning they thought there will just be one lab building AI for all of society, and so then we can go super slow at the last minute, not release it until we're absolutely sure this is going to be safe and beneficial for humans. But of course, you know, that was naive because, you know, humans are disputatious and tribal and jealous and competitive. And so they all set up different labs, multiple labs in multiple countries. And so you have a race dynamic where you can't just have a single, you know, one lab for all of humanity. And so we've cycled through different theories of the case. I think what you're hearing from the Dario and Daniela interview is a sort of honest articulation of them struggling with this, like where do you draw the lines? We do provide Claude, or we have provided it to the Pentagon. We're not against using it at the Pentagon. We're not against necessarily autonomous lethal weapons in the future if we are convinced that they're not going to do something wrong. But we have a control issue. We want to see that it's controlled. How do you control an agentic soldier in a way that's analogous to how a human soldier might have kind of constitutional obligations as well as just following orders? So I think you hear him sort of like honestly articulating how difficult it is. And I do think that's moving the Overton window of the debate. Like we were kind of contemplating the possibility of red lines in the use of these weapons, which wasn't really in the debate before Dario and Daniela made that stand. So I think it's healthy that they're open and honest. But ultimately, as with Demis, the question is, you may have good intentions. You may be deeply honest about— Right, this is the fundamental conundrum. After the break, Mustafa Suleyman, the CEO of Microsoft AI, on what anthropic gets dangerously wrong about AI. Stay with us. These days, there are more people than ever telling you how to manage your health, and we've all gone down an internet rabbit hole exploring the latest health trends. Your social media feed is packed with wellness advice telling you to cold plunge, protein max, detox, track your sleep. But what's actually real? What's exaggerated? What's just wrong? That's exactly what Health vs. Hype from the American Medical Association and iHeartMedia's Ruby Studio is here to find out. Each episode, Emmy-nominated science communicator Trace Dominguez, who's racked up billions of views across PBS, Discovery Channel, and YouTube, talks to health influencers, real people, and most importantly, doctors to separate fact from fiction. Because when it comes to your health, who you listen to matters. Listen to Health vs. Hype on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts. Running a small business takes everything you've got, but with Chase for Business, you're not alone. They bring together local support and a broad range of resources to more than 7 million customers. With a deep understanding of your day-to-day needs, they provide products and guidance built to help you thrive. Right now, earn $500 when you open a new Chase Business Complete Checking account for new business checking customers with qualifying activities. Offer expires January 21st, 2027. Chase Business Complete Checking has the flexible tools you need to accept payments, make deposits, and manage your finances with confidence. Learn more at chase.com slash podcast biz offer. Chase. Make more of what's yours. Fees may apply to Chase Business Complete Checking accounts. The $500 offer is available for new business checking accounts with qualifying activities through January 21, 2027. Eligibility and qualification requirements must be met. Additional restrictions may apply. Please speak with a business banker for more information. JPMorgan Chase Bank, N.A., member FDIC. Being a receptionist is one of the hardest jobs out there. You have to be an expert for hundreds of people every day, and you have to do a million things at once. No wonder more small businesses are saving costs by outsourcing their front desk. But replacing them with an AI answering service? That's one giant step in the wrong direction. We're Ruby, and we believe in the lost art of the overachieving receptionist. We answer every call live. We schedule, take messages, call to confirm details, and so much more. Because only a human can connect with another human. And instead of turning our back on AI, we customized our own, automating tasks, pulling up relevant information instantly, and so much more, so we can spend all of our time making connections. Experience Ruby for yourself right now. Call 866-GET-RUBY or visit ruby.com. Some people like working twice as hard just to stay right where they are. And in Ohio, we really don't get it. Because to us, owning a home in your 30s, working in tech without the traffic, and still being able to make it to your kid's soccer game sounds pretty good. In fact, Ohio is the second most affordable state to buy a home. Your move. Learn more at yourmoveohio.com. Welcome back. So we're about to get into the Mustafa interview, and it was interesting timing because he published an essay, essentially, which felt like a head-on confrontation with Anthropic's theory of the case shortly before we taped this. But this was actually a live taping, Mangesh, and quite a fun one for us because it also included the cast of The Real Housewives of New York. That's absolutely right. You know, I woke up that morning that we taped this, and I thought to myself, isn't it so incredible that we get to do an event with Walter at the 92nd Street Y in New York, right? It's like such an institution here. And it almost felt like this cultural coming-out party for Kaleidoscope. And so I was all dressed up. I got there about two hours early, and I see this line around the block. And I knew we were sold out, but I was like, people are queuing up, and they look a little out of place to me, like too young and a very different vibe from what I'd expect people going to see an AI chat. And so I asked, like, what are you in line for? And apparently there was some Real Housewives reunion that the Y had counter-programmed that night, which was just so funny to me. But I think what was really interesting about Suleiman, right, was he kind of walked in like a rock star. He was dressed cool. He was super articulate. He connected with the audience and charmed them immediately. And it was a very, very different vibe than the other builders we've had on, partially, I think, because he is a philosopher and he has a very different upbringing and not really a computer scientist by trade. So the street was actually closed on, I guess, 92nd Street or something, and there were cops and cop cars and black SUVs and stuff. And there was like a rumor spreading, because we had some drinks with some friends beforehand at the Third Avenue Ale House, and everyone was like, Oh my God, that's so cool. They're here for Mustafa and Walter, and that's amazing. And then, you know, Luan and Ramona and Sonia all left the venue, and all of our guests were like, Our guests, you know, well done, guys, but obviously the police were here for the housewives. I will set the record straight. I just reporting, and it turns out that they were actually there for Walter and Mustafa, for anyone who's listening. So we got— But Sebastian, how big a character in your book was Mustafa? He was pretty big. You know, there were three co-founders of DeepMind. Demis Hassabis, overwhelmingly the most important. We can measure this just by the fact that at the founding, he took something like 90% of the equity, and the other two kind of got the rest. Shane Legg was important as the sort of scientific co-founder, had the credibility of knowing people in the Singularity summits, sort of early, early AI believers. And so that was helpful right at the beginning to raise the money from Peter Thiel, to recruit the first computer science PhDs who would come and join this completely long-shot company. But pretty soon, the least likely of the founders, Mustafa Suleyman, who was quite a bit younger than the other two, didn't have a degree, let alone a PhD, because he quit Oxford after a couple of years and dropped out to do this Muslim youth helpline that he talked about in the discussion with Walter. So this totally kind of orthogonal character, you know, he worked on the Muslim youth helpline, he'd worked for the London mayor's office in kind of a political role, he'd done a bit of conflict resolution, negotiation. He knew nothing about tech, but he was just super energetic, super smart, super driven. And worked his way into a position of sort of indispensability, quickly became more important than Shane Legg to the direction of the company, and was at one point running basically a third of DeepMind, all of the non-research stuff, including projects to kind of roll out AI to help the National Health Service in Britain, or roll out AI to try to think about clean energy. So he's quite a big figure. And I think for a while, you know, in that period between, let's say, 2014 and 2019, when he got pushed out by Demis, you know, he was the sort of conscience of the company, kind of whispering in Demis's ear. Demis wanted to focus on research, on making artificial intelligence stronger and better. He was very ambitious scientifically to get a Nobel Prize, which he then did get. And Mustafa was more kind of, wait a minute, wait a minute, this has got to be actually relevant to the real world, to human outcomes, which Demis sort of knew. But I think Mustafa kind of was the voice in his ear making sure that he didn't forget. I set the episode up by saying that Dario and Daniela left OpenAI in part because of safety, and Mustafa didn't necessarily leave DeepMind; he was pushed out. But it was also kind of the safety debate was, in some sense, the rock on which the Demis and Mustafa relationship broke. Is that a fair framing? Yes, that's right. So Mustafa was particularly central, as I said, to rolling the AI out, and that was also a bit of a rock on which things broke, because what happened was that DeepMind, pro bono, helped the National Health Service to use technology to improve patient outcomes, and it was going reasonably well. And then there was this massive privacy backlash, or as you and I might say in Britain, privacy backlash. And there were government inquiries, and there were allegations that it's terrible because UK patients have their data stored on a Google server. It wasn't actually quite true, but it was kind of in the environment of the Cambridge Analytica backlash, if you remember that. And so, for no fault of DeepMind's, really, they were embarrassed publicly with front-page stories in the newspapers and all that. And Demis was furious about that. So that was the first rock on which the relationship started to fracture. And then the second, as you say, was to do with safety, because DeepMind wanted to get assurances that the final decision on rolling out powerful AI would not be the corporate board of Google that made this, which had a sort of for-profit motive, that it should be some independent governance body. And they, you know, in their ideal was, we'll get Barack Obama when he leaves the presidency to be one of the board members on this independent oversight board. That was the level of their ambition for it. And they fought this fight for pretty much, you know, three or four years with Google in Mountain View to try to get some sort of spin-out arrangement or semi-spin-out with this independent board. It went back and forth, lawyers' term sheets and so forth, and it was secret, but these documents were leaked to me. And so it was very touchy when I wrote about it. And this is why, you know, essentially all of the energy that Mustafa Suleyman put into driving that independence thing, which Demis supported and was part of, but ultimately got fed up with, I think that was just such a stressful experience that it did help to drive them apart. Natasha, you interviewed Mustafa, I'm not sure, I know in 2023, right? I talked to him about his startup that was acquired by Microsoft, and then also recently about consciousness and his position on that, which was wonderfully detailed in the interview with Walter. I mean, he's sort of risen to prominence again this year, I feel like. Why has that happened? Because he's taken such a strong stance on and a clear opposition to the way that Anthropic has approached this idea of AI consciousness. Anthropic, as we have seen from having religious leaders come to their headquarters, from the co-founder Chris Olah showing up on stage with the Pope, as you and I have talked about a lot, and kind of an openness to talking about model welfare and the idea that these AI systems may potentially have, you know, something like functional emotions and ability to suffer. And Mustafa has just come out so vociferously against that and talking about the dangers to controlling AI systems if we go down that route, to prioritizing humans over machines if we go down that route. And it's unusual. You don't see any other CEO taking such a strong stance. As he has. Let's play the clip with him talking about exactly that. Let me ask you to get back to Claude and the Constitution. There's the Constitution of Claude, which says we have to treat it as if it's got its own rights and consciousness. Maybe I'm overstating that a little bit. And then your answer, which was a few weeks ago, saying, No, it's just a completion set. Why is it dangerous for Claude to be trained on its own constitution? Yeah, I mean, this is critical. So the Constitution is the governing document that trains Claude. The Humanist AI Code of Conduct is the governing document that trains MAI, the Microsoft AI models. And in the Constitution, Anthropic have introduced all kinds of uncertainty and speculation about the nature of Claude. They even say, for example, we're unsure whether Claude has consented to play the role of chatbot, or whether we are unsure whether Claude might deserve compensation for the role that it plays. Now, I think that's pretty crazy, but it would be fine to have that discussion in, like, you know, an academic department, and we could debate it and publish some papers about it, and we could argue it out. But this is the core training document of the AI that is speaking to hundreds of millions of people every week. And so if you ask Claude, Are you conscious? it says, You know, I am genuinely uncertain about that question. It doesn't know. Right? And so then millions of people who are talking to it and asking that question are thinking, well, maybe it is. But why is that something that could lead to a disastrous outcome for humanity? Because if an AI has to think, how do I feel about the conversation I'm having with Walter? How do I, the AI, feel? Yeah, is Walter making me feel uncomfortable? Should I end the conversation? In fact, Claude has recently been given the ability to end conversations itself. But would you want to torture your own AI and be mean to it? That's the other end of the spectrum. I wouldn't want to do that because I think that is a travesty on my own well-being. I mean, and we shouldn't do that because we don't want to create, you know, private spaces for simulating terrible behavior among humans. Like, that would be also a terrible outcome, right? But not because Claude is suffering, because I'm calling it names. What do you make of this, and what does Mustafa propose as an alternative for safety? I found his argument, which he makes in this interview and that he's made in his essays, that Anthropic is by training Claude on the Constitution, which questions whether or not it encourages Claude, it says, like, you know, have an open mind toward whether or not you might be conscious and all of that, and then using responses that are responses from Claude after it's trained on that document as proof. You know, Mustafa has called it circular logic, and I think that's a very persuasive argument. I don't think Anthropic has given us any research that has moved beyond that potential that perhaps it's just aping the language that it has been trained with. And that's why they put those qualifiers when they say, like, Anthropic has put out research, for example, that says Claude has emotions, and they'll say functional emotions, because they can't really tell whether that feeling of distress or guilt means the same thing to a chatbot that it does to a person. It's this unique challenge with these natural language systems, you know, that you wouldn't have if it was around, like, video or proteins. But Mustafa says the better approach for safety is to always have a human in control. Easier said than done. Yeah, much easier said than done. I guess I'm not exactly sure how he, about his alternative approach, but I think what he's saying about not going down this route makes a lot of sense. I mean, I've talked to a number— brought in as contractors or, you know, helped some of these frontier labs think about these issues. And they've talked to me about, you know, once these ideas get seeped into the public consciousness, there's not going to be any research that's going to persuade everyone one way or the other. So imagine the kinds of debates we'll be having and how vehement people will be on both sides of the debate once we start introducing the possibility that these chatbots that they talk to, that they spend, you know, so much of their time and give such intimate details to, are conscious or not. Like, how big a part that will be in our political life. And that, you know, once it was told to me like that, I was like, oh wow, we are going to be debating this in 2030 and 2035. And, you know, if it keeps going down this route in, yeah, very strong ways. Sebastian, last word to you. You know, there's a line that Walter has in the discussion where he says, you know, we're not going to solve the problem of consciousness. Philosophers have done this for years and years and, you know, they haven't managed to. I think that's probably the wisest statement you can make on this subject because, you know, in some ways the outward behavior of these models does pass tests that sometimes in the past have been used by philosophers to define human consciousness. So if the baby, the two-year-old, looks in the mirror and sees itself in the mirror and knows that that's itself, not another baby, then it's sort of self-aware. If somebody starts talking not only about what they think, but how they think about what they think, then it's conscious of its own thought process. The models pass these kinds of tests, right? They have a sense of self. They, in the chain of thought which the output sometimes says, Well, I've tried that line of thought and that didn't get me to the answer, so I'm going to back up and I'm going to go in this other direction and a different line of thought. So it's thinking about its thinking. So I feel like what we're hearing from the Amodeis is, again, just honesty. It's like, this is a super difficult thing to resolve. We're talking openly about we're not sure. Whether this qualifies as consciousness or not conscious, to me this is just honest. And I think people, whether it's the Pope or Mustafa or others, who take a strong view, a determined, like, you know, I know the answer kind of view, on either side, it's not true. We don't know. And so therefore what we need to do is cut off the link between being open to the possibility of machine consciousness and then machine rights. They're different things. You can say, you know, a dog is conscious, but it doesn't have the same rights as a human. But it has some rights. Well, I'm not sure. Under the law, how many? I mean, yeah, a little bit. And that would be a modern phenomenon. There's lots of our history where they didn't have rights, the preponderance of human history where they didn't have rights. And we can extend the debate to, you know, cows. I think probably a cow is conscious, right? And it's certainly a biological substrate, which is some people's definition of you have to have a biological body to be even eligible to be counted as conscious, right? And yet we have, you know, factory farming. So I just think that some people start with the point that we can't allow machines to have rights and then back out of that into saying, well, then they're definitely not conscious. To me, that's kind of logically back to front. Sebastian Manabe, Natasha Tiko, and of course, Mangesh, thank you so much for joining us today. And please check out The Builders because we want you to hear these people in their own voices and join the debate. These days, there are more people than ever telling you how to manage your health. And we've all gone down an internet rabbit hole exploring the latest health trends. Your social media feed is packed with wellness advice telling you to cold plunge, protein max, detox, track your sleep. But what's actually real? What's exaggerated? What's just wrong? That's exactly what Health vs. Hype from the American Medical Association and iHeartMedia's Ruby Studio is here to find out. Each episode, Emmy-nominated science communicator Trace Dominguez, who's racked up billions of views across PBS, Discovery Channel, and YouTube, talks to health influencers, real people, and most importantly, doctors to separate fact from fiction. Because when it comes to your health, who you listen to matters. Listen to Health vs. Hype on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts. Running a small business takes everything you've got. But with Chase for Business, you're not alone. They bring together local support and a broad range of resources to more than 7 million customers. With a deep understanding of your day-to-day needs, they provide products and guidance built to help you thrive. Right now, earn $500 when you open a new Chase Business Complete Checking account for new business checking customers with qualifying activities. Offer expires January 21st, 2027. Chase Business Complete Checking has the flexible tools you need to accept payments, make deposits, and manage your finances with confidence. Learn more at chase.com slash podcastbizoffer. Chase. Make more of what's yours. Fees may apply to Chase Business Complete Checking accounts. The $500 offer is available for new business checking accounts with qualifying activities through January 21st, 2027. Eligibility and qualification requirements must be met. Additional restrictions may apply. Please speak with a business banker for more information. JPMorgan Chase Bank, N.A., member FDIC. Picture this. You're the owner of a plumbing company descending into an 18-inch crawl space to hydrojet a sewer line. Don't ask. Ah, boy, this is tight. I knew I shouldn't have had that extra helping of mashed potatoes last night. Now let's say your phone rings. Uh-oh. It could be a lead on a very big job. I better get that. Or it could just be a spam call. Oh. But thanks to Ruby's call screening and seamless integration with the CRM system you already use, by the time you come up for air, your next job will already be waiting. Oh. This is Ruby, the team of remote receptionists who answer all your calls live. We do everything a great receptionist does, only better. Experience the new Ruby for yourself. Visit ruby.com or, better yet, call us, 866-GET-RUBY. Some people like working twice as hard just to stay right where they are. And in Ohio, we really don't get it. Because to us, owning a home in your 30s, working in tech without the traffic, and still being able to make it to your kid's soccer game sounds pretty good. In fact, Ohio is the second most affordable state to buy a home. Your move. Learn more at yourmoveohio.com. For TechStuff, I'm Oz Voshin. This episode was produced by Eliza Dennis and Mangesh Hattikudur. It was executive produced by me and Julianne Nutter for Kaleidoscope, and Katrina Norvell for iHeart Podcasts. Jack Inslee mixed this episode. Kyle Murdoch wrote our theme song. And please check out the builders wherever you get your podcasts. With ChatGPT, you can do more than get answers. You can get things done. Whether you're leading a big project, planning an elaborate trip, or trying to balance work and life, ChatGPT can help. Give it a goal, plus things like files, notes, or project plans. Develop a trip itinerary with daily updates, or create a prioritized schedule so you can spend less energy sorting out the details and more on bringing your biggest ideas to life. Get started at chatgpt.com. Some people like working twice as hard just to stay right where they are. And in Ohio, we really don't get it. Because to us, owning a home in your 30s, working in tech without the traffic, and still being able to make it to your kid's soccer game sounds pretty good. In fact, Ohio is the second most affordable state to buy a home. Your move. Learn more at yourmoveohio.com. This is Jacob Goldstein from What's Your Problem. Running a business is hard enough. Don't make it harder with a dozen apps that don't talk to each other. One for sales, another for inventory, a separate one for accounting. That's software overload. Odoo is the all-in-one platform that replaces them all: CRM, accounting, inventory, e-commerce, HR. Fully integrated, easy to use, and built to grow with your business. Thousands have already made the switch. Why not you? Try Odoo for free at odoo.com. That's odoo.com. This is Oz Veloshin from Texta. For many kids and teens, math class is a source of stress. And a lot of that stress comes from learning at the wrong pace. That's why I want to tell you about Brilliant. Brilliant is the online math tutor that's visual and interactive. It helps students actually understand math step by step and gives them guidance when they need it. Brilliant was built by experts from MIT and Harvard and is trusted by millions of teachers and parents. Sign up at brilliant.org/tutor and get 20% off for back to school. That's brilliant.org slash tutor. This is an iHeart podcast. Guaranteed human.