The Secrets of Claude's Platform From the Team Who Built It

A year from now, where do you think the platform will be? We'd want to experiment with directions where Claude actually gets so good at understanding itself, it figures out what model you should be using. It figures out how to spin up all the sub-agents. You don't have to think so much about what kind of architectures are there because Claude is actually able to understand itself enough that it can write itself on the fly. In that world, if Claude is on the fly or your agents on the fly are becoming what they need to become in order for you to do what you're trying to do, the platform has to seriously scale. How close are we to Claude making me a billion dollars? That's really what I'm asking. Angela, Caitlin, welcome to the show. Thanks for having us. Yeah, thank you. So for people who don't know, you both work on the platform at Anthropic. So Angela, you're the head of product for the cloud platform. And Caitlin, you are the head of engineering for the cloud platform. I'm really psyched to talk to you because A, you've been launching a bunch of stuff. You have cloud-managed agents that came out recently. You've been launching new features for it. And I think that it comes at this really interesting time where it makes me think about what actually is a platform in AI for a model company. Because in the GPT-3 days, the platform was a completion endpoint. You'd just send a prompt to get a response. After that, it was like a completion endpoint with tool calling and a couple and like chat sessions, like that kind of stuff. And now, like with cloud-managed agents, you're essentially getting a cloud on a computer with memory and all this other stuff. So I'm just trying to, I'd love to help, I'd love for you to help me unpack that trajectory and like what it means to build a platform in AI. Yeah, I think like, you know, your characterization is like very accurate. I think like as we've kind of like, a lot of these kind of like technologies have evolved with the LM like first starting. And then I think like putting that behind an API was very fun. A lot of people were like, wow, I could like do some, at the time, I think it was very cool. Now we'll probably look back on it and be like, oh, that was like really basic. And then, you know, I think like we've moved more and more towards like a slightly more like stable world as you kind of like want to persist the kind of like sessions, state to be able to make sure that the kind of performance of the model is like better and better. I think that that's probably like actually the, the through line, like as a lot of these kind of like, as we make improvements to Claude and as it continues to get better and like more autonomous, we find ourselves like basically needing to kind of like evolve the platform to be sort of like higher and higher order abstraction, but it's in the pursuit of like helping you get the best outcomes out of something. Like, I think in the very beginning, you know, we were very like, everyone was very exploratory. It's like, you'd have no idea what people are going to build with these LLMs. And you wanted to kind of have as much possibility out there as, as available. And then as those use cases started to kind of narrow down, like people started building products with it. People started now like building agents with it. And more and more of that is about, you know, like customers coming to us and being like, how do I get the best out of Claude? How do I like set up my tools? How do I run the loop? And so on and so forth. And you have some people who are like really, really experimenting and they're on the edges. And that's great. And then you have like just a whole host of other folks that are coming in who are like, I kind of want a lot of this stuff like out of the box. And in our pursuit for getting, making sure that like Claude is basically producing the best outcomes, we find ourselves like enriching the platform to be richer and richer and richer. And that's, you know, contained in that is like both the state. It's like the tools that you start to see us adding. It contains a lot of kind of like, almost like sort of the cloud components of a lot of these types of things. But it's in pursuit of the same mission of like just making things literally as easy as possible. And I think in probably, you know, the forward state of a lot of these things in terms of maybe the philosophy of what a platform ultimately ends up doing, it probably ends up just being like whatever it is, it's like the set of primitives and infrastructure that enables you to basically get the outcome as fast as possible with actually as little of work as possible. And I think that that tends to follow a certain form factor, at least in this current state. But yeah. How would you characterize like what the primitives are today? So maybe that's just asking, what are the primitives in cloud-managed agents? Yeah, so cloud-managed agents is built on all of our same primitives that you could otherwise build on directly, so the messages API. And within the messages API, we've built a whole bunch of, I guess, maybe innovations around the API. Like you could just get tokens in and out if you really wanted to, but you know, you can use some of our built-in tools. You can use stuff like code execution, spawn a sandbox, and execute work. You can use, I guess, like, you know, web search and all these sorts of different things. And so I think we've taken what we see as all the most powerful of those things and put them together into a harness and a set of infrastructure that is, you know, just the way to get what we think is the best outcomes out of Claude. So I'm sitting here feeling this sense of, I've been thinking of it as like time deflation. Like my time gets more valuable in the future as opposed to the opposite, whatever the opposite would be. My time gets less valuable in the future. And the reason is because we're, so for example, internally for us, we're building an agent. We're building some agent products where it's like agents that do specific things for us internally and then hopefully for customers. And in order to do that, we've like, you know, we have a couple of Mac minis with, you know, Claude running in a loop on the Mac mini, right? And a lot of that, and it's like a thousand line Python file or whatever. And a lot of that mirrors what you guys are building in cloud managed agents. And so for, for, for me, and I think for a lot of people building on cloud or on the cloud platform or ecosystem, there's a, there's at least I feel this, maybe we should just wait for you guys to build it. Um, but then I don't know what the lines are. And uh, and I, yeah, I'm sort of wondering if, if I want to build an agent, like what is the best path to do that in a way that aligns with what you guys are doing? Yeah. Um, I think, you know, this, this part of the, the kind of platform business is actually somewhat similar to any other form of platform business where you do have customers like yourself who are building and you know, you're kind of thinking, should I go ahead and do it? Because maybe I have this like immediate need, but at the same time, I don't kind of want to like, you know, repeat the work per se. And you could have just, when you could have just gotten a native free out of the platform. And also infrastructure sucks. It sucks so much to like spin up servers. I can't believe you do that all the time. Like that's actually a big one. That part everyone's like, that's, that's the hardest part. But I will actually say, um, part of why we ended up building cloud managed agents was because Anthropic ourselves had gone through enough of these iterations where we built products that were agents that you could run autonomously in the cloud. And we did that, stand up the infrastructure so that it works well, sort of work enough times that, um, we ourselves were like, okay, we're done building this for ourselves. We're doing it once in a way that's going to really work from everything that we've learned, but also for all the people who are doing it. Like you can run whatever you're running on a couple of Mac minis, maybe. Right. And for a lot of people that could work, but I think if you're building agents into your product and you're running something really at scale, right? Like that's where it really starts to become more and more challenging to get that infrastructure right. That's really interesting. Yeah. And then maybe to answer the other part of your question, I think we have like two pieces of the philosophy here. One is, is a bit in the way that we kind of design managed agents, which is that we try to have it be modular enough. Like we want to be opinionated about some pieces that we feel like should be, you know, very well, like married to the cloud model. Um, but then we, uh, like oftentimes like the way we want, for example, we want cloud to like very specifically use like file systems. Um, that's like a very particular like cloud style. In a specific way or just file systems in general? Just file systems in general. We also really want to lean into skills. I know like a lot of folks like skills, but like, that's something that we like, we want to have our harness to be really opinionated about that. And so we're kind of particular about like those kind of primitives being the case. So like use the file systems, use the skills. They're really basic. Um, but at And you do kind of hit like a little local maximum and rethink like, okay, maybe there's like a more generic approach to doing it. Yeah, yeah, yeah. Interesting. I want to take a step back and ask you something that maybe I should have asked at the beginning, which is, who is cloud management for, right? Like, I set one up earlier today. We've got some people already using it in production inside of Every. And I just, I just do one today. I really loved the sort of like getting started chat experience that you, that you had and the sort of some of the examples that you had. And it felt to me like even if I was not technical, I might want to use this to set up an agent. It, it might be a little bit complicated, but what I actually did is I just, and I'm sorry to say this, but I did it in the codex in-app browser. So I had codex driving the, uh the manage agent set up and it like, I had a Slack bot working pretty, pretty quickly. It was like, it was really cool. So how do you think about when you're designing stuff, when you're designing cloud management, who it's for? Yeah. So it's interesting because I think you're right that especially with that quick start experience, which we actually felt pretty strongly about launching, not specifically for the sake of making it so that non-technical people could go and build agents, but actually just for anybody technical or not to be able to wrap their head around the primitives like the APIs. Here's what it can do and here's how it fits together. Yeah, exactly. Like, you know, the kind of education portion of it. But I think when we think about who it's for, we think about a couple different things. One is, we're seeing people internally within companies build automation or build really powerful platforms or systems. Like we've seen people say, I want, you know, a full end-to-end software development platform, right? And like manage agents is a perfect solution for something like that. Or, you know, I want to automate a little process over here where like legal has to review my marketing copy, right? And things like that. And so you shouldn't have to reimplement memory and like all that stuff every time you're doing it. Right. You can get started really quickly and you can get something running quickly. The other user that's top of mind for us is people building into their products that they expose to their customers. And so that's the other one where actually, yes, like you do still want a lot of customization. You do still want to make something that's going to be really powerful for your product. But we still definitely, definitely believe that not spending your engineering resources on the infrastructure and on all the little harness engineering tweaking sort of stuff is worthwhile. Why couldn't we have talked like a month ago? You would have saved me so much time. We'll just need to talk more. But I am, I am sort of curious. Okay, so maybe infrastructure is one of these things, but when you see people setting up agents, what do they, what do you see them think the hard thing is and what ends up actually being the hard thing? And are they the same? Good question. I, I maybe this is, I don't know, spicy. I'm not sure, but I think, I think people think the harness engineering part is the hard part. Um, and so actually, like, you know, in the past we launched the agent SDK, which is what you guys I think are using on your Mac minis. And for a lot of people, they were like, okay, great. I don't have to do the harness engineering part where I have to do prompt caching and I have to maximize my context window and all these sorts of things. I think we're just actually using just cloud in batch, like the cloud dash P command. Oh wow. Okay. It's, it's pretty good. Yeah. Yeah. That's cool. Okay, cool. And, but regardless, like you guys did that because it takes off your hands building the harness, right? Um, but I do think what we saw with a lot of customers was, okay, now I want to go and take that thing and like get it into production and scale it. And everybody hits an infrastructure wall. Like everyone hits the same problem of like, Oh wow, I either need to like keep a server constantly running or I need to use infrastructure that will spin up and spin down. And I need to store the transcript data and I need to secure sandboxes and all these sorts of things. And so, um, you know, and like if you boot a cloud code session or you boot the agent SDK in a sandbox and like that's the thing that you have running, but your sandbox loses connection and dies or whatever, your whole agent dies. Right. And so I think the infrastructure part especially is the wall that most people end up hitting, but they're more expecting that the actual harness engineering and like getting the most out of the model is the part that's going to be harder. Yeah. I totally agree with that. I was just going to say like, you know, we, we talked to so many people who are now at a place where they're like prototyping really quickly and they're super excited and it's like, it's doing the thing. And then yet there's like a class of people who are, you know, really pushing and being like, okay, I, I do want to Hill climb. I really want to edit the harness. Um, but then once you have that thing, like productionizing is just a freaking nightmare. Um, especially for like the more interesting kind of long running async ones that you want to do a bit more remotely that are a bit more autonomous. Um, and everyone kind of runs into that wall. And it was a big inspiration for why we built what we built. I feel like, uh, one of the like ER examples of the shape of an agent is OpenCLO. Um, and in particular, the, the thing that it has brought to us internally is you have an always on agent in Slack that has its own personality and has its own like part of the world that it like ends up working on. Are you guys like, is, is that a possible future for like, okay, one click agent that lives in my Slack that, yes, I can go set up all the internals, but like I don't have to really think about all of the, um, you know, the technical infrastructure stuff. Um, because I, I think you, you all have the beginnings of that, but it's still like a lot of steps from the current managed agent to something that's always on in my Slack that I have to like set up and customize. So is that, does that fall in the realm of platform's job? Or is it like too far in the product direction? No, it, it definitely is something that we'd really want to do. I think like, you know, we, we focused a lot on kind of the infrastructure piece to start because that's where we just see a lot of these like pain points. Um, but yes, like I think in like in, it's like, you know, I don't want to say exactly it's a final shape, but in its like advanced shape, we actually want to make it so that you can kind of deploy these agents really, really easily. Like, um, we've made like some light steps in this direction. Like for example, we included faults, um, as one of the primitives as just kind of In vaults to store your like keys and stuff, like your OAuth keys. Credentials. Credentials. Yeah. Um, as like, you know, kind of solving some of the lower level pieces as a starting point. But once you kind of wrap some of these more sort of like agent identity type of primitives in a more secure way and you can handle it really easily and it works with like the whole like system, then, uh, you know, I think it's very natural for us to get to a place where maybe you are either one-clicking a Slack integration or alternatively, you know, maybe just telling, you know, Claude, like, add Slack and it just like handles absolutely everything. And then before you know it, your little bot is just pinging you on Slack. I love it. I can't wait for that world. Um, what are the best internal use cases of agents? Because I think there's this big question happening right now where, okay, yeah, everyone's in codex or cloud code, but then now we have these agents that are out in the cloud. Now everyone inside of a company can like have their own agent. There are team agents, there are company-wide agents. So what are the patterns that you see for when people make really useful internal agents, what they do and what they look like? Yeah, I would say we, similar to, and we've actually seen a few examples of these in some of the more like AI-pilled AGI-pilled companies like Stripe built Minions. And they talked about that a lot as their kind of like end-to-end development platform that their engineers could use. Um, I think Ramp did something similar. And um, we've done similar things as well, right? Like, yeah, we've built kind of platforms internally that are, you know, I have agents running that I can talk to you from Slack or from wherever, right? And um, at a certain point that becomes actually like a pretty thin layer on top of managed agents. Like you don't have to do very much to accomplish it. That's what I was thinking. Like I looked at minions or whatever ramp does and I was like, why, why, you know? So is it, is it actually useful to have a sort of like thin coding agent that anyone in the company can use or like, why not just install the cloud app in Slack? Yeah, I would So sometimes they're probably going to submit stupid stuff that wastes time, and so what are the what are the right ways to either organizationally, like culturally or technically, like, make that possible without ruining your lives? For this particular one that we've constructed that Kayla's given as an example, we actually have like a couple layers of abstraction away from like that kind of like PR layer. So at the very beginning, it kind of like started that way. And to kind of like basically prevent users from kind of foot gunning themselves a little bit, they kind of get to a place where oftentimes their way of interacting with the agents that they own, like whether it's the marketing team who owns the marketing agent requesting or if it's the legal team, you know, owning the agent that does the review, they actually engage with those agents through Claude itself. So they actually spend more of their time like kind of talking directly to Claude and then Claude will oftentimes figure out what should be the right way for them to go and handle it so that they're not kind of like, you know, hopping straight down to the absolute core bits and doing something that may result in, you know, some complications. And they're talking to Claude or Claude code, like Claude chat or Claude code or co-worker or any of them? It's a different instantiation of Claude that we made that actually is a managed agent in and of itself. So it's just kind of like managed agents all the way down in that construct. But we found that each layer, if we kind of tune and prompt each variant of the managed agent, it helps to solve like, you know, different parts of the problem for users. So at the end state for, you know, like that marketing person or that legal person, it is like a really simple interface where the way that we tell them is like, you're just talking to Claude, but under the hood, it's many, many Claudes engaging with each other to get to the part where then they, the Claudes themselves are doing the more complex work that the human doesn't really necessarily need to interpret. Interesting. You guys just launched multi-agent orchestration. What are the coolest, what are the coolest things that people are doing with that? One of the more interesting ones is like, I think people are using it to like construct sort of different harness techniques. And that one I'm personally very excited by because like there's different techniques that people have experimented with where You know, like, for example, we recently did like the advisor strategy one, but really if you were to genericize it, you just separate like execution from advice. And there's also one where you can have like two, you know, modes where there one is generating someone something and the other one's adversarial to it. And then there could also be sort of like, you know, you split it into a bunch of different like little tiny pieces and then they kind of recombine. And then there's ones where maybe it's kind of something closer to like best of N kind of like style of thing. And then there's so many more. And like in each one of these different types of like architectures or strategies, they are good for a very specific use case. So some of them are much better for like uh deep research or wide research type of uh style use cases, right? And there are others that are like, these are like the kind of ones where they all sort of swarm together are better for like bug hunting, for example. Um and so like that's like really cool to see that like if we can make the primitives very Lego-like, um then people can put them together to solve things at a slightly higher form factor, which is more like an architecture or like a strategy. Um and they get much more like interesting results out of that. Um and that's like really exciting to see cause it also suggests that you can actually hill climb at multiple layers um of abstraction. How do you know if an agent is successful? How do you measure success for an agent? Yeah, I mean, there's like evals and stuff like that, which everyone has talked about like ad nauseam. One direction that we really like is like uh this kind of verifiable outcome. Um we've been somewhat opinionated on that one, and it's almost like in the absolute end state of, you know, we talked a little bit about what's what's a platform, the end of things. Um going from that philosophy, it's like our kind of principle of like maybe the end state of some of these things is that everything should kind of compress down to an outcome and like a budget. And that's probably like about it. Um and everything else should be figured out for you to kind of resolve exactly across those parameters. And so for us, we're kind of, yes, we still have evals. We have a lot of these other things that we measure um that are domain specific. Like, you know, some coding evals would be like, you might want to measure like just the actual PR getting merged. Those are more verifiable. But we have to get to the place where, um, you know, like an outcome is actually a spec that you are just as a human able to define and our ability to interpret that and regrade itself over and over is closer to what we care about. Claude, make me a billion dollars. Your budget is $10. Exactly, and then say no mistakes. Go. Exactly. Maybe Mythos could do that. Um and then one of the things that we've been running into that I'm curious if you have a solution for is agents like get outdated pretty quickly. Um sometimes because there's no human attached to them, sometimes like they're just running an old model or there's an old, or in an old architecture or whatever. And it feels like there needs to be a um uh end of life cycle for agents. Like we've talked about having like a little like funeral for them and like having like a little page on our website that's like, here's all the decommissioned agents and stuff. Um like how, how do you manage, especially in a company, in a really big company, how do you manage the all of the agents that are like sort of out there, but and maybe they're like in Slack, like pinging stuff once a week, but you're like, this is super stale. How do you make sure that uh you you retire them as quickly as you are making them? So one of the things we have actually done is we have made skills that help you do things like upgrade to a new model when a new model comes out, right? Like, we've actually put a good amount of work into making it easier to do exactly what you're talking about. Um and I think maybe some of the most like AGI-pilled people are like running agents that are monitoring their agents to see if their agents are, you know, like outdated and in need of that sort of stuff. But I think for, you know, the way that we like to talk to customers who ask us this question, I do think the, the most interesting instantiation of this is, there's a new model and now I need to go upgrade my agents or maybe be done with those agents because, you know, the new model enables me to build agents that are way more powerful and do more interesting things than the old agents did, right? But I think that upgrade process and that migration process is like something people have had to wrap their heads around as like, it's like a breaking change and I have to like put actual energy into making that work. Um and obviously, sorry to talk about evals, but like if you have evals, this process is easier and things like this, but um, I do think that's one of the things we've tried to do is how do we give you skills and how do we give you the right like just tools to make that process easier? Um and then you could go be AGI-pilled and choose to actually automate more of that with more agents. Yeah. So a year from now, we're back at code with Claude. Um where do you think the platform will be? What will I be able to do, uh, and how it will be different from what I can do today? Do you wanna go first? You can go first. A year is a long time. It's in this, in this industry especially. How close are we to Claude make me a billion dollars? That's really what I'm asking. I think we're probably won't be sitting here in a year. Yes, yes. We will be asking Claudus. I mean, yeah, like we wanna get closer and closer to that, that state where I think we, we kind of, okay, so a couple things. I think in a year from now, I mean, one thing that we'd love to get really, really close to is actually that kind of like simplicity, and this might be a significantly higher order of abstraction. I don't know what the form factor will look like or whatever, but the kind of parameters we will care for from users will be that outcome. And of course, it has to be verifiable. There are some parameters that, that have to be restrictive and, and the budget. And I think like, we'd want to experiment with, with directions where Claude actually gets so good at understanding itself. It, it figures out what model you should be using. It figures out how to spin up all the sub-agents. I actually don't think you need to think so much about harness engineering in that world. Today, you know, you don't have to think so much more aggressively about like tool construction, for example, like we've kind of made that a little easier and you get it to leave a little bit of that scaffolding. Less prompt engineering to do. Yeah, exactly. Exactly. And I think if you just keep going up that stack, like today, a lot of the innovation is happening at this kind of like, like, like really high level, almost like harness architecture like level, which is really fun, but I think a