#107: How synthetic users are changing product decision making | Tom Charman (Co-founder @ Blok)

Amazing. Welcome, Tom. Thank you. I'm so excited to have you. I'm excited to have you. Yeah, I really enjoyed your roundtable session. Go ahead, Mark. Yeah, I was going to say that I've been really excited to have Tom here with us since we had that private roundtable in Supra. It was a big hit, probably one of the biggest ones we've had this year. And I think the reason why it really resonated is because it spoke to two things. I think you made the concept of synthetic users, which is like a term that gets thrown around a lot and way more almost easy to understand. And then also you kind of talked about this key topic that's basically PMs are becoming the bottleneck, because when building gets easier and kind of building the right thing, it's hard. And I think that's maybe what I want to start the conversation. And there's a lot of tools today, thanks to AI, that are making parts of the PM job and parts of the builder job easier. You have cloud code that allows you to maybe ship what before maybe took months and maybe hours. You can write specs also in 15 minutes, thanks to a lot of these great LLMs. But I think what's getting harder is there's no tools that are giving people the intuition for what to ship or to help them think critically when they make changes to the product, to the users. And I think you're someone who is building in that space. So maybe let's start there. Is that matching what you're seeing when you talk to product teams all day? Yeah, 100%. I think knowing what to build for who and when is kind of like when people say, what's the vision of this company? It's like helping people understand what to build for who and when. That's the underlying thing that we're trying to solve. And I think it's such a hard question to answer because lots of people need lots of different things. There's so many things happening in the market right now. Lots of people are talking about adaptive user interfaces or dynamic interfaces, whatever the term of the week is from that perspective. You're seeing this collapse of roles where we as product people are expected to do three times the amount of work with a third of the amount of resource. Their shipping is quicker than ever and we can ship faster. But then at the same time, because we're shipping faster using artificial intelligence, suddenly like you probably saw recently with Anthropic basically saying that the new models that they're building are kind of fundamentally reshaped cybersecurity because they can break things really easily. So what is the future of even pen testing, going into that InfoSec world look like? So I think we're seeing this whole thing change really, really quickly. And I think a lot of the product teams that we speak to are like, what on earth does the world look like? We're used to this 2005 A-B testing, StatsSec, how do we get there? Is this the right thing for the right user, single user interface? And all of a sudden, all of our worldview has been completely challenged and we're being like, oh, actually, we're going to rethink this whole thing. It's like the internet's being rebuilt. Yeah, go ahead, Mark. The only thing that I want to say is it almost feels like things are changing very quickly, but the way product people get intelligence to make decisions hasn't changed that fast enough. A-B tests haven't been redesigned and you still need a lot of power to get statistical significance. You need a lot of people to use your product and time. So it's almost like there's this waiting game with A-B tests, that's one. And then the other one is user research. User research, there's all these great agents, you can go out into the wild, they'll do the interviews for you, but that still takes time. People need to make time out of their days. They need to spend 30 minutes and then you need to talk to a bunch of people, aggregate the insights. That's also not instant. Maybe it's a little bit faster. So I think that's what's interesting too. No one has to rebuild those two things from the ground up. But Ben, what were you going to say? Yeah, no, Tom, do you want to react to that before I ask my follow-up question? Yeah, I mean, the thing I will say is 100% right. Two things are true here, right? From a user research perspective, when we talk to user researchers, they're kind of like, look, speaking to humans is the most important thing because they're going to give us candid feedback. Two things that I often challenge are most people are paying people to do this user research, which means that fundamentally, if someone pays me money, I'm going to tell you nice things. I'm going to say you're amazing because you paid me money to do this. I'm being unfair, and I'm saying that everyone's going to be nice from that perspective, and that's not entirely true. But I think the other thing that we find with user research is people will also say, well, it's so important that you test the full plane of different types of customers that exist. But then when you actually dig into it with user research teams, it's sometimes really hard to find that 58-year-old from Minnesota who's got a very niche set of interests. So you often just default to the people that are your top users, the people that are using your platform all the time, and then you forget about all of the people that actually aren't going to give you feedback. So you're skewing your data in multiple ways from that perspective. So I think that's something that just needs to change. I think the other thing that's true is the A-B testing side of things. It is exactly that. We need a lot of data to get to StatsSync. For the newsfeed on Meta, like for Facebook's newsfeed, you can get StatsSync very quickly. There's enough people using that tool to be able to work out when you ship something whether you're actually shipping the right thing. But as we move towards a world of adaptive user interfaces, well, let's assume that you've got 15 different versions of the newsfeed. Well, in theory, you now need 15 times the amount of data to get to that same level of StatsSync that you needed before. So I think two things are happening in the world right now. One is obviously we're shipping faster than ever. And two is like this personalization that we've been promised for, I don't know, two decades of people being like, I'm going to give you personalized everything is suddenly getting closer. And as it gets closer and closer, there are tools that we're using to try and understand whether people actually like the experiences we're building for them become more and more kind of abstract and more difficult to kind of qualify. And then the third thing I'll very, very quickly add is there's this other thing that's happening very quickly, which is agents are suddenly becoming a plain product. You know, is the agent now a customer? How are agents going to react with their flows? How are agent humans' reactions different to human-human interactions? Like some of this stuff is also something that we're having to spend a lot of time thinking about and work out as things move forward as well. Yeah. And that's actually an interesting shift, right? Because I think in terms of product, I think for a long time, people were like, hey, like actually don't worry about like the people who are not power users or the people that don't love the product, like build for people that are huge fans of what you're doing. And then just trying to find more people like that. And same thing with like, you know, like the best way to fix churn is not to like listen to people that churn is maybe to like do more, like find more people of like this. So yeah, it sounds like you're hearing maybe like, hey, like there might be a need to also start kind of like broadening that aperture a little bit more. And is that the case or is it more of like kind of you're starting to segment the power users into more like detailed personas and kind of knowing that you're moving? Yeah. Yeah. Yeah. There are two sides to it. Like, I think we are hearing product teams wanting to speak to the churn user more and more to understand if it comes down to like a fundamental disagreement with the product and they're like, they're just never going to be a power user or it's because the experience wasn't quite right for them. And as we now can create more dynamic experiences, you can potentially open up your market from that perspective. So I think that's the kind of the first thing that we're seeing product teams think more about. And I think the second thing is up until now, one of the things we see product teams do a lot is kind of like optimize for first order magnitude principles. Like it's kind of like, okay, I want to optimize conversions. So in the short term, I'm going to make product changes that are going to drive conversions within the platform. But then you're not necessarily thinking about second and third order magnitude problems, which might be, hey, we've created an entirely new onboarding flow, which is making it easiest for our users to convert and become customers. But actually they're becoming a little bit more confused about what we actually are as a business. So what was a customer that was sticking around for 12 months is now suddenly churning away at six months. So then the second and third order magnitude effects that are quite literally killing your business in the longer term as something that product teams are also spending a lot more time thinking about. And it's hard to predict what's going to happen, except if we just sit and wait for six months or 12 months to see what happens. The benefit of simulation is you can begin to think about some of these second and third order magnitude effects and predict what's going to happen. Yeah. I feel like we kind of jumped in and let's definitely keep exploring from where we're at. But just for those listening who might not know as much about either block or synthetic users in general, can I give just like a quick recap of my understanding based on what I understood and you can correct the record and then we can kind of keep exploring. 100%. So I'm sure I'm going to be dramatically oversimplifying this concept, but my understanding is that one of the major types of use cases for a product like block is a team that has amplitude or mixed panel or some kind of like data warehouse situation going on will pipe their data into a product like block. And then you will kind of figure out the various types of personas. I don't know if you call them agents or if you call them personas, I think it might be the word. It's un-interchangeable. Yeah. Yeah. And the idea is each of them have defining kind of like perspectives and jobs and like, I don't know how deep the rabbit hole goes on what you could pack into a persona. But the idea is that I can run some kind of like experiment with a bunch of like a virtual population. So like there's a virtual population that can be inhabited by a combination of personas and they can go and I can just like throw them at my, this new landing page that I'm thinking about testing, for example. And I can ask a question, which is basically like, will this landing page convert better than my current landing page? And if so, will it also maybe like retain better three months down the road or something like that? And I, so I can drop a question like that into block, run the experiment. There's a tremendous amount of complexity that goes under the hood to power this experience. But my understanding is like the results will then come back and I will be able to get some kind of, I think you try to stay within, I think you said 80 to 85% like confidence interval on like accuracy is like, but if I'm trying to improve conversion by some amount and you're like, yeah, we think this can improve conversion by 30%. Then when I ship that, I should see conversion go somewhere between 25 and 35% ideally. So you're trying to de-risk and all that happens without talking to a single human. I can just do that programmatically in your product. Is that feel free to, yeah. Tell me how I'm wrong or like what I'm missing about the high level definition maybe of like synthetic users and how it works. No, a hundred percent. You hit the nail on the head. I think the thing that we are spending most of our time really focused on is going deep. Like there are two approaches to this problem. There is build a population of people and basically synthetic people can simulate them out and see how they're going to behave or ask them how they're going to behave, which is kind of how a lot of people in the space of building right now. I've always been of the view that like show, don't tell, like I would much prefer someone to show me a product and show me essentially how they would behave as opposed to tell me how they would behave. So the approach that we take is slightly more complex than basically just asking a question. You are, you as a user are just asking a question. What we're doing under the hood is we're then basically taking these personas. We're giving them arms and legs as we call them internally, which basically means mobile using capability so you can upload your own apps. Browse using if you want to test on browser. Figma if you want to test on Figma. Giving you the ability to actually like not just ask it a question, but see it go through and actually interface with the product in real time. So you're getting this kind of like stream of consciousness that's coming from the agent and it's kind of like that's the stream of consciousness of the human mind. Like where am I getting frustrated? Where is the drop off happening? Why am I giving up? What's concerning me at a step-by-step breakdown as opposed to does my signup flow make sense? Our view is like the signup flow could make sense, but there could still be areas of frustration within the flow. And if you fix those areas of frustration, you're just going to give people a better experience. So that's kind of like the big thing for us is like give each of the agents their own mailboxes. Give each of the agents the ability to interface with things. Like when we're building up mobile apps, you can literally interface with a virtual environment, a virtual mobile phone. They can close the app. They can go to other parts of the phone. They can interface in different ways. They can do two-factor authentication with email. Like really it's trying to create as much of that human experience as possible so that when you're testing something, you're seeing as true a version as if a human was doing it as possible. So those are the limited number of personas. That's like my QA persona or my copywriting guru persona or something. Like I can define these things maybe. But then there's also this other type of, I guess, what would the term be to use to describe like the, let's say if I have millions of customers and I want to have, I want to get a good feel for what like elderly people in Central and like in Middle America might be thinking about my experience as they go through the product. Like is that a persona or what is that called? Yeah. So that's like a behavioral persona and you've hit the nail on the head. Like with the behavioral personas, we're looking at four things. We're looking at demographic data. So like where in the world are they? How old are they? All that kind of stuff. We're looking at psychographics. So that's kind of like the personality of a persona. We're looking at the emotional behavior. So it's kind of frustration and like how is that going to affect how someone interfaces with something. And then you've got memory, which is like how is the persona actually going to behave within the product? The brash example that I tend to give is most people that are building in the agent space right now is give agents maximum context, maximum compute, maximum memory so they can complete the task as best they possibly can. And our expert personas have access to that. They have access to the context you feed it. It might be what our brand definitions are, what a good QA test looks like from that perspective. When it comes to behavioral, the example that I give is like imagine if I were to punch someone in the face. I'm not going to do that. But like imagine if I were like the next five times and probably and hopefully more than five times, that person's going to have a pretty negative view of me. They're going to be like, dude, you punched me in the face. Like what's wrong with you? The same is true with user experiences. If you give someone a crappy first time user experience, the next time that they come to your platform, they're going to remember that negative connotation of when they first use your platforms. They're going to come to the platform with a level of skepticism. And it's, you know, being able to model that deep and it's, you know, modeling even further than that and saying like, hey, do you know what? We remember the bad things more than we remember the good things. So even if you give people a great experience the first time and the next time they use it, there's a bug. Well, then they're going to forget that great experience they had the first time. They're going to become a cynical user. You now have to spend a bunch of time fixing. So bringing that extra context from the kind of behavioral, emotional, psychographic and demographic information is really what allows you to see how different types of users are perceiving your product. Got it, Mark. I'm thinking about how, you know, like when we're designing our landing page for Insider Loops, for example, like we're trying to, you know, there's like a bunch of assumptions we're making about the visitors to our website and we're trying to design the website in a way that kind of streamlines and creates like a path for certain people. But in my head, as you're talking, Tom, I'm curious how some of your customers are thinking about systematizing this, but I'm sure there's some people that come to our website that are just curious, right? Like they saw us post about it. They're just like, oh, what is this thing? Right. And then there's probably some other people that come to the website and they're like maybe skeptical and they're like, oh, like, I don't know. This is probably not what they say it is. And there's probably other people that maybe their best friend told them this is really good and they should go check it out. And they're like, OK, I know exactly how my interview is scheduled. I'm going to go buy the guide for this thing right now. So are you finding your customers also kind of like creating almost these different starting points, I guess? What's the terminology even to think about defining these different audiences? Yeah, they're these different events. I mean, they're different. All of those are different behavioral personas, essentially. So you might run an experiment on your landing page with each of those three types of users. And so many things are going to influence that experience. Like if you take an anxious user as an example, if an anxious user hits a neobanks website and there's like I'm giving you an example of like our anxious agent that tends to go through things when we're evaling stuff. Like if it sees no credit card required and immediately some logos that that particular neobank is working with already, there's a level of trust that goes up. So suddenly that anxious user is more like, OK, cool, I'm actually going to move through the next step and I'm going to continue. Equally, if that same anxious user tries to put in their email to join the waitlist and they get an error message, now you send that anxious user into a panic and they now distrust your site. This is super high level stuff. But like this is kind of like some of the stuff that you can begin to do. And this is where you're beginning to see these friction points of like, OK, cool, like sure, we can QA test and we can make sure that there are no bugs on the page. But it's one thing to QA test and make sure that there are no bugs. It's another thing to QA test from a human behavior perspective and work out whether you've got the right fundamentals on your page that is going to convert the right type of user for you in this case, like it might be the anxious user. So testimonials, reviews like that kind of stuff. That's what's going to give this person courage to be able to say I'm prepared to take that on this product. Yeah. Yeah, that makes total sense. Sorry. Go ahead, Mark. No, I think that's what's the cool thing about this type of products is that you get like the micro details of like screen by screen or even field by field like reaction. But then also like the like the macro of like, hey, does it actually like the answer to that question? And I'm sure like the answer to that question is like the combination of all the micro experiences, which makes a lot of sense. Mark, can I actually can I can I ask just a quick clarifying question before you before you keep going? So I'm just like imagining like a basic someone trying to like improve conversion or gauge gauge likelihood to like increase conversion rate with certain experiment. Let's say today in the wild, it takes someone 30 seconds looking at the landing page and then like 10 seconds looking at the sign up page and then like 20 seconds to do email verification or something to get into the product. Right. There's these like flags. Does your product seem like simulate the those like human judgment lapses like the time pauses or do you or does it actually wait that long to like do these things? It's not built into the persona. So we're not like wait 20 seconds before you make a decision. It's going interface and it will probably take 20 seconds because the behavioral traits, the underlying behaviors, like that's the thing that then essentially like, okay, I'm going to spend a bit more time on this homepage. I'm going to scroll down. I'm going to read more like, yeah. Okay, cool. The agent's not reading. It's just ingesting the content and making it's going to give you again a stream of consciousness as to how it feels about things. But like in essence, yes, like, you know, someone that is a inpatient user that just wants to get to the sign up flow as quickly as possible isn't going to spend 30 seconds on the homepage. They're just going to go straight to sign up and they're going to try not to try and sign up for the platform. That anxious user is going to spend a bit more time. They're going to scroll. They're going to learn a little bit more about the brands. They're going to try and find the brands that they find and perceive to be ones that they resonate with. And then that's what's then going to get them to scroll back up and then click the sign up button and actually go through the flow. Amazing. Okay. Thank you for clarifying that for me, Mark. Keep going. That was super helpful. Yeah. Yeah. Now I want to get into the, I think the second degree order impact and the third degree order impact. There's something really interesting here because I think we've all seen like the growth teams that kind of optimize for like those like first degree, right? Hey, like I want to increase conversion. And that's, that's kind of like how they measure success, right? They're not responsible for churn. That's like a different team. And so it's almost like people are not trained to think about like the second degree or third degree until there's like a problem. You're like, oh shit, like our retention is shit. And people are kind of stopping and using the product after six months. Yeah. And I think what it's, and in a way products like Block can make this worse because like if you, if you like, cause if that's the only perspective that you're thinking about, like then you're only testing like those for a degree consequences. But at the same time, there's also like an opportunity for you guys to almost like educate people, right? To be like, hey, like this is a real, like conversion is like a really good question. But like, I would also encourage you to think about like long-term, like what are you actually optimizing for? Like, are you trying to like get more users or are you trying to increase revenue? I'm curious, like how do you as a builder yourself, like how do you think about the role that you want to play in that world? Because I think it's like a cool opportunity. I think we're like, we're already doing some of the education. Like for us, it kind of comes through in the comparisons. The example that I tend to give is let's assume you've got two versions of an onboarding flow and you run those two onboarding throws through an experiment and you compare the two together. Now let's assume variant one has got the highest number of conversion predictions. Like, you know, you've run the experiment, you've had five different personas that have completed the site 50 different times. You've got 250, like essentially kind of run through each of the personas, each of the variants. And on version one, you're seeing 230 convert to being a paying customer. And on version two, you're only seeing 50. So for, particularly for a growth team, they're going to look at that and go, well, why on earth would I go with version two? But the big thing that we're spending time on is saying like, well, because version two, those 50. Well, because version two, those 50, they were the ICP. Like we ran it five times on five, we ran it 50 times on five different personas. And of those five personas, 50 of 50 in the ICP category actually converted, no one else converted to the platform. So going back to that question mark of like, should we be building for the ICP? Well, yes, while there's still a one size fits all interface you should be focusing on the ICP. As we move towards dynamic interfaces, as we move towards a world where we can have a different experience for an anxious user versus a different experience for an impatient user, this whole thing changes. And that's something that we're trying to educate on. But for us, when it comes to comparisons, it's like, how do we basically show them that while variant B in this case has a lower level of conversion overall from the user base, it has the highest level of conversion for the ICP. So if you go through the path of releasing this new onboarding flow, you could actually be in a position where you piss off the ICP because they're like, I don't like this onboarding flow, it's not as good. I find it more difficult and I'm more confused about the product. This is quite literally something that happened when we first started building this company. It was working with a company that was trying to do exactly this problem. And what essentially happened is they spent a load of time thinking about conversion and trying to get more converting users from ads that they were spending. So you convert the users, you send the results back to Google, Google then sends you more customers that then convert. And in theory, things look great because you've got a high conversion rate. But the problem is Google's optimized for the wrong type of person, which means that person is actually churning at the six month mark because you haven't spent time thinking about the ICP and working out whether they actually care about the product that you're releasing to them. So we try and do as much education as possible in showing those kind of second and third order magnitude effects that come through when it comes to building products. So interesting, because I used to work in the kind of like general ads, personalized ads part of Facebook. And when we were trying to help advertisers get better outcomes, the lever that made the biggest difference, generally speaking, assuming a baseline of budget and kind of like audience definition was clear, was the creatives. So the idea is that when you could show the right creatives to the right people, that was your best way of kind of increasing conversion. And what I just realized is like, you're describing kind of a world that we're moving into where your landing page on the website or like the interface itself, it just becomes like another type of creative. It's like, you've got these creatives that are out there in the world for like ads. And then in a world where there's a perfect handshake happening between the creative someone saw that drove them to the site, now the site will, there's maybe like thousands of potential variants of how that site could show up to someone based on which creative they come from. And by the way, if this is already the state of the art, like maybe I'm behind a little, but in my mind, we're still, I'm still thinking mostly in a world where like, yeah, maybe you could AB test like dozens of variants of like your header or your hero, but like not the fundamental layout of like the whole website, right? So is that kind of like where, is that where we're going? Is it like the same way I can think about like infinite numbers of creatives, like infinite numbers of user interfaces, each trying to kind of be the right thing for the right person at the right time? Yeah, I think I don't buy into the like, the fully individualized UI. Like, I know that there are people building for that world and like, I would love for that world to exist. I think the reason I don't buy into it is like, us as product people now have an infinite number of versions of products to manage. It just is- Sounds like a nightmare. We're not doing that, we're not doing that. Like, it's just not happening. Like, unless we're going to hand over the keys to AI and it's going to manage it for us, in which case, like we can go sit on a beach and drink Mai Tai, like, it's like, what I do see happening is getting to the point where we will absolutely segment audiences down much better than we currently are and having different creatives for say, like, you know, you might have 25 different versions for landing pages for 25 or more than that, but like, what is manageable for product teams, 25 different versions of that site, which is then optimized for that particular type of persona. That absolutely is the world that we're moving towards. There's no question. Yeah, that makes sense to me. Like, I think, I mean, I'm also like product people and founders are, let's just say control freaks is one way to put it. Like, I think we like to be in the driver's seat, that we like to make the decision. So there's, so I'm trying to hold that idea with this other idea, which is like, it seems like the job is evolving, like this like non-deterministic way of building software is forcing everyone to kind of wrestle with how do you bring your controlling superpowers, right? Because like, these are superpowers or shadows for a lot of people. So it's like, how do you bring these controlling superpowers to creating like an opinionated set of guardrails or constraints that ensure the experiences are happening? So maybe like that's where it's heading. Maybe it's not even a deterministic number of landing pages. Maybe it's still kind of like an infinite number of landing pages that could happen, but you've gotten much more comfortable with like de-risking how it can go off track or like. Yeah, 100%. And I think like that's the thing that's interesting for me is like, we are just seeing this again, like it's like 2004, 2005, when kind of Facebook was first kind of getting going. It's like, essentially they invented Static. Like I believe it was, it kind of comes down to meta to have invented Static and A-B testing. But I think the thing that's happening now is like we're seeing this collapse where like, because we're shipping faster, because we're kind of like testing more creatives and like doing a lot more and creating these dynamic interfaces, like the bet that we're making, and I think it's a fairly fair bet to make now is this stack of like user research to then products, to then initial interface mock-up, to then probably going back to user research, then going to engineering, to then shipping, then feature flagging, to then A-B testing, to then full deployment. That process is probably taking the average company, anything from two to six months for a big product change from that perspective. Little things happen a lot quicker. Now we're shipping faster than ever. Now we're testing more things. And to be honest, we're not testing quick enough. Like I think the big thing that I spend a lot of my time talking about is the shift of like the point of failure has gone from the engineering team that often takes time to build things to some extent the product team who are like, hey, we need to ship fast, but we also need to know that this is actually gonna be the right thing to build for our customers. It's the onus is falling on us as product people again. This whole stack that has been built for the last 20 years is about to be entirely regenerated. And we kind of call it the agentic loop, where it's like, you know, you simulate something, you have a read, you do some research, you confirm it, then you pass it over to Cloud Codecast or whatever you're using from that perspective. You iterate, you build, you develop. But if you can then, if you've got this knowledge, if you've got this kind of directional certainty that or confidence that something is going to happen in the way it's going to happen, thanks to simulation, well, then you don't need to worry so much about maybe testing. You don't need to worry so much about feature flagging because you can push changes quickly. And as long as the system is proactive enough, it's going to be able to make those changes before anything bad really happens. And I think that's the world that I'm excited for, where systems are given a lot more, I guess, control to be able to change as people that are interfacing with them are changing as well. If you're enjoying this conversation, please check out the links in the show notes to support the podcast. Mark and I do this out of love, but to keep it going, we also need your support. Thanks, and now back to the episode. And at what point of the product process do you think people should bring like synthetic users into the picture? Is it like kind of like towards the end of the design, almost review of like, hey, I built a prototype. You know, I already know exactly the problem that I want to build. I think I have a good solution, you know, before maybe I move this production and connect it to like our backend or database. Like, is that the ideal time or is it more of like complicated, like it depends type of answer? I think for us, that's where we play in the sense of like, that's the point where we can really provide value. So, you know, from that perspective, it's like, how do we help you prioritize the 10 different versions of the UI that you've built and you want to go and test them before you get the engineers to actually build it or draw code to build it. But I also think like it can come much earlier in the cycle. Like one of the things that we're kind of testing is like, can we basically allow you to chat with these personas? Can we allow you to ask questions? Can we allow you to kind of like bounce back or like bounce just pen and paper ideas, then turn those pen and paper ideas into something you then go and build out into design and then push out into Figma, et cetera, et cetera. So I think it should be like the minute that you have 200 different ideas that you want to test is the minute you can like essentially test those 200 ideas in a week instead of a year. Yeah, that's really cool. It's interesting. Like when I look at the kind of best product teams that are operating at scale, and I think one of the things that they have that's kind of unique, but in common of these orgs that they have a really good, almost product rituals and process. And where I think there's like someone at the top that has like that intuition and then they have really good like product discussions. Like we had our friend Nikki from Duolingo and she shared like their product review at Duolingo and the fact that they make it open is such a key element of their culture and why their craft is so high. And when I look back at really good product reviews, I think the best product reviews is where you have a lot of data and almost artifacts to react to. And then, because then at that point you can elevate the discussion and you can have actual real like trade-offs. And almost at the same time, you're kind of building that collective intuition of like how different product changes move the needle in different directions. And so I'm almost saying at a point where like, man, if you can have those discussions with executives, be like, hey, like not just react to this design, but here are the trade-offs that we're making with each design. Like this is more beautiful. It's gonna lead people to convert much faster. But in six months, we might see this thing. Like, I just feel like the conversations are gonna get way more strategic and everyone in the team is gonna get smarter. So I think that to me, it's like a really exciting world to live in. I can't tell you the number of customers that we have that are like, we use you because it eliminates the loudest person in the room. There's like, there'll be an executive that's like, this is the way we should build this based on absolutely no data points whatsoever. This is the way we should build things. And like, I'm the CEO. So like, what I say goes in product teams, like, okay, I'm gonna go and do this. Like, it's the wrong decision, but I haven't got the data to be able to prove it. So I'm now gonna have to deploy it, get the data and then go back for the second meeting and be like, hey, that was a waste of time. This is what happened. We've got a bunch of customers that are just like, hey, this is great because now I'm in a position where someone will say, this is the way that we should build something. We can mock that thing up. We can test it on synthetic users. We can see whether this is actually going to be the right thing to deploy to the users. We can then take that information back to said essentially executive team and have meaningful conversations where we're like, this is how we should actually be doing things or this is a good move. But as you say, this is the potential risk. Six months from now we lose customers. So it's just pushing that needle forwards and it's also giving executives insight into a level of product knowledge that I don't think they necessarily had before unless they were involved in the product or to some extent engineering team. So this is super interesting. So what's happening in my head is like, let's imagine that I'm the PM who's, my intuition is telling me that, yeah, we might get higher short-term conversion, but the downstream kind of like retention or the LTV of those customers is gonna be way lower. I have no data to prove it because it's an intuition. I'm going up against an executive who's basically like, so you're saying the conversion is gonna be better, right? And I'm like, yes, but I think there's, so they're like, we should just ship it, right? So let's say that's the scenario. How do I as a PM using a product like Block prevent confirmation bias from coming into the results where I ask the question in a way that's likely to give me the ammunition that I'm kind of already seeking? Yeah. It's a good question. I think like there's no easy answer to that other than saying like for us, there are two ways in which you can design an experiment on the platform. And actually this is really interesting because the way that you've said that, I've always been like, so we have basically a chat interface and we have a form interface. The form interface is like, the reason that we have this is because in, from our experience, if we speak to 10 product people, seven in 10 of them will have an idea of what they want to build, but don't necessarily know how to ask the right question. Three in 10 of them will know exactly what they want to build, exactly the question that they want to ask and exactly who they want to test it on. So that's what the form is for. I've always been pro form because I guess I've always come to the conversation with like, I know exactly what we need to test, like slight level of arrogance to some extent, candidly. The other seven in 10, where you're asking the questions, you're not actually setting the test objective or the question. You're basically allowing us to understand the context in which you are looking to test. And we are then generating the question and the hypothesis, which we're then passing over to the agent to actually complete the task. That's, I guess, how you get through that problem. And it's really interesting because I've never thought of that as a potential use case for the chat, but that would be the way that you get around it. It removes that confirmation bias. But if there's like a, I mean, it's just like really interesting to think through, but if there's a chat, if there's a chat box and I'm like, just dumping into the chat box, it's like, there's this executive, they're being super difficult, they don't believe me, right? Like there's all this stuff. Can you run a simulation that kind of proves that my intuition is right? Like, let's just say I'm on the extreme end of like literally trying to bias the results. Will the results literally come back more biased if I'm overt, like if I'm overtly doing that? They shouldn't. And again, this is why for us it's show, don't tell. The question that the hypothesis that you're setting up is not going to influence the behavioral personas and the synthetic users that you're looking to test on. So like you can ask a really pointed question, but it's gonna go and complete the task. And if it doesn't meet the pointed question, then it's gonna be like, no, you are wrong. We went and ran this experiment and actually this is what happened. And you may have thought that this was gonna increase conversion or increase chance six months from now, but actually we found that users are really happy and this is what we think is actually going to happen. So I think because we're taking the approach of show, don't tell at every step, it's not going to bias the model in order to give you a, like most LLMs, it's not gonna basically be like, oh yeah, you're right. Like we should definitely not do this. So the visual that's coming to mind for me is like I'm imagining kind of like an old school laboratory that's got like an airlock inside of the laboratory and they're doing a bunch of experiments inside the laboratory. You're saying that the user input is kind of almost like a conversation that happens outside of the laboratory. There's kind of like a level of alignment about what makes its way into the laboratory. And by the time that the quote unquote scientist in this case runs the experiments, it makes sure that the experiments are being run in like a scientifically rigorous way that has removed as much bias as humanly or agentically possible, let's say from the equation. Exactly, yeah, yes. And it's not perfect, it's not perfect, but like that's the way that we think. Yeah, like me, who I love to always just find ways to find the loopholes, like kind of the way I would think about it is like, you know, you go read the output and like you go screen by screen, see how the agent reacts. You change that thing to see, okay, like this is probably affecting conversion. You change, like, so you could, in a world you could spend maybe like a day iterating on the design to get it where you want based on learning on how the agent acts, right? But then it's like, at that point, it's like, is that the best use of your time? And then it's like, what are you optimizing for? Right, like, are you trying to build the right thing or are you just trying to like get this standard approval? And I feel like as a product person, usually like just pushing to get that standard approval rarely is the right thing for the customers and for your career long-term. And, but I think that the part that was kind of interesting what you said, that was like, so you said like, what was like three, like three in 10, like kind of have that hypothesis and the seven in 10 don't have that hypothesis. I'm like- Three in 10 know the question, I think that they want to ask. Yeah, yeah. Seven in 10 don't. Yeah, like question or hypothesis or like they have a thesis. And that kind of feels wrong that like just someone built something and they don't really know why they built it. Like, is that, and is that, has that always been the case? Cause I feel like my organization, you know, when I work with user research and data science, like the first thing that they always push you for is like, Hey, like, like, what are you trying to learn here? Like, and like, what is our goals? And like, what are our hypotheses? And then if that's not clear, then it's like, wait, why are we even like, what are we talking about here? So I'm curious, like, yeah. Is that maybe like- I came to the conversation with the same view as you. It's like, well, we started with the form that we had a form that was like, what's the hypothesis? What's the test objective? Who do you want to test it on? Go run the experiment. But the more people that we worked with, the more people like, I know I want to validate hypothesis and I think it's to do with my signup flow, but I don't really know how to ask the question. Or I don't really know what it is that I want to ask about the signup flow. Other than, you know, I want to validate that my signup process is really easy to sign up. Sure, okay, cool. Like that's a, I guess to some extent an objective, but this is where we're trying to remove some of that bias where we're like, okay, cool. If you put words like very easy, it's like, you're not biasing how the agent behaves, but you probably are biasing the output because you're like, is my signup flow very easy? Then you're basically asking the agent, is my signup flow very easy? And if the agent gets stuck, it's going to be like, no, it's not very easy, it's crap. But if it's like, you know, that's where kind of like, so going into that process of getting them to define the question better and define the objective better leads to objectively better reports at the other end, which are then much more focused around like, how do you actually fix these things? What's good, what's bad, et cetera, et cetera. Totally, it's like the quality of the question affects the quality of the answer, right? Like, sorry, Ben, you were going to say something. Yes, that makes a lot of sense to me. I mean, I'm thinking about like, Mark, again, going back to like our homepage, for example, like we've, you know, we ask ourselves a bunch of questions because we're kind of like early. So we're like, do we even have the content that people are wondering about? How are we presenting things in the right sequence on the website? Is there a better way to write our hero? Or, you know, like there's a bunch of these questions. Do we have enough social proof? Right, exactly. Do we have enough social proof? Or if we, and if we don't, like what should the social proof be? So I can see myself maybe being in the bucket of like the seven out of 10, who's kind of just like, can you help me actually come up with like what the right question is to be asking here? Cause I've got like maybe 10 questions I'm thinking through in my head. Tom, is that like why someone might fall into the seven out of 10? It's like that kind of thing where there's like a bunch of variables you're just not sure. Okay. Yeah, it's like, you know, it's something you want to test. You just don't quite know what that thing is. Yeah. And then, okay, so as a follow-up to that, is there like a right, I know Mark asked earlier, like where in the, at what stage in the development process someone might want to think about synthetic users. But if you kind of like take a, if you zoom out from that question for a second, ask like, is there a right, is there such a thing as a team or a company being too early to start using synthetic users? Like for example, like a two founder YC startup barely has any users yet, or maybe a very early traction is that, or people with an idea and just like mocks or prototype versus like IPO, post IPO company, where have you landed on maybe what the, how the stage might play into people's ability to get value out of a product like this? We say post series A. So I would say like candidly, we typically work with larger enterprises. You know, we work with some scale ups, but typically speaking, post series A is where the light value really comes from. I think the reason for that is because pre that you don't have the data. Like we can, sure we can run experiments and we can put Figma flows through and kind of this kind of funky stuff. But if you don't really know who your users are yet, we're basically just biasing your user base from day one. So my view is like, get some users, understand who your customers actually are, and then use that information to properly understand, you know, whether we're building the right person. What if instead of having real user data from like an amplitude or a mixed panel or something, I go out and I do a research sprint and I interview 30 people and I have perfect transcripts of their different personalities and stuff. And I can be like, okay, like forget about these 10 out of the 30, but these 20, I think these 20 are like really the target audience. Can someone take that kind of qualitative data and put it into the system and have you craft these personas? Yeah, and they do. But I think you still need that quantitative data that comes from the analytics to give you the bigger picture of the kind of longer term. But like for us, when we're ingesting data from our. But like for us, when we're ingesting data from our customers, it's a combination of quantitative and qualitative data. The quantitative data gives us a trajectory of like how people are using your product overall, where the concerns are, blah, blah, blah. And then that qualitative data is that thickening kind of agent that allows us to deeply understand more of the like emotional, psychographic, behavioral information that comes through to help us better build out those personas. The two combined is the optimum, but I do still think you need some quant data in order for it to be successful. Is there a way to like bootstrap the psychographic behavioral data without having it come from my own first party data collection? Like is there a way I can get that from somewhere else? Us, so like, you know, the path that we're going down is we're building our own models, which means that we've built out some top down personas where you can have some tried and true psychographic profiles involved in the process. You know, like we've seen people that are like, I think like the kind of elephant in the room, could I just use Claude code to pretend to be a particular type of persona that's really anxious? Blah, blah, blah. Like the answer to that is yeah, a hundred percent you could. The two big problems with that is one, every step it takes, if you're using prompt based personas, every step it takes is kind of defaulting towards the media and the LLM, the kind of the behavior that every LLM has. And then you could say to me, cool, well, I'll re-inject the persona every step. Sure. Re-inject the persona every step. You now have no memory because the persona is essentially being reset every step. So I think like there are ways of doing it at a scrappy way. Like we see some startups trying to use Claude code and stuff like that. But again, I do just genuinely think it's adding a lot of bias to the way in which they're building products. Hmm. Totally. Yeah. Like we could, we could basically create, you know, five agents or five, you know, different skills or agents or however we want to set it up in Claude and have each of them have their own collections of psychographic data that we just plant, right? Like we're going to, we're going to start it. And then just like I have end-to-end tests that run every time we deploy to production, we could have like tests that run through the lens of each of those five personas and tell us if there's any risks that should be flagged to us. And like, are we basically making a trade-off we might not be aware of that favors one persona at the expense of a different persona or something like that? Yeah. But to your point, that will be, it's like every time we're starting from square one by doing that, unless we build in a feedback loop that it, that takes any sort of like, I guess there's no real behavioral signals that we could reflect. It's more about just like updating text files about things that they should know about those personas or things to do. Whereas you're, you're creating more of like a system of record about people's actual behaviors over time. Yeah. And the, you know, the difference is, is you're getting anything from 30 to 60% accuracy with, you know, kind of like existing, like using an LLM versus on average, 87% with us. So, you know, like it's, well, anything under 50%, there's no point doing it. You're better off flipping a coin, like candidly, like, you know, that's just like guess. So 50 to 60, there is some value there, like you can get to 60, but when you're up near 90, like that's actually true value that can guide whether this is operationally viable data that you can use to move the business forward. How correlated are the quality of the results that you get with synthetic users to the quality of the data that you're collecting? Like, because what I'm thinking is like, Hey, let's assume I'm a founder today. Like how obsessed should I be about like how much data I'm collecting, how I'm collecting that data, how I'm structuring that data? Or is that like a silly problem to spend time on? And like, basically like a solve problem that, you know, amplitude and mix panel think about? I think the way in which you collect data has never been more important. And I think that it's really interesting, like most of the companies that we work with, they have these huge data lakes and they don't really know what to do with the data and it's just not being collected in a very good way. So we can still work with it, but it's a lot more work on our side. There will be a next generation analytics platform that does it in a better way. Like I think some of the things I'm hearing from VCs is like, Oh, why are you like, this is a super interesting path, but another really interesting path would be like, could you build out a proper CDP that allows you to like, you know, better predict behavior from that. It's like, yeah, you could, you could go into sales. Like there's a reason that we're going down the path of building our own models. We're building our own models because for us product is tried and true. It gives us trajectory data. It gives us backtesting data, which allows us to prove that our models actually work. And it also gives us eval data that we can use to improve our models. Once those models exist, you could use it in sales. You could use it in marketing. We're already getting point by for those industries. You could use it in engineering. You could use it in infosec for like preventing, you know, kind of like looking at human pen testing and by human pen testing. I mean like how humans behave to penetration and hacks and stuff like that, which is becoming more and more important. All of these things, people can build, build products around them, which is why we've decided to go down the model path. Yeah. So what I'm, just to make sure I'm clear on that last point, what you're saying is some of the conversations that you've been having with VCs or folks that maybe push your thinking a little bit, maybe let's say they're trying to embolden you to get, to bite off more stuff. Is that they're saying, Hey, like your entire, if the way that the data is collected is still today being done in a way that we were collecting data in a pre AI world from like a few years ago, but the way that we need to use the data is ahead of the way that we're collecting the data. Someone needs to come with like almost like a way of collecting data that is redesigned from the ground up to make the data more useful in the types of simulations that we're talking about. Yeah, exactly that. What does that mean? Like what, what is what, like in my head, I, my mental models, there's like users and there's sessions and there's events and there's properties like are, is the, I'm sure you have lots of thoughts on this, but like in a world where we've redesigned the way we collect data is, do we no longer have like users, sessions, events, properties, like those kinds of things? I think we have all of those things. I think we collect them better, like we connect them together better. I think we find a better way of building out a fuller picture of a human. I think like the, and like the reason that I'm waiting for someone to do it as opposed to going and building it is like, there are lots of question marks around privacy and data in that world. And I would prefer to work with the data that I need in a way that can be anonymized out and built. So like synthetic data is the lovely thing about synthetic data is it's synthetic data. Like there's no PII and stuff like that, which is quite nice. I think it's an incredibly interesting, like building a next gen analytics tool is incredibly interesting, but incredibly difficult to achieve in a way that still allows you to have a level of safeguarding around yours, mine, everyone else's data. Is it, is a, is a synthetic user the way you guys define it today? Like if one of your companies has a synthetic user, should I be thinking about that synthetic user as having like their own unique user ID? Like is that something that persists over time as like an actual identity? It's just not tied to a human. No, it will, we're not quite going down the path of building digital twins. We are basically, well, you kind of are like, like what the user sees at the end of the day. No, basically we're building out personas that will represent a number of your, your real users. So we're taking the approach that going to this end of one and building out personalized interfaces is not the right path. And for us, it's identifying clusters of users that have a similar set of behaviors and then turning those similar sets of behaviors into personas. Got it. So personas being abstractions of, of users, like groups of people, got it, got it. Yeah. Which is more true sometimes than like individual. It's like sometimes the abstraction is actually more true than the individual instance of the thing. Yeah. Yeah. A hundred percent. Go ahead, Mark. No, I think that I was very curious about too. Especially because Ben and I have been thinking a lot about like social proof as well. I'm going back to the example, which by the way, I think it's such a compelling example, like the PM getting like friction with an exec or someone more senior. Cause I think like you waste a lot of time waiting for an A-B test, but like, unless you're like an incredibly skilled, like diplomat and have amazing soft skills, you can also waste a lot of time getting that internal approval. And I think that actually is the thing that really burns people out as opposed to like just waiting for results. Cause like, I don't think very few people enjoy doing that. But I'm curious, how do you build trust with like that, like skeptical, like stakeholder that's like, has a lot of capital power to be like, Hey, like, why should I trust this tool? Right. That like, that, that, you know, just like, is this new and hasn't, hasn't been running a lot long enough that like, like it's just, was just implemented, you know, like we can not tell like what happened six months ago versus my intuition, which I've been, you know, like building companies and teams for like 30 years and you know, I was around before the internet. Right. I'm like, I'm, I'm a legend. Like, yeah. How do you argue with that person? Maybe like help them see the light. That's why like product is such a nice space to be building in it. It comes down to the backtesting and it comes down to the fact that you've already got A-B tests that you run on real human beings and we can build out personas and you can run A-B tests on us and we can compare the two side by side and we can see the Delta between the two. And, you know, when you've done that a couple of times and shown that you're within the realms of kind of five, 10, 15% difference, it becomes a no brainer because your average A-B test is taking anything as I say, from kind of like, I don't know, two to six months to like fully run out. Or like the average process is taking two to six months to say the average A-B test takes two to three weeks. I can now say to you, cool, you can wait two to three weeks to get this result from human beings from your real users or you can wait 12 minutes and we can give you a pretty good directional confidence that this is what's going to happen. And in those 12 minutes, in an hour, we can make five changes and those five changes would have taken you 10 weeks. Like that's the kind of like the benefit that they see. What is the exact, what is the pushback to that? It seems like when it's framed that way, it's kind of such a no brainer to get a good amount of leading indication or early data, simulation data in 12 minutes versus waiting two to three weeks. So like, is there a compelling argument for nah, like let's do the, like, let's not do that. I'm just trying to, I'm trying to put myself in the head of the naysayers internally that have been effective naysayers, right? Like what's the pushback? The one that you sometimes hear will be like, well, I still want to test it on real human beings. Like, you know, is, is like, this is kind of like a, you know, like, but we have to test it on real human beings. I don't disagree. I'm not saying let's stop A-B testing yet. Like candidly, I am saying that in the longer term, but like, you know, like, like I'm not, I'm not saying that right now. Like there needs to be some more education before we can get there. But what I am saying is like, it doesn't make, like use A-B testing for the final 10%. Don't use it for the 90%. Like that's the kind of like, and I think, you know, as much as I love the fact that feature flagging exists, it's the only reason we feature flag is because we don't trust the results. Like we don't trust that the thing we built is actually going to be good. So like, if we get to a world where we can have enough confidence that something is going to have a positive impact, then we don't need to A-B test. And again, I'm not saying let's get rid of it tomorrow. And we also don't need to feature flag. So that's two markets that felt like, because they're both like reactive, they're not proactive. I think the biggest thing that's happening in the product world right now is we're suddenly being given the tools that allow us to be proactive as opposed to reactive. And that's the biggest change that's happened. Like you 2005 is like, we need something that's reactive, pro reactive is going to allow us to make the changes, it's going to make sure that this doesn't happen again in the future. And now we're saying, Hey, well, why don't we just be proactive, cut those three, six weeks down into hours, and make sure that we're actually building the right thing for the right person from day one. And yeah, we're not going to be right 100% of the time. But because we can ship so fast now, even if we don't get it right, as long as there's something that's proactively monitoring, you can make those changes really quickly and ship the change that you need to ship within a couple of hours. Mark, I have one last question, which maybe we can cover quick. And then we can we can we shift into wrapping up? Or did you have a follow up? Can I just get a really quick just to make sure on that, because I'm really curious about social proofs. So in this sales, let's say you're trying to get a bring a company on board. It's basically what you're doing. Like you're like, Hey, like, give me a historical A-B test. Don't tell us the results. And basically, like, we'll run it in front of you. And that's how you basically that's how you prove it. Basically, is that basically what you're doing? Yeah, we're doing it with like a big company at the moment. And they were like, we're running some A-B tests. I'm like, we'll run alongside them. We'll see what happens. And, you know, I've seen instances now where like, they'll wait three weeks and they'll come back to us. And they'll be like, you told us this three weeks ago. Yeah, I know. Basically, your sales cycles is dependent on how quickly the big companies are running. I love that. I love that. My quick maybe it's not if it's not quick, let's skip it. But like you said, maybe in this world, we might not need feature flags anymore. I'm a big believer that like feature flags that linger are awful. And like, we should not have lingering feature flags. But there's a use case for like, feature flags for like controlling, you know, let's say, like, which customers get early access to something before it goes out to everyone. You're not saying we shouldn't have feature flags for that stuff, right? I just wanted to be clear. I'm actually saying feature flagging changes and is kind of the thing that is going to power adaptive UI. So it's kind of like as we move towards this world, or we have different interfaces for different types of people, some new gen feature flagging tool be the thing that we use to actually manage it and roll it out. So it's gonna be more of like, instead of feature flags, it'll be something that adaptive adapts the UI to the psychographic and all the other persona related kind of attributes of a given customer, like they're in this kind of tier of company or whatever, in this region. So they should see this kind of feature, but it won't be controlled with feature flags. Got it. That's my expectation. Yeah. Cool, cool, cool. And again, I think it feeds into the theme of like proactive versus reactive, right? It's like, you're not flagging it because you're not confident that this might be bad is more of like, you just want different experiences. And you have confidence. Yes, totally. Awesome. Well, this has been amazing. I wish we could keep going. But I know we're coming up on time. So two quick questions. And then Mark has a question he'll ask. So, Tom, where can people follow what you're thinking about and learn more about what you're doing these days? And how can the audience potentially be helpful to you? So I'm notoriously bad at posting online. But like, the only place you're really going to find anything that I'm saying is probably LinkedIn, but obviously like the website. So like, yeah, joinblock.co. I'll be candid and say we're rebranding. I hate the URL, but it's got us to where we are today. So that will be changing. But yeah, joinblock.co is where everything that's happening from the kind of company perspective right now. Amazing. And can any way that people can be useful to you? Just check those links out? Yeah, 100%. Honestly, more than anything, let us help you. It's a kind of like, you know, I really, at the end of the day, like, yes, it moves our business forward. And like, yes, it's sales and like all that kind of side of things. Fundamentally, like we want to be able to prove to the world that this can be done. It's the biggest thing that people can do to support us is really kind of try the products and see the value that we can deliver to teams in and show how much easier it can be to run prototypes through to live deployment in this world that we're moving towards. Nice. You're waiting for three weeks of A-B tests. Yeah, hit me up. Yeah. Or if you have an exec that's by projecting your beautiful product. Yeah, yeah. This is for you. I love that. Amazing. Tom, you've had such an incredible career, multiple-time founder. I think you're building a very consequential product. And you have an awesome team that I've got to meet parts of them. And yeah, maybe let's take a moment to maybe call someone out or a couple people that have played a key role in your journey. And yeah, who comes to mind for you? So many people. I mean, the team, obviously my co-founder, Olivia, like this is the third company that we built together. I've never been so confident. And I hope this doesn't bite me in the ass by saying this, but I've never been so confident in what we're building. You know, in previous companies that we've built, they have been really exciting companies, but not as exciting as the one that we're building today. But I would never have been able to do any of the things that I'm doing today without her. I think from that perspective, she is the person that keeps the ship moving while I get to have conversations like this. So from that perspective, by and large, Olivia, my co-founder, equally like the rest of the team, we're a high-stress environment. That's what we're building. It's a startup. And the work and effort that the team put in to try and make this a reality is incredible. It's the best team that I've ever worked with. But then I think it would also be unfair not to shout out a couple of extra people. Some of our investors, like Hannah from Blank, Ashkan from Oddbird, Marlon from Mac, they're people that have, by and large, gone above and beyond to support us and shout our praises, but also to be able to go to Kevin and Brendan from Rackhouse and ask questions that are deeply technical or things where I'm not sure about something. Kevin created surge pricing for his sins, for Uber, but obviously also deeply understands human behavior. These are some of our early investors who have been incredibly helpful. And I've raised money in the past. I've built companies in the past, as you mentioned. I've never had investors that have been so supportive and thoughtful of the company that we're building today. Amazing. Wonderful. We'll link everyone in the show notes. Sweet. Yeah. And I just want to say thank you for building what you're building. I'm not actively in a full-time PM seat right now, but I think about so many instances in the past and what you're building would have been incredibly helpful and just made going to work more engaging and intellectually interesting for me and focused me on less BS at work and focused me on doing good work. So I'm really rooting for you. And we'd love to have you back on maybe post-rebrand and talk about the new direction. Yeah. Honestly, I'm selfishly building this for myself. Like this is something that I wish existed when I was getting frustrated with the experimentation process and fed up of people with big opinions, but little data to show for it. Like for me, I want this to be able to, again, show don't tell, show me the data, show me that this is what people are actually going to do. And I'll believe you every day of the week. Amazing. Yeah. I'm excited. When you talk to people like you and you see products like blog, I think you get more optimistic about where the product and the PM role is going. So thank you for that. This has been an awesome conversation. And yeah, that's a wrap. Thank you both.