Pavel Samsonov - AI Can Build the Solution... You Still Have to Design the Problem

it is sufficient for product to have an idea and for the engineers to go and build it. I call this the double square model. That's how most companies run, as opposed to the double diamond. There's no divergence or convergence. It's just I have this one idea, go and do it. We did it. Cool. Here's your next idea. That makes terrible products. The majority of product work is a problem that we would like our customers to have, because that's the problem our feature solves. This is where a lot of AI companies are at, where they built a thing that solves a problem that no one actually has. And now they're trying to convince people that, hey, you should use this anyway. Hello, and welcome to One Night in Product, the show where I chat to some of the brightest minds in and around product to help you see product management in a whole new light. If that sounds up your street, don't forget to dive into the back catalog on your favorite podcast app or on YouTube. And of course, follow, share or drop me a comment or review. It all helps keep the lights on. On this episode, I'm speaking to Pavel Samsonov. Pavel's a human native designer, writer and speaker who says he's the only UX designer who's ever used Microsoft Windows on purpose. So if this interview crashes at any point, we'll know exactly who to blame. Alongside his day job as a principal UX designer, he's also regularly posting insightful, thought-provoking commentary to his adoring social media followers, as well as writing his newsletter, The Product Picnic, where he set up a little blanket, bought a selection of cold cuts and a bottle of warm wine, and inviting us to sit down and talk about connecting today's product discourse with classic insights that the industry has already forgotten. I don't know about you, but I'm already feeling hungry. Pavel, thanks for coming. And welcome to the show. Thanks for having me. It's good to have you here. And as I said, just before we started recording, I've not done like a UX flavored interview for a little while now. So I'm looking forward to sort of digging in, seeing what's going on in the world and what we should all be thinking about differently. But before we do any of that stuff, obviously, you and I have been sporadically kind of going back and forth on the socials for a little while now. But for those who don't know you, what are you doing for a day job at the moment? I know you used to work at AWS, for example, but what's filling your time at the moment? Yeah, so for folks who don't know my background, I've bounced back between products and UX periodically. Currently, I'm on the UX side of things. And when I say that, I mean service design, which is kind of a strange animal in that sense. It's really a combination of systems design, org design, as well as a lot of internal and external tooling. So I manage JustWorks right now, which is HR for small business, essentially, except that the majority of the things we do, we have to do kind of as a service organization as well as a product organization. So really, my work bridges together what can users click on an interface with what does then happen kind of in the in the world of atoms rather than pixels? What do actual people have to go and do? And where do they have to queue with the government to get the form to do the other form that then comes back to you as a little green checkmark on your screen. So it's, it's very interesting. I work quite closely with product and engineering, obviously, but also a lot of operations teams. And it's, it's kind of having one foot in a different world from your other foot in a lot of different ways. But that's also the type of software that if I were being kind of unkind, I might say, based on personal experience could be quite lagging in kind of UX terms like this kind of historically, almost like you might consider back office sorts of things that kind of just just throw a few things in there and just hope that it works. And, you know, maybe it just takes 19 clicks to do one thing. And it all kind of, yeah, I've got friends that have been looking at government websites, and which obviously yours isn't, but like, it's kind of that same sort of, you kind of go in there, and it's just not quite right. I'm assuming that you're going to come and say that yours is quite right. But is there a tension between trying to get features into that sort of thing and trying to make it kind of, you know, good and, and actually delightful to use? So that is, it's funny that you bring up government websites, because I actually think that the UK is a fantastic example of government websites done very well. And especially in terms of content design, the is that the UK digital service has like, led the world. So credit where credit is due. The UK has done an incredible job with doing government websites and reducing what is very complex, regular forms and things like that, to something that is very easy to understand, or at least easier than it used to be. And I think that is really the key that makes this kind of B2B design so difficult. It's because the work is complex. I see a lot of job descriptions that ask for consumer grade experience design, hiring for a B2B role. And I think that's hilarious, because first of all, what other industry is there where consumer grade is the good grade? Absolutely no other industry, everywhere else in the world, industrial grade, you know, military grade. Those are all hallmarks of here's a thing that doesn't break consumer grade is plastic garbage. And so for for us to start coming out and saying consumer grade is really good. Well, it's the plastic garbage of software. In a way, it's really easy to build a good and simple and clean interface when you have nothing going on, when the entire thing is the app. But the further you get out of that, the more the app has to interact with the real world, the real world is complex. And so that complexity comes into the app. And I think that designers who misunderstood the past 40 years of Apple think that Oh, symbol, I need to make it simple. No, you do not need to make it simple, you need to make it understandable. You need to show the complexity to the user in a way that they can actually understand it, control it. And you need to show it to them in a way that they care about. Yeah, there's a lot of terrible B2B design that doesn't do that. And to me, essentially, every single dashboard is that way. Everyone loves to have a dashboard. When I say everyone, I mean, stakeholders, stakeholders will say we have a serious product. That means it needs to have a dashboard. And what do you put on the dashboard? Well, they don't actually know that, because there's no purpose to which that dashboard is created, except to have a dashboard. And so they put a bunch of data on there. They're like, we're data driven now. But they don't do anything with that data, just look at it. And then that's it, like, let me go and do whatever they wanted to do in the first place. But at least they have this dashboard. And so, you know, I talk a lot about how do you actually make our dashboard actionable? Well, if you had to act on the information that you saw, there'd be a lot less of it. Because it has to be opinionated, it has to be about what, like, you see one thing, because that is the one thing you need to see. You don't see anything you don't need to see, you don't see anything that's not actionable for you. And in a way, consumer oriented design has it really easy, because everything you see is relevant to you, because you have an quote, unquote, organization of one person. And even then, they fail. This was covered, I covered this in one of my more recent techniques, where a lot of companies now are swinging back into kind of like quantified cell phones and tracking and all those things. And none of that data is useful. It's just it's not medically speaking. Doctors say we don't use any of this, we don't care. We have our own measurements at our own devices, like cool, your heart rate, great. It doesn't, it's not actually helpful. So a lot of these products, they essentially sell your own data back to you. And then you don't do anything with that data, except use it as a comforting blanket to be like, Oh, I have so much visibility into what's going on. But then you don't do anything with it. And so a lot of what adjust works we're trying to do, and I think are successful to a significant degree, is not show you the things that you don't care about. A big part of that is because the focus is on the service organization. And the software obviously is there, but it's not the sort of the sole focus of the company, we're able to just say, this thing's complicated. This is a thing that you will handle on the phone, or over email with an expert who sits at a desk, and is plugging away at doing this work. And you can just call them or email them and you can tell them all that you need them to know. In like whatever format you have, there's no form, you want to need to fill anything out. And obviously, the line between that and self service is always shifting self service is always popular as kind of a pitch, because it creates this impression that you can scale indefinitely. In reality, self service is one of the least scalable things there is, unless you nail it really well, because there's so much limitation to what you can do. As soon as you add even one human touchpoint, you actually expand the amount of things that can be quote unquote self serviced by a huge amount, because you're suddenly capturing all those exceptions, you don't have to, you don't have to pre design kind of interfaces that account for all of them. And you're always going to have that issue. And the biggest issue that we face then is not how do we make more things self serviceable, but it's more how do you seamlessly transition between self service and full service? How do you dip into a human assisted workflow and dip out of that workflow as soon as you have what you need really, really smoothly, and actually wrote about that on on our tech blog. For anyone who is interested, you can look up the just works tech blog, and this was out, I think in February. But to me, it's a, it's a very different way of doing design. And the way that design owns, and ownership is, I think a really controversial part of the way we talk about the industry today. Because notionally speaking, the product is owned by a product manager. Like that's the job. That's what they're there for. I mean, some places call them product owners. And I think if we were to debate the difference between a PO and PM would be here for probably another day or two. Yeah, let's do a different episode on that one. But sort of my whole thesis there is that product management has a very vertical approach to ownership, where everyone owns, like the product triad owns a touchpoint or a domain, and they're really good at making sure everything in that domain works from a vertical point of view that like the backend, the frontend are talking to each other, and the customer can access the frontend, and the organization can access the backend and so forth. And you have these domains kind of stacked in a sequence, and we call that a workflow, except customers don't really care how we've organized our internal teams. To them, it's all one thing. And depending on how good those silos are at talking to each other, you can have a lot of seams in between those parts of the experience, because no one owns them. No one is like, oh, I need to take care of this. They have all their own metrics and all their own data coming in. And it can look really good on paper, but then put together, it's kind of not a very good experience. I think also government is an excellent example of when that doesn't work so well. And the way that UX has its opportunity for influence, I think is not so much fighting over the ownership of those vertical slices. But it's really easy to fight over the vertical slices, because they're so salient, they're so obvious. Someone goes and says, here is my roadmap, here is these decisions that I have made. And once someone has made a decision, it's really easy to say, I disagree with your decision. I would do this differently. But then you're always missing those gaps in between, where no one has even noticed that a decision has to be made. And so it's kind of pushed off and off and off. And it becomes kind of a decision made by default, where either no one has even touched it, and it's kind of just emergent from the system itself. Or it's fallen on some developer who is working on their 20th year of the sprint to figure out what makes sense here. And they kind of make a snap decision based on whatever's fastest. And your experience sucks. And you don't notice because you're only doing essentially unit testing of your one slice. And your one slice is great. And so one of the opportunities I think designers have, if they don't want to spend all of their days in meetings fighting product managers, or who gets to do product management, is to look at that end to end, holistic experience and point out those opportunities for well, no one owns this because no one is thinking about it, I'm going to fix it. And then I'm going to go to the kind of the people on the edges of that decision and say, hey, I already did this. And you just plug it into to what you're doing. And I think it's a much more effective way of increasing your scope. Because it doesn't step on anyone else's feet. No one else is thinking about that stuff. This almost feels in the like, sort of UX design, or at least design of organizational design versus any kind of systems design. I know you talk a lot about sort of systems thinking and stuff as well, this idea of, of like, trying to have oversight over the system versus because like, yes, it does kind of devolve into silos quite a lot even between even just if you consider product managers, you know, you kind of have a product manager looking after this bit and a product manager looking after this bit. As you say, they've all got their own different metrics and OKRs and this all been sort of incentivized just to hit those and not necessarily think about the greater good. Now if you're going to step back a little bit to the wider company and see that, well, even outside of the product and design space or sphere or whatever, you've got then sort of commercial parts of the organization that may as well be talking a different language and also have their own goals and targets and such and kind of feels though then that what you're talking about with the opportunity for designers is that's important, but it's also incredibly difficult to do well and could potentially have a lot of resistance from just about everyone in the organization. For example, I'm a consultant, I go in and talk to companies about this stuff and try and do some of the stuff that you've just said, try and make companies better. Now, even when people, even when the CEO brings you in and pays you a chunk of money to do that, you still get resistance or doubt from different parts of the organization, either status quo bias or they don't trust it or they don't think it can change or whatever. So I imagine that whilst what you've just said around kind of the systems lens on stuff has a lot of promise that there could equally be a lot of resistance or gravity stopping designers from doing that. Oh, absolutely. And on the internal process side, which obviously is my bread and butter, you kind of, you know, in companies that understand this, you have program manager roles and the program manager is essentially a product manager for process and they go and they herd cats and they get all of these different silos talking to each other. And I actually think that designers partnering with program managers to do some of this stuff could move a lot of processes that currently feel sort of set in stone where everyone knows they don't work. Everyone knows that you have to work around it to get things done, but no one can sort of muster a coherent response across the organization in order to actually get things moving. And so that's why I'm speaking specifically about product decisions. Like you go, you go to two different teams that are adjacent to one another and you kind of get them to talk to each other. It's a small scale thing. I think designers are always tempted to come in with like, here's the ideal future state. Here's what the perfect world would look like. It's just like, okay, that's valuable. That's a really important directional kind of point of view to have, but how do we get there? Uh, you can't boil the ocean. Certainly you can't boil it by yourself, but you're not in the middle to boil it with together with everyone else, unless you have that kind of coordination. And so starting small is really, really important. And I think also just helping other people understand how they're affecting other teams is really, really effective. Um, like we were talking before we started recording, I personally believe that everyone in your organization wants to build good products. And I think that's a very controversial thing to say, surprisingly enough, in a lot of discourses where designers blame PMs, PMs blame engineers, engineers blame everyone else. Um, and everyone thinks that they're the only one who wants to do good work, which is obviously ridiculous. But part of the reason that everyone's talking past each other is because they're kind of sitting in their own silo with their own set of requirements. And this is why I don't like the sort of the triad approach. Um, when it's articulated as well, the PM is responsible for, for viable and for designers as possible for desirable and engineers as possible for feasible, because what you've just done is you've separated a team into three silos and you have three silos, each with one person in it, which is the stupidest thing you could have done because the point of the triad is to bring those three domains together into one. And then you have obviously the upstream and downstream of those tribes. And this is why I think good program management is so important. And without it, you kind of, all you have is service design because teams, generally speaking, do not care about what they send to downline. Whoever's downstream of you, they get what they get and they deal with it. And a lot of productivity hacks, quote unquote, and especially in this era of cheap AI, it's really easy when you're not the one using the output to kind of just cobble something together with a prompt or two and scan it and say, this looks good, this looks fine. And then you just email it to someone else and then you never see it again. And so you actually have no idea whether or not it's fine. And it actually turns out that where you save five minutes, it's cost someone downstream of your five hours to go back and find where the problem came from, find that it came from this email that has a human name on it, but it's actually written by a robot, find in that email where the problem is, solve that problem and then go back. And then essentially they're doing your job as well as doing their job. And they're doing a bunch of like detective work on top of that. And so one of the things I did a lot at Amazon, this was a professional service role. So it was kind of notionally similar to what consultants would do, is start asking teams about inputs and outputs. So we had obviously the whole like empathy map persona thing with the four quadrants to which I added two more. And so I guess they're not really quadrants anymore, but I added one for inputs and one for outputs. So this person, this persona, this role, what do they depend on for their inputs? And who depends on them for their outputs? So their outputs become someone else's inputs and then someone else's outputs are their inputs. And then talking about what makes those things fit for purpose actually unlocks a lot of aha moments. If people don't think about it that way, people are like, well, it's done because I finished it. I clicked send, I shipped the software, therefore it's done. Just like the definition of done is an obsession of Scrum, but no one never talks about a definition of good, which is what is actually, what is good enough? What is fit for purpose? What is suitable so that the person receiving it from you can actually use it to achieve their goal. And you kind of just shift around a lot of labor that way, where it's easier for you because it's now you've made it harder for someone else, but you can't see it. It doesn't show up in your metrics. And maybe it doesn't show up in their metrics either because you have a person absorbing that cost themselves on their own time before everyone else can do their job. And, or they can push it down the line, especially if they have organizational power to shift the blame as well as the work. Because shifting the blame is much easier than shifting the work. You can just not do it. Then you can say, well, it's not my fault. And I think that is very common now where executives have these big ideas that really have no substance to them. It's just at the beginning and end of it is let's add AI to our product. And then, you know, after six months or whatever of building this thing, everyone hates it. And the blame always falls on the people delivering the work. They say, well, my idea was great. The problem was the execution. You need to be more productive. You need to work faster. But the problem was never the speed. The problem was that your definition of what they should be doing is meaningless. There was never anything there. Or, you know, pointing them completely in the wrong direction. I used to hear this all the time. Users want personalization. That is code for we don't know what to build. So we're just gonna build everything. And then users will go into this big grab bag and make their own design decisions. And you're never going to be able to build a good product without having an opinion of what it is supposed to do. No one is going to customize it. I don't know off the top of my head the exact stats on how many defaults were made unchanged, but I think it's something like in the 80 to 90% region that outside of extremely specialized workplaces, defaults are gonna stay that way. And it doesn't matter that it's customizable. No one's gonna end up using all those features you put in. Effectively, you could have saved yourself a ton of time and a ton of work by doing a bit of research into what those defaults should be and then only build the defaults and then build around the edges when there's actual real use cases for those additional options. It's very common to come up with fake use cases and be like, well, users might want to X, Y, Z. Well, no, might is not good enough. Everyone's talking about efficiency and return on investment. How are you gonna place a bet with a million bucks behind it on might without doing a hundred dollars worth of research? But research is slow power. That's the problem. It's much better to start building stuff straight away and then find out it doesn't work down the line than take a little bit up front, right? But you talk a bit about problem design as part of your ethos as well. Seems to touch on some of this as well, this idea about effectively trying to work out, I think exactly as you say, what do people really need or want to achieve versus just going straight to the solution. So I don't know if problem design is like your specific framework that you've come up with or it's just a kind of philosophical model, but why don't you talk us through a bit about like how someone should go and solve that problem through this kind of problem design lens. A problem design is just something I started calling myself as a joke, mostly because I think we were in yet another cycle of we must not call ourselves product designers. It's like something else. So you're a problem designer now. Exactly, and I'm like problem designer. First of all, it's funny because it can be misinterpreted as a designer who is a problem, which is fair, but also it's really about the fact that within the framing of the problem is 90% of your solution. And if you haven't taken the time to actually think about what you're trying to solve, you're gonna end up with a ton of waste. You're gonna end up with essentially widgets and toys that don't roll up into anything. And I've written about this. I've called this the form factor trap where we put so much of an emphasis on ideas in this industry. We say, oh, product managers live and die by their ideas. They have really good ideas. You kind of go from idea to execution. And where do ideas come from? And no product management framework really thinks about that. If you look at, for example, Agile and Scrum and all those things, what to do, the requirements seem to sort of appear fully formed at the very beginning of the world. And they change later also for some reason. And the entire focus of the initial Agile manifesto is, oh, the requirements are just gonna appear and then one day they're gonna change. And when they change, we must adapt. But there's no notion of what creates those requirements and then why do they change? Because it's not that users suddenly want a new thing. Users typically want the same thing. They have enduring needs. And a big part of the Amazon way of thinking about products is you need to build for enduring needs so that you can make a bunch of money today, but also like 10 years from now, 20 years from now. So when you're talking about solving real problems, you're talking about where do ideas come from? And if you're not thinking about this from a problem design perspective, your ideas come from using other products and seeing their widgets and being like, oh, this is cool. We should add this to our product. Like let's add this thing because that's how Amazon does it. Let's add this thing because it's how Meta does it. I had a manager very early on in my career who said, hey, can we make our pop-ups in the software full screen like Slack? He couldn't explain to me why, like why is that better? Well, it's how Slack does it. And Slack had just didn't come out fairly recently and it was kind of a big thing. And so this fiddling around with a form factor it's essentially a bunch of wasted work and you can put it in your portfolio and it'll look nice, but it has no really added value to the actual software. And so I always try to kind of work with my product partners when they come to me with a form factor rather than say that's stupid. It is stupid, but if you just tell them that they're not gonna ask you that stuff. They're just gonna go ahead and ship it because that's one of the unfortunate things about user experience is you're kind of the most optional part of the triad. It is sufficient for product to have an idea and then for the engineers to go and build it. I call this the double square model. And that's how most companies run as opposed to the double diamond. There's no divergence or convergence. It's just a half this one idea, go and do it. You did it, cool, here's your next idea. That makes terrible products, but you can do it. You can make a career out of doing it. And so for UX, you kind of have to be a little bit gentle and a little bit subtle when helping people think about what they're doing in a broader sense. And so I like to walk people kind of up a ladder of, well, here's this form factor that you've given me. Okay, cool. It's easy to find code something these days that sort of works and sort of does what you want it to do. Now, what user behavior are you trying to support? And is this the only idea that you've had about how to support that user behavior? If yes, go back and try some other things. There might be better ways of getting the user to do the thing you want them to do. But at the same time, you should ask yourself, why do you want the user to do that thing in the first place? What outcome is this actually even connected to? Are you even at the right touch point of your service to provide that outcome? Maybe the user behavior needs to be enabled in a completely different part of the service, further upstream of where this problem is manifesting itself to prevent the problem from happening in the first place. And so now you're looking at what touch point do I need to engage with to solve the problem? And suddenly it doesn't actually matter what the form factor is, because you haven't answered the fundamental question of why are we building something in the first place? Why are we spending money on a thing? Why are we spending our opportunity cost on building this thing and not literally anything else? And then ultimately you kind of get to the point of, is this the right problem to solve? Because, and this is kind of coming from sales as much as coming from UX and product. People have lots of problems. I have a hundred problems right now that I can see sitting at my desk. But I'm not solving any of them, I'm talking to you. And that's because solving those problems isn't actually worth that much to me. Or it's worthless to me. Yeah, yeah, yeah, exactly. It's worth less to me than other things. If someone happens to come by and solve the problem for me, they'll be like, sure, but I'm not gonna go out of my way to do it. And so really that's what it all comes down to is, are you solving your user's most important problem or just a problem that they have or just a problem that you imagine they might have? And I think you'll find that the majority of product work is on those third things. It's a problem that we would like our customers to have because that's the problem our feature solves. And this is where a lot of AI companies are at where they built a thing that solves a problem that no one actually has. And now they're trying to convince people that, hey, you should use this anyway, that you might have this problem or a similar problem. And they're really struggling with getting paying customers because getting something for free, you're like, cool, sure, I'll play around with it. But then when someone asks, how much is this worth to you? The answer is $0. That does remind me a little bit of your earlier point around sort of service versus self-service or kind of curated or white glove, like what some people might call non-scalable solutions. I know obviously you have your own opinions on that, but this idea that, and this is something I'll be kind of rolling around my head a bit because a couple of recent clients have had this or been on this journey, this idea that just because it makes sense from a business perspective for it to be self-serve because, you know, scalable, hockey sticks, all of that stuff. If your customers don't want it, if they don't want to self-serve it for whatever reason, either it's too complicated or they've not got enough time or you can't make it better enough that it's worth them doing it like that. You can't, just because it's a good business idea, you can't make them want a thing. And then to the AI point as well about exactly as you say, yes, people are perfectly happy to play with stuff, but if it's not solving a problem for them in a way that actually makes their lives better, then what's the point? But do you have some kind of canvas or you talked about a ladder, but like, is there some kind of approach or let's say framework where you can start getting people to take that step back? Because like everything you just said, completely agree with, obviously. I mean, that's probably anyone that works in product properly is going to agree with most or all of what you just said. But is there like a, I guess it's a two-parter. Is there a framework to kind of get people thinking about that that you'd like to use? And also, is there a framework that works with people like, for example, the sales team that you just mentioned who don't think about things like that because they think about, well, we need the thing to close the deal or we need to fix this because of the one customer that said it or whatever. So how to get it working in a product team, but also then how to kind of inculcate that around an entire organization that maybe doesn't get it yet. Yeah, so the framework that I've kind of come up with that I've been using for this, and I say this very lightly because I personally am not a huge framework or Canvas person. My belief is that if you know what you want, you will easily be able to create the tools that fit your needs. If you start with the tool, the tool will determine your needs for you and you're not gonna end up in necessarily the place that you'd want to end up in. But there's a pyramid that I've been using. I think if you just search, Pavel says a form factor trap, I have an article in Medium with this pyramid of how do you ladder up from a potential form factor expression of a solution all the way up to what is the actual problem that we're solving. And last summer I published a course on Nielsen Norman that deals with a lot of this stuff. It is very much UX centered, but it kind of goes a little bit into what that pyramid looks like and how to ratchet up it. And I think that's probably something I'm gonna expand on in the future. But in terms of how to actually work with your colleagues to get that point across, I actually have a workshop that I'm doing on the 31st in Philadelphia with- Oh, a live workshop. How old fashioned. With ThruLine. So you can look that up if you're in town. It's a full day. I'm doing it with Sarah Wachter-Boettcher. She is really great. She's part of the duo that funds the ThruLine conference. So she's doing one on influence mapping, and then I'm doing one on what I call the undercover process, which is, I mean, I'm a huge adherent to design process. I've spoken on it many times. As I've said, though, you should not serve the process. The process should serve you. And so- 100%, awesome. Doing that the opposite way is why, when you say process, everyone hates you. Because they're used to like, ah, they're gonna go and they're gonna make you go into a workshop and do sticky notes and then gonna do wireframes and it's gonna take forever. And why can't we just bytote the high fidelity version from the beginning? Whereas process is actually just the way you make decisions. It's just a standardized way of, you know, ahead of time, before you make the decision, knowing this is what we're gonna use to make sure we're thinking about it correctly. And especially when your organization is skeptical of what I call a formal process, you kind of need to be able to sneak in the right pieces. And a big part of that is alignment. It's convincing your stakeholders that your interests are their interests. But it's also about helping them discover what their interests are. Because in the same way that someone latches onto the first form factor they saw in someone else's product. factor they saw in someone else's products, they latch on the first idea they had, the first suggestion a customer has made, and helping them articulate what it is that they want, then it makes it much easier for you to say, well, right now you don't have what you want, you're unhappy, I'm unhappy, here is how we can collaborate to get to a point where we're both happy. And you've never said design or UX or user or anything, but you've earned that trust by aligning yourself with them, rather than aligning yourself against them, and expecting them to just be like, okay, yeah, we'll do what design says, because it says you have good taste. So yeah, influence by design, through line conference workshop, you know, put it put it into Google. Oh, yeah. But, but there's an interesting point there about almost vocabulary. And this is something that I taught to product managers about as well. But I think you've just touched on it from the design perspective as well. And maybe also from designers speaking with product managers as well, this idea that you're not going to go up to someone with, again, a canvas or a framework or a book or a quote or a talk, headline or whatever, and just win them over by talking to them in a language that maybe they don't quite understand, because they've never worked in that industry, you know, using buzzwords are your buzzwords and not their buzzwords. So what I really liked about what you said there, and again, something to advocate for with product people, product managers is try and frame things in their terms and let them know what they're going to get out of it, versus making it sound just like you're trying to get something out of it. So do you think that's something that designers or product managers, for that matter, are generally good at? Or is that something that's kind of a development area from the people that you've worked with and kind of interacted with? I think that is something that everyone is bad at when they start. And advancing in seniority in any role, in a significant way, requires you to be able to learn how to do this. I think that it, it really depends. I think that to a certain extent, in engineering, you can get by on not doing this, to a certain extent, in user experience, you can get by on not doing this. I think as a product manager, you can't not do this. If you want to, you know, rise above senior, and it's actually really funny, I think that our industry considers senior to be the second least senior level that you can be. If you want to rise above that level, I think you need to be able to speak cross functionally in a language that people understand. And oftentimes, I think that it requires you to be very, very precise and deliberate with what you are saying and with what others are saying. For example, the word prototype. No one knows what it means. Everyone thinks they know what it means. And then when they talk to someone else, and they, you know, they think it means a different thing, you're going to be talking past each other, no matter what. Yep. Because a lot of people think prototype, it just means that, hey, this, it's a it's a finished product is just running on my computer instead of in production. Other people think it is running in production. And it's essentially just an MVP, other people will tell you to never say MVP, to say MLP. And then you go on this, like, pointless conversation of what does this word mean, versus what is this other word? And so this is why I like to steer the conversation towards, don't talk to me about the form factor. Don't talk to me about the shape of the output. Talk to me about what you want. Do you want something running in production? If so, why? Is it because you think you can only learn what you need to learn from real users using it? Because there are a lot of things you could only learn from real users using it. But the majority of things that you need to learn to make decisions about the product, you could actually learn from a couple of interviews, you can learn from 10 people using it on the local machine, you can learn from showing it to a domain expert, and you know, just setting up an hour for them to tear into it, and explain all the ways it's not going to work. And all of those ways are much cheaper and much faster than putting it out there. And looking at your analytics crash, and then trying to figure out why your analytics crashed. Because quantitative data, it's like, yeah, you can't argue with it. But you can interpret it in lots and lots of different ways. And you need qualitative data to make that interpretation. Otherwise, you're just doing guessing. And qualitative data is really cheap and easy to get through these kind of interview methods. And so when people come to me and say, I need a prototype, I'm like, alright, let's talk about what decision you are trying to make. And let's talk about how I'm going to be part of that decision. Because a lot of the time, what you're actually doing with a prototype is you're documenting a decision that was already made. And like, I'm not interested in doing that. Because a lot of the time, then it's just going to sit in confluence, and no one's going to care about it. Or it's just going to become JIRA tickets. And if that's what you want, you should just make JIRA tickets. If what you want is a demo, but you can show to stakeholders, then make a demo, but you can show to stake holders, that's not a prototype that requires different things. And it gives you different things. Because if you want, for example, budget, to build it for real, you will make something very different from if you've already built it, and you just want to do a victory lap to make a case for your promo packet. And obviously, the type of thing you want to accomplish with this prototype also impacts how interested I am in helping you because if it's going to help my promo packet, that's one thing. But why would I help you to do your victory lap? If this is already done, the decisions are made, the design essentially is made and all you want is visuals. Like if you've already called all the shots, then go and ask Claude to do it. I don't care. I have better things to do. I only have so much time to dedicate to a lot of projects in the organization. I think that this is also something that a lot of people could take as a lesson. They try to do everything. And they turn out to not achieve anything in a very significant way. The quality bar drops to the floor. They feel like they have no choice but to use AI to, quote unquote, increase their own productivity because otherwise they're drowning. But what do they do with that extra productivity? They take on more work. And so they're just as drowning as they were before. But the quality of their work has gone down. Quantity of it has gone up. They're now context switching 20 times per day. And they've essentially sabotaged themselves. And this is also not a unique suit to design or to product management. I think everyone is really struggling with this right now. And what I'm calling problem design is really a cure for this because you can step back and you can use this logic to figure out why am I doing this? How much effort should I put into it that is commensurate to the amount of benefit I am going to receive? And if the answer is not a lot of benefit, well, then don't put in a lot of effort. Do a bad job, half-ass it. If no one's going to know, if this is not an input into anyone's work, then it doesn't need to be good. But if this is an input into a really important workflow, and it's going to be visible as an outcome, and your name is going to be on it, that's the stuff you should put all of your effort towards. And that's the thing you should do rigorously. That's the thing that you should do without AI, because the risk of hallucination turning into a high-profile mistake with your name on it is huge. But I mean, you just talked about AI. And of course, that's one thing that everyone's talking about. And you mentioned vibe coding as well. One of the hero use cases for vibe coding is pretty much that anyone, designer or otherwise, can now make prototypes or working mockups or MVPs or whatever you want to call them, again, as you say, of varying levels of quality, but at the same time, they can make them. And that then leads us to situations where non-designers are going to be going out there and building stuff, maybe even without the design team's knowledge, and start to show them to customers, start to maybe do their own kind of research or qualitative or otherwise, with customers, and then maybe coming back with something, to your point, like a pre-baked decision. And in some ways, some people might sit there and say, well, it's good, because it kind of flattens everything, democratizes everything. On the other hand, you might sit there and say, well, to your point around hallucinations and half-assing that maybe it's not going to do a proper good job that's well thought through from the off. And part of that is the problem design part, and part of that is the execution part. But before we talk about the problem design part of that, do you think in general that it is a positive thing that anyone can just hack together a prototype now and show it to a customer, whether they're a designer or not? So the whole democratization issue started in the US even before AI, because there was this trend around democratizing user research. And we touched on this earlier, the idea that a product manager can go and talk to a customer and do their own thing. And on the surface, yes, it's faster, because you don't have to go through a centralized team. And let's face it, user research team is typically underfunded. And so they become a bottleneck. Because if every process goes through them, it takes a very long time to actually get out to talk to a person and get your data back. And the problem is if you go off and do research, and I go off and do research, and we happen to be researching the same subject, but we have no standardized methodology, we have no standardized training, we will come back with two different results. And we will each think that these are research informed results. And we're going to go and build something and then we're going to come back and present it to stakeholders. And they're going to be two different solutions that each claim they are the right thing. And now we have just broken in half our organization's understanding of reality. And now imagine every single product manager does this and you have a kaleidoscope of multiverses basically, where every team is living in its own world. No one is coordinating with one another. People are arguing that hey, your thing is wrong. My thing is right. No, my thing is right. Because I did research. Well, here's I did research. And so to a certain extent, without going through that centralizing synthesis function, you've just wasted all of that effort, because you've created a ton of extra work downstream, untangling what the hell actually happened, and what the reality is. And so now we come to AI where anyone can go and jump into making this, this product. And like I said earlier, people get quote unquote ideas based on what they see other products do. They get ideas for a form factor, they prototype form factors, they go show form factors to customers and customers say, Oh, that's cool. Because no one says no to a new feature. Everyone's always like, do you want this thing for free? Do you want free ice cream? Sure, I would love free ice cream. You're giving me vanilla ice cream, I'm going to take it. You're giving me chocolate ice cream, I'm going to take it. Maybe I want, you know, a cherry sundae, but you have to pay for a cherry sundae. And you're offering me vanilla for free, but I'm going to take vanilla. Sure. Like that's not necessarily the right thing. But I'm gonna, I'm not gonna say no. And so lacking the very, very fundamental basics of research rigor, you now have anyone promising anything to anybody. This was actually part of the topic of my latest issue of the product picnic around this idea of trust and being able to substitute trust by just extruding something with AI that looks like multiple people have thought long and hard about it and made a bunch of decisions. And in reality, there was no rigor, nothing was made, everything you are seeing could very well be wrong. And that's actually something we're tackling now at JustWords, where I think, practically, you cannot stop people, as long as they have access to a model and a token budget from doing this. To the extent that the Jared School thing of everyone as a designer is true, I think that everyone is capable of recognizing that there is a problem and taking a stab at solving. Once again, if it's a problem they feel like is worth solving in the first place. But that output, what they created, it's not the end. It's the first step of a long and complicated iteration process that does not run through production. And the problem is that it looks like it's done. It looks high fidelity. It looks polished. If you're lucky, it might even have picked up your design system and actually made it look like it's part of the product. That is very difficult to do. Nothing does it consistently right now. But there's always a chance. And going to a stakeholder and saying, look, we made this thing. I showed it to someone. They liked it. Let's put it into the product. It feels productive. It feels like, oh, you've just accelerated development by 20 times. Because what used to take us several sprints to do, now just took you a day to do. Well, you just skipped all the work. You didn't accelerate it. You just skipped it. You skipped the alignment. You skipped the understanding of what the edge cases are. You skipped the prioritization. And so you have no guarantee that this thing you've just extruded out of your software is actually the right thing to build. And now development teams are buckling under the weight of all of these PRs for things that no one really cares about. Because you haven't actually ever tested in a real rigorous way if anyone is going to want this. And you're certainly not going to be the one maintaining it. You're certainly not going to be the one integrating it. That's the dev team's job. And so your product is going to start falling apart. You're going to lose all sense of information architecture. Because now you have these features popping up that don't really fit in anywhere. That don't necessarily... You haven't done the work of deduplicating them from one another. Because you're not talking to anybody. You're talking to Claude. Why do you need anyone else for? Odd is a genius. And then it basically degrades your product into a mess. And engineering teams are already trying to push back on that. And so the solution, I think, is you can't stop anyone from prototyping. But you have to have a reasonable intake into a real design process. Into real prioritization based on the value of what you're building. And a real feedback loop between deployment and iteration. Because we all talk about this. We all talk about you build, measure, learn. In practice, it's just build, build, build. It's especially funny when building, measuring, and learning are the job of three different teams. But even if it's the same team, your roadmap doesn't have room for learn. Your roadmap says, we build this thing, we build this thing, we build this thing. Well, what if you build it, and then you found out it's the wrong thing to build? You don't have any room in your roadmap to fix it. You have to go on to do the next feature. The next time you have time to fix the thing that was broken is next year. And so it doesn't really matter that you shifted faster, because the information coming back to you is coming back at a human pace. It's coming back at the pace of how long people try to use your product. And in B2B, that takes even longer, because they have to integrate what you've built into processes. Then they figure out, oh, it doesn't work. Then they try to fix it themselves. They try to work around it. And finally, they're going to come and tell you, hey, this sucks. And that takes a long time. You can't accelerate it with AI. But again, you can skip it. You can just say, well, we generated this, ship it. We generated this, ship it. And then you ship enough stuff, you make yourself look really good, then you go work somewhere else. And by the time the problem has come to light, you've already left and been promoted to middle management at Hyperscaler. And you're now the vice president of AI or something or other. And you don't care anymore. And I mean, that's not new. Also, that's been a problem for decades. But Vivecoding has made it easier to do. And so by the time everyone realizes just how much money you spent on generating these things no one needs, there's no consequence. The feedback loop never actually closes. Well, another way, though, that people are trying to speed this bit up is to try and speed up that evidence gathering at the beginning as well. And that's something I know that you've written, spoken about this idea that research is a human skill, shouldn't be outsourced to AI. And I think the way you put it is that user research, unblocks productivity, not just velocity. And we could all argue about whether many companies are actually showing velocity versus just directionless speed. But one of the ways that they are trying to speed stuff up is to, for example, use things like synthetic users, LLM based user research to try and almost short circuit that thing. Because, you know, going out and speaking to people takes a long time, you've got to go and find the people they might lie or whatever. Whereas an LLM, as you say, it's a genius, it will just come up and it'll just say, It's a genius that will just come up and it'll just say all of the things. And you'll be able to just make some decisions off of that. Where were you on synthetic users? Is it complete garbage as far as you're concerned? Is it, is there a use for it in the toolbox? Uh, is it something that you should fully use? Like where do you sort of fit on the synthetic user scale? Well, to put it plainly, synthetic users are a scam. It is a dream being sold to stakeholders who think they already know everything. Hey, why don't you just put all of your data into this big bucket and then, you know, put, poke your hand into the bucket and pull out some insights. And there's been a lot of research into this and every single paper that hasn't been retracted says it's crap. There was one from Bain Capital recently. And you know, they're not exactly the most, uh, human centers company out there. And their conclusion was that, um, it's unless you have perfect data that is already structured on your customers, unless you basically essentially have a real digital twin, it doesn't work. And the thing about that is if you already have all that data and you already have a digital twin, you don't need the LLM, you already know what the problem is down to the microphone, because you've already modeled all of it yourself. And if you don't have any data, uh, most companies don't because they see this as a shortcut. And so they're like, cool, we'll just, you know, ask the LLM what to do. Well, the LLM doesn't know what to do because he didn't tell it what good looks like. He didn't give it to the data. He didn't train it. And so it's going to give you some generic crap. It's the same generic crap that it gives to everybody. And so you've essentially commoditized yourself in a, an industry where commodities lose, you have no margin on a commodity, the, the way you build a product that makes a ton of money is you build a moat that no one can cross. What kind of moat are you building when all of your insights come from Claude, um, or from whatever other, like synthetic users don't come from a, you know, their own model. These are all foundation models with some stuff on top of it. So it's, it's going to be commodity insights. It's going to be commodity software. And then someone else is going to outflank you. If you want real insights, you can't go to a piece of software that is probability based, but it's going to give you the most likely answer or the most like, um, it's going to give you a response that is shaped like the most likely response to your question. You're not going to get something insightful. You're not going to get something new from that. You're only going to get that from talking to people and from understanding that what they are telling you is not the common wisdom that everyone else already knows what they're telling you is something new and it's an opportunity no one else is exploiting. And then there's a lot of other work, of course, that goes into maintaining that moat. Uh, but that's really the first step. And I think that is much more, maybe obvious from a design perspective than it is from a product management perspective. If all you're concerning yourself with is how do I fill the roadmap so that my engineers have something to do, then sure. Absolutely. Knock yourself out, go ask Chad GPT. What are the top 10 problems of my user? Because at the end of the day, you don't care. Uh, if like, if the job you are doing is keeping another team busy, you can keep them busy for days. If the job you are doing is delivering value to your company, then you're not going to be delivering any value. I think that everyone really knows already what all the problems are. Um, if, if you've been solving a problem for a length of time, you have a massive backlog already of things you need to do that you've had to put off because of competing priorities, just do those things. Why, why do you need to come up with new things? And why do you need a robot to generate them for you? Oh, because it's cool, right? Like it's, uh, it's exciting and it's fast, I guess the devil's advocate position there, and you know, the interest in your take on this would be that there are some problems, let's say, which, especially if you're, you know, let's imagine you're starting a company in an industry that's not your industry. Like you didn't work in an industry before, but you're kind of interested in it, or you've seen a problem that you think needs to be solved and that you could use an LLM or LLM based or synthetic users to give you a sense check, some of the basic needs that are kind of obvious to someone that knows that space, but maybe you don't know it as well as they do kind of almost a substitute for, oh, that's an obvious answer type sort of subject matter expert type responses. Like, do you see there at least being value in that? Or would you still quite clearly be wanting people to just go out and speak to some people that are actually part of that market? So there's, um, there's a question I like to ask when I talk to someone and I say, well, what's your strategy? And they say, we're going to do this. We're going to do this. We're going to build this thing. We're going to increase consumer demand by 30%, whatever. And that question is, what hasn't that already happened? What has prevented our sales from being 30% higher already? Why are customers buying this much and not more than this? And this is exactly the same question. Sure. You can generate a list of the sort of the basic needs, but you cannot generate why are those needs not currently met? Cause if those needs are currently met, you don't have a market. You have, you know, um, a bunch of incumbent competitors. You need to disrupt somehow. Um, and if they're meeting those needs, that's not how you're going to be disrupting that market. So you need to look at that list, but you also need to know are those needs being met and if not, why not, and the only way that you can get to that point is going on the ground with the people actually doing the work and looking at how theory meets practice, because I mean, I think every product manager will tell you that their day-to-day work looks nothing like what's in the scrum manual. It looks nothing like any of the textbooks. And so, you know, one of the, one of the critiques you always hear about people like agile coaches who come in with pure theory is, well, your theory doesn't have a place to stand on in my reality. There's a disconnect between the ideal general, like high level concept that's in the book and the actual real life situation that I'm facing, and that's going to be the case with every kind of generic insight that you get and you don't get more generic than an LLM. So once again, if you have the training data that will let an LLM give you those responses, just use the training data. Just read through that stuff yourself. If you don't have that data, there's nothing that can tell you that's going to be useful. Well, fair enough. So it sounds like humans are still very much needed for the short to medium term, although you never know the next model down the line, who knows what will happen, right? Yeah. The notion that I'm going to add there is tacit knowledge is real, and it's not something you can put into a dataset and train a machine on. So I think that people say, Oh, the technology is getting better every day. And that's like me saying, well, if I start working out, I'm jogging faster every day and someday I will run to the moon. You never know. You're not just cause you haven't done it yet. Right. But, uh, yeah, I dunno. I guess, yeah, there's just this, there's this, I dunno, I kind of, I know exactly what you mean. And I have this kind of quandary in my head, because on the one hand, I like to say that I'm kind of one announcement away from being made to look stupid by saying that things aren't going to be possible, like, you know, I don't think that there are certain things that are gonna be possible, but there'll be one announcement tomorrow and all of a sudden, oh, damn it, that's possible now. And I look dumb, but actually I think that's a positive thing as well, right? To actually be able to sit there and be inquiring and ask good questions about the, the limits of some of this technology versus just kind of just accepting what the LinkedIn or the Twitter people are just sort of trying to shovel down your throat because, you know, we, we need to use these things today as well. Right. I make good decisions with these things today. So, uh, yeah, just be, just be careful. If, if I didn't want to sound stupid, I wouldn't be on the internet. I, that's kind of my philosophy is I'm not afraid of sounding stupid. I ask stupid questions at work every day too. Um, and there's stupid questions up until everyone says, well, that's obvious. The answer is this and everyone's answer is different. And then they say, wait a minute, that's wrong. And they start fighting with each other and I'm just sitting here with like popcorn, uh, and it turns out that the stupid question was actually not that stupid after all. Oh, there you go. Well, if people want to come find you after this and ask you a question, stupid or otherwise, you know, chat about burning products, talk about the power of problem design or find out where to pull up a lawn chair at your product picnic. Uh, where can I come and find you after this? Absolutely. Um, I'm on LinkedIn a decent amount, probably more than it's healthy. Um, product picnic.beehive.com is the newsletter. And you can also find me on blue sky. Um, as well as wherever other podcasts happen to be that I'm on. So yes, you'll, you'll see me around. Um, my name is quite Google-able. Um, there's one other Pavel Samsonov, uh, that comes up and is a professor at Lafayette university. So whichever one's not him is probably me. Uh, so, you know, pop it into your, your favorite search engine. I'm told that Google has gone, um, all in on AI now. So I switched to DuckDuckGo, um, you know, use Bing, use, I don't know if Ask Jesus is still around, but, uh, you know, whatever AltaVista, um, a little blast from the past, um, you know, some, something exists. Yahoo. There you go. Yeah, just, you know, whatever, whatever works for you. Um, open, open a phone book if you're in New York city. Um, you know, so some of those old school ways I think are, uh, are starting to become more reliable than, uh, some of these new approaches. Like, uh, Marty McFly and back to the future. Just go into the cafe and tear the page out and then come, come find you at your house. There you go. Well, also the, the Terminator. So, um, pick, pick your poison, right? Well, I'll do my best to make it easy for everyone and link as much as I can into the podcast show notes anyway, and hopefully you'll have a few people wandering in your general direction and maybe the other parallels as well. If, uh, if they like the car, these jib as well. Um, but no, it's always good to tap into your brain, always enjoy your content and, uh, and your newsletter. So, uh, you know, it's good to have this kind of chance to do some one-to-one and go into some deeper meaningfuls. So obviously we'll stay in touch, but, uh, as for now, thanks for taking the time. Yeah. Thanks for having me.