The Wirecutter Show: Tips for Using A.I. Smartly With Kevin Roose

Well, Casey, here we are. It's the holiday season, and we've got a special Tuesday episode for our listeners. We do, and boy, is this a special one, Kevin. And in this sense, I'm not really in it. Yes. The best hard fork episode ever, they're calling it. No, I recently went on another New York Times podcast. This is The Wirecutter Show from the fine folks over at Wirecutter. And we had a wide ranging conversation about which scented candles you should put in your house. No, it was about AI, obviously. They asked me to come on to talk about how to make AI tools work for me and for others. And we thought it might be something that hard fork listeners might like to hear. Yeah, and actually, recently, The Wirecutter named this episode the best option for people who don't like the sound of my voice. So something fun for you. And even as we bring you that today, Kevin, we have even more planned for you this week, because on Friday, we will be back and I will be participating in our annual Hard Fork New Year Tech Resolutions Show. And we're also gonna be answering a bunch of your questions, listeners. So stay tuned for that. Thank you so much, Hard Fork listeners and viewers for being with us in 2025. It has been a real joy to bring this show to you. And we hope you are having a wonderful holiday break and that you have a happy new year. And we hope you have an old Lang sign as well. We do, we hope you have an old Lang sign. And if your Lang sign is looking too old, get a new one. Yeah. Yes. Does Wirecutter have any recommendations for the best Lang sign? Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. I rely on Wirecutter before I buy anything. And I am desperate for someone to tell me which language models are good for which things, because otherwise, I am spending so much money on these things, and I'm spending all this time trying to figure out what is good for what. And I would just love it if you all, who are the experts in the world at testing things and figuring out what they're good for, would do that work for me. I'm Christine Cyr-Classette. I'm Rosie Garren. And you're listening to The Wirecutter Show. Rosie. Hi. Hello. Hello. Welcome to another episode. Today, I am gonna be chatting with Kevin Roos, New York Times tech columnist. He writes about many facets of tech and talks a lot about AI, both in the newspaper and on his New York Times podcast, Hard Fork, which he co-hosts with platformers Casey Newton. Kevin is so great. The podcast is so great. I'm really, really excited you're doing this. I am too. So Kevin and I are gonna talk today about something that we haven't covered on this show before, AI, which, of course, everyone is talking about AI. But the reason we're gonna talk about it on The Wirecutter Show is because there is this intersection of consumer products and AI at this point. Absolutely. And especially with these large language models, LLMs, which are the chatbots like ChatGPT and Clod, he knows a lot about all of these. And a lot of people are using them at this point. And so we're gonna really dive deep into all of that. And I have to say, in this episode, we talk a lot about AI chatbots. So we do need to make the disclosure that we work for The New York Times company, which is suing several companies, OpenAI, Microsoft, and Perplexity, over alleged copyright violations. That's right. And with that, I'm looking forward to hearing you and Kevin discuss AI's changing the way people shop, how it's being integrated in hardware products, and hopefully hearing some of Kevin's best tips on actually using these LLMs smartly and strategically. I'm really interested in this intersection between AI and Wirecutter. After the break, Kevin Roos. Welcome back. I am really excited to have Kevin Roos on the show today. Kevin is a technology columnist for The New York Times. He is also the co-host of The New York Times tech podcast, Hard Fork, which he co-hosts with Casey Newton of Platformer. And this is a great podcast if you are curious about what's happening with AI and who isn't curious about that at this point. So welcome to the show, Kevin. It's great to have you here. Thanks so much for having me. I listen to Hard Fork quite a bit. I love the podcast and I feel like in the early days of chat GBT a couple years ago, you and Casey were how I was staying up on what was happening with AI. And I will tell you that I kept saying to my bosses at Wirecutter, you guys, have you listened to the newest Hard Fork? And what are we doing about AI? We are doomed. What's gonna happen with shopping? And at that point they were like, you're like chicken little, what's wrong with you? So I feel like you have taught me a lot over the last few years. Well, I'm glad. We didn't intend it to be an AI-focused podcast. When we started it, we actually thought it was gonna be a crypto-related podcast. And that's why we picked the name Hard Fork, which is sort of an obscure crypto programming term. But you know, things change. And all of a sudden we found ourselves in the chat GBT world talking about AI every week. Well, and it's very, very helpful. I am curious though, you really have taken a hard line and you're reporting on AI all the time. You're using it all the time. How has AI really impacted your life on an everyday level? Are you using it all the time or is it something that you're only occasionally using in your personal life? What are the touch points for you? I use this stuff constantly. I am AI-pilled, as they say. I pay for more subscription AI products than streaming TV services. And I pay for a lot of streaming TV services. I use this stuff probably dozens of times a day. And what are you using it for? Walk me through a day. How are you interacting on a personal level and I would imagine a professional level too. Yeah, I mean, there's so many things. It's hard for me to even list them all, but here are a few. So I wake up in the morning and I get my AI-generated summary of my email inbox. I use this program called Quora and I have hooked it up to only my personal email. I'm not doing anything like this on my work email, which is more sensitive. But on my personal email, I have hooked this thing up so that it synthesizes and summarizes all of my emails and pre-populates drafts to respond to anything it deems important. Then I'm also constantly using it for things around the house. You could go back through my recent queries and probably 60% of them would be some version of like, how do I fix this air fryer? Or how do I install these training wheels on my kid's bike? What is this weird plant in my garden? That kind of thing. And I'm using it for work. I use it for not for writing my columns or my podcast, but for research. I am writing a book right now. And so I am constantly using AI to look things up for me, to help me sort of piece together various primary sources. We could talk about any of that in more detail, but that's sort of how I'm using AI just today, for example. Yeah, I am super curious about that. And we will talk a little bit about that in a bit. And your book, just to be clear, is about AI, right? I mean, it's about like, isn't it kind of like you're writing towards the singularity at the end? Yeah, the book is essentially the story of the race to AGI, which is artificial general intelligence, which is what all of these companies, OpenAI and Google and Anthropic are now sort of pushing toward, this vision of a sort of human level AI system that can do anything the human brain can. When you look at the general population, what do you think people are using AI for right now? What are you hearing from your listeners? What are you hearing from people out in the world? So I think it's a really wide mix. I hear from people who are using AI tools for everything from medical research and developing new drugs to doing story time with their kids. It really runs the gamut. There was a really interesting study that OpenAI's economic research team did this year where they looked at 1.5 million or so conversations with chat GPT and how people were using it. And they found that the biggest use cases among their study were what they called practical guidance, which is people basically using it to teach them things. How do I fix this appliance? Maybe do some health or fitness coaching. Number two, the second most popular category of use was what they called seeking information, which was sort of as the kind of Google replacement. How do I get to this place or book me a flight or something like that? And then the third category was writing. So as you would expect, a lot of people are using this stuff to write emails, business memos, to write maybe papers if they're a student, to help them with translation, things like that. So those seem to be the largest categories of use across the user base. I talked to a lot of programmers and a lot of engineers, people who are technical and work in tech. And so I'm just hearing the craziest stories about how people are incorporating this stuff into their work life. The programmers I talked to, because these tools have gotten quite good at writing code, they'll tell me like, I don't even really code anymore. I just supervise and orchestrate kind of this little team of AI coders. And my job is sort of reviewing their output and stepping in when necessary. But basically I sort of like set them off on a task and then I go make myself a cup of coffee. Something that you didn't mention is companionship. Recently on your show, you've been talking about Character AI, which is a site that people can have sort of a relationship or a conversation with user-generated characters. How much do you think people are really using AI at this point as a relationship? I think it depends. You know, in the study that I mentioned, companionship was not one of the sort of top usage categories, but I think that maybe under counts the number of people who actually do feel somewhat attached to these products on an emotional level. Maybe they wouldn't go so far as to say like, I have an AI boyfriend or I have an AI friend, but you know, they sort of rely on this stuff. I'm thinking in particular of conversations I've had recently with young people who say like, oh yeah, I talk about like chat as if it's just my friend, even though they know it's not a human, but they do feel connected to it. So I think there's also a big generational piece of this. I am an adult. I have too many, you know, human friends to keep up with, but I think if you're a teenager, this stuff is coming on really strong and really fast. And I think that's one of the most underappreciated parts of this AI revolution is that a year or two ago, barely any teenagers would have said, I have an AI friend. And now something like half of teenagers are regular users of these AI companion products. Yeah, that's wild to me. I'm a parent of a 12-year-old and I know that she uses chatbots sometimes to kind of navigate tricky social emotional situations with her middle school friends, which I mean, that's just like a whole world that if anybody remembers being in middle school, there's so much drama. I don't know that she's using anything that's an actual character that she has a relationship with. I think there's probably, would you say there's like a distinction there between typing into chat GPT versus using like a service where you've got a character that you've got a relationship with, or is it kind of blurry? Is that line blurry? I think that line is blurring. It used to be that the sort of mainstream chatbots were all very like formal and business-like and it sort of sounded like you were talking to like a, you know, a Wikipedia article. But now these companies have made strides towards making their chatbots more personable. So they, you know, Sam Altman at OpenAI just recently said, you know, they want to make this a pleasant experience for people. They're gonna even let, you know, adults do sort of erotic conversation with their chatbots. So these companies, I think, are all trying to figure out what their lines are. But yeah, I think the difference between sort of the mainstream chatbots and these more tailored companionship products is getting more blurry by the day. I'm curious, can I ask you a question? Yeah, ask me. How do you feel about your 12-year-old using AI for this kind of middle school drama? It's interesting because I sometimes feel like it's fine and I have actually used it to navigate friend drama or family drama. And so I think she's seen me doing it and thought, oh, I should do that. I do go over and read what it has said just to make sure that, and she's pretty open with me. If my kid were not very open or I didn't have confidence that she was sharing what it was telling her, I might have more concerns. But so far, the advice seems to be pretty boilerplate. It doesn't seem to be problematic. But I think it's an interesting use case for sure. I think this is really tricky because I remember being 12. I had not the easiest time in middle school and I don't think anyone has an easy time in middle school. But I think if I were 12 and these chatbots had existed, I would have been tempted to spend a lot of time chatting with them, and it would be more time than was healthy for me. And I think it really, it sort of matters what kids are talking about with these chatbots, but it also matters whether they're using them as substitutes for some real world human connection that might actually be more fulfilling even if it's less efficient or less sort of reliably available to them. Yeah, that I think is a great point. I will also say she uses it to be her stylist. She asks it about wardrobe advice, which I think is pretty hilarious and awesome. She's well-dressed. I do wanna ask you about the different chatbots because you have used a ton of these. I think a lot of listeners will have probably used some of them, whether it's ChatGPT or Clod or Gemini. I think you've talked with my colleague, Jason Chen, about maybe Wirecutter should do a review of these. We're not quite sure yet. It seems like they're just changing so quickly. If you were to do a Wirecutter guide, which ones would you recommend and why and for who? Because it seems like different chatbots are better for different tasks, essentially. They definitely are, and I'm happy to walk you through my current feelings on this, but let me just make my case to you directly since I have you. Here's why Wirecutter should review large language models. For me, I need this desperately. I rely on Wirecutter before I buy anything, a spatula for my kitchen. I look up the Wirecutter review, and I am desperate for someone to tell me which language models are good for which things because otherwise, I am spending so much money on these things, and I'm spending all this time trying to figure out what is good for what, and I would just love it if you all, who are the experts in the world at testing things and figuring out what they're good for, would do that work for me. So it would be a huge personal favor to me if you would do this. Now, I realize that is not a reason to make editorial strategy decisions. You all are very capable of running your own website, but that is my case for why Wirecutter should review AI products. It's a convincing case. Now, here is my current setup. This could change in a week. It probably will change in a week. These things, as you said, do fluctuate so wildly release by release, week by week, update by update that I often find that the tool that serves me well one week is no longer the right tool the next week. So here, as of this recording, is what I'm using. Right now, I am using Perplexity's Comet browser. That is an AI-powered browser based on Chrome, but sort of with some additional AI features built into it. I use Claude from Anthropic for my sort of daily driver AI model, mostly with creative work, coding. To the extent I'm coding, I use Claude for that. And then what I call like matters of the heart, like things that I need advice on, things that I'm struggling with or navigating in my personal life, advice on relationships or parenting, that kind of thing, I use Claude for that. I find it has like a higher level of like what you could call emotional intelligence or some convincing replica of that. I use Google's Gemini for research and for working with big chunks of text. And I use a separate Google product called Notebook LM for my book. Notebook LM basically allows you to dump a bunch of documents into a single notebook and then kind of chat with them. So I put all my research materials for my book into one giant Notebook LM. And then I can just ask it questions. I can say, oh, who were the three people at this meeting in 2019? Or who would be six good people to interview about this topic? Or who was the person who said that thing that I forget now, but that I think it had something to do with this? And it will actually pull out from my sources that I have uploaded what the right references are. And then what I love about Notebook LM is it will give me a little citation where I can go back and check that like, actually, yes, that citation is correct in the original source file. Chat GPT, I use less than the others, but mostly for personal and professional reasons having to do with the fact that our employer is currently in litigation with OpenAI and Microsoft. I use Chat GPT just to sort of see what it can do and test it out, but it is not sort of part of my daily workflow as much as the others. Yeah. And I should also say there's one more tool that I really am using multiple times a day. And that is a tool called Super Whisper, which is basically an AI voice dictation tool. So one of the things that these AI models have become very good at is taking audio speech and turning it into text. Obviously, that's not a new feature that's been built into every like iPhone and computer for years now, but this tool is built on top of something called Whisper, which is OpenAI's speech to text model. And it's quite good. And what I like about it is that it can sort of clean up your filler words. It can present to you like not an exact transcript, but something that sort of reflects what you were trying to say. And so I use that a lot. I now dictate a lot of emails. I dictate some writing that I do. And I basically talk to my computer about twice as much as I type into it now. That sounds super useful. I really want you to deliver me like cartoon characters of all these different chatbots because I feel like they would all look very different. Yeah, I feel like Claude, I sort of picture as like a philosophy grad student, wise and eager to help. Sometimes, you know, a little too philosophical and you're just trying to like get something very basic, but like a sort of empathetic and wise person or chatbot, I guess. Gemini, I would say, is like a reference librarian, like a thing that is able to like hold massive amounts of text in its head at all times. And then ChatGPT is sort of fluctuates based on what you're using it for. I think it's a very versatile model that can kind of act any way you want it to. I've used Claude quite a bit. And I find that when I use Claude, Claude is so nice, like overly nice. Its responses are so polite. And oh, like you're so brilliant for asking that question. Oh, what a wonderful way to think about that. And sometimes Gemini does that too. And I feel like I'm not sure I'm asking questions in a way that will give me an honest response. Are there tricks or tips that you use to get good information from these chatbots so that you're pulling information from the right places, essentially? It's not just stuff off the internet, it's quality information, but also that it's not sort of like pandering to you, if that makes sense. No, this is super important. And this is something that I have spent a lot of time thinking about, because I have noticed, as you have, that if I do nothing and just talk to these chatbots, they will tell me I am the smartest person who has ever lived. They will tell me that all of my ideas are great. They will tell me that my taste is unparalleled and that I'm basically a modern-day Leonardo da Vinci. And so I have had to add custom instructions. Do you have custom instructions on your chatbots? I don't, no. Okay. Tell me how. What do I need to do? This is a pro tip. So in Cloud and ChatGPT, I'm not sure about Gemini, but at least in those two, you can go into your settings and actually give it custom instructions for how it should. custom instructions for how it should talk to you. And these will kind of be invisibly appended to every conversation that you have with the chatbot. So for example, my custom instructions for Claude are, I'll just read them. Claude should talk to me informally like a wise and trusted friend. I don't like preamble, just get to the point. I appreciate honest feedback and don't like sycophancy, but I also appreciate praise when warranted. I am not always right, but neither is Claude. I value Claude's perspective and appreciate being pushed to consider views I may not have considered. Don't end every response with a follow-up question. So that is sort of like my little attempt to like make the model less flattering, less obsequious, less telling me I'm great at everything. And so I recommend that everyone who is spending serious time with these models go in and write your own custom instructions for how you want it to behave toward you. Thank you. We're going to take a quick break. And when we're back, Kevin and I will talk more about how AI is changing the way people use products, both online and in real life. Be right back. We're back with Kevin Roos, New York Times tech columnist and co-host of the podcast, Hard Fork. In the early days of Chat TPT coming out, you and Casey on your podcast talked a lot about how AI was going to change the way people shop. And of course, at Wirecutter, we're thinking a lot about this. Do you think that at this point, AI has already changed the way that people shop? And if so, how? Absolutely. I mean, it's changed the way I shop. My first stop is always Wirecutter if I'm buying something for my house. But then if you all don't have an article on it or it's just something that sort of wouldn't be that commonly shopped for, I'll go to a chat bot and I'll say, help me decide between these two things or I need a new string trimmer for my yard. Help me decide between these two models and I'll do it that way. I'm not the only person who's doing this. Lots of people are using these things for shopping. This is a big category that companies like Google and OpenAI are very interested in cracking. And right now, these things mostly do not have ads in them, but all these companies have expressed an openness or a willingness or are actively working on trying to basically direct people to products and take a cut of the resulting purchases. What do you think I should be worried about as somebody who works for a review site? I mean, look, I think the danger is that this stuff can just sort of collect and synthesize all of the work that you and your colleagues at Wirecutter and also all of the other review sites on the internet are doing and present that to their chatbot users and cut out the sort of middlemen, as it were, and take the cut of the affiliate purchase directly. That would be a very bad situation to end up in. I think that a lot of companies are trying to figure out how to optimize their pages for chatbots the way that they used to optimize them for search engines. This is a big area of focus and investment right now is like, how do I get my Airfryer review to show up at the top of ChatGPT's responses? Because it is not necessarily the same techniques that we use to show up at the top of Google results. That is very interesting. From a user perspective, for people who are listening to this who might be thinking about using a chatbot to shop or who are maybe already doing that, are there invested parties behind the scenes who are paying to be served higher basically on the page? The answer is yes. There are companies now that are calling themselves specialists in AI optimization and selling their services to large, you know, Fortune 500 companies. And what they're telling those companies is we can make your products appear higher in chatbot results. And the ways that they do this are not always clear or transparent. I think it's very fair to not only ask whether people are gaming these chatbot results for shopping queries, but also whether the AI companies themselves are going to start prioritizing the companies that they have business relationships with in their search results. So we don't have any examples of this happening right now, but for example, OpenAI now has partnerships with Zillow and a bunch of other companies. And so maybe in the future, if you are searching for real estate on ChatGPT, the first results that you get served will be from Zillow rather than one of their competitors. Now, that is not happening yet that we know of, but I think it's very fair to question the sort of integrity of these chatbot results, especially as companies are spending more and more money to try to game them. All right, so I want to pivot just a little bit and talk about physical products. Have you tried any physical products that have AI features built into them, like a robot vac? What do you think so far? Any of them good? Any of them like terrible? Um, AI hardware is a little bit of a mystery to me. I don't know if you've heard of it, but there's a lot of AI hardware out there that's built into a robot vac. AI hardware is slower to happen than AI software, and so there actually aren't that many, like what I would consider good AI hardware products yet. I have tried robot vacuums, as you mentioned. I have two of them. Their names are Bruce Roos and Bruce Roos Deuce. They sort of use various forms of AI, but they are not like, it's not like they're powered by like ChatGPT or anything. It's like a just different kind of AI. I've tried the new Alexa Plus, which is their sort of AI-enhanced experience from Amazon, and that was pretty terrible because while it can do all of these cool things that the original Alexa could not do, like write you a bedtime story or synthesize some complicated topic for you or find you a recipe, it cannot do things like set timers reliably, which is arguably the best thing that the original Alexa could do. So they still need to do some work on that. I'm excited to try the new AirPods, which have the language translation AI feature built into them where someone can be talking to you in Russian or Japanese, and you can kind of hear them in your native language. Those are on my shopping list. And then I think we can expect some new kind of hardware experiments. OpenAI is working on something with Johnny Ive, the famed ex-Apple designer, but they have not released that yet. So I think we're starting to kind of move into the era of more interesting AI hardware. I know everyone in New York especially hates these like AI pendants, the friend pendants. Yeah, it seems so sad. It just seems sad that you would go home and watch a movie with this pendant and you don't have anybody to hang out with. Yeah, I don't know that I like this particular idea, but some wearable something having to do with AI does seem like it will eventually work. I just don't know what it would look like. Yeah, that makes sense. One thing I did want to ask you, I think it relates to the hardware a little bit, but more so for software, for the things like the chatbots. There's this saying, if you are not paying for the product, you are the product. You know, you subscribe, you pay, but I use a lot of these without paying a subscription. What am I giving these companies for their service? Just all my data. Yeah, probably for the free versions, your conversations are being used to train future, you know, generations of the model. You know, basically none of them are using ads in any real way right now. So it's not like you're using that, but they want you to pay, right? The sort of free versions, they top out after a certain number of messages. You don't get access to the most powerful models. And so their goal really is to like get you hooked so that you'll convert and become a paying subscriber. And right now, at least that's how they make most of their money. All right, you did mention Bruce Roos and Bruce Roos 2, and I have been wanting to ask you. Bruce Roos Deuce. Bruce Roos Deuce, I'm so sorry. Which one is your favorite? And what are your robot vacs? What are you using? So the original Bruce Roos is a Roborocks or one of these like disc, these circular vacuum robots. And the Bruce Roos Deuce is a newer one called Matic, which I believe Wirecutter has written about. Yeah, I saw it in action last week. Yeah, yeah. What do you think? So I like them both. I like all of my robot vacuum children. Can't choose a favorite. You know, they both have their pluses and minuses. Like Matic is a little quieter. The Roborock goes under furniture. I take a team approach to keeping my house clean, but the team is really struggling right now. I have a three-year-old and I have two very large dogs, both of whom shed. And so even with two state-of-the-art robot vacuums, my floors are constantly a mess. Oh man. So the Matic, I was in the office when that one was running last week and we have eyeballs on ours, like huge eyeballs on the front of it. Do you have eyeballs on yours? I have not yet installed the eyeball stickers. They send you a bunch of stickers, like you can make it look like a dog or you can put little like googly eyes on it. So I think they want this to feel less sinister and more cute. For listeners who aren't familiar, it looks almost like out of WALL-E. It's kind of like this white little boxy robot that will go around your house cleaning up. Yeah, it's cool. I like it. I think I have just assigned it an impossible task. And so I'm bearing responsibility for the chaos that is my house and all of the stuff that ends up on our floors. I'm trying to have some empathy for these poor robots who just have to go try to clean it up every single day. Before they like take the world over and kill all of us. They're gonna be so mad, by the way. They're gonna be like, you jerks made us clean your floors for all those years and you never said thank you. All right, Kevin, we have one last question we always ask our guests. So I wanna know, is there something that you have purchased recently that you absolutely love? This is a purchase I made literally yesterday. It is now in my house, which is the Wirecutter's recommended artificial Christmas tree. Nice, which one did you get? My experience with Wirecutter is that I am a sheep. I will buy- Do you just buy the first thing on the page? The first thing on the page. I try not to think about it too much. Okay, so I got, this is what I got. I got the National Tree Company seven and a half foot field real down-swept Douglas fir, literally the top pick on the site. And it came, we decided to go a little early this year because we needed some extra joy in the house. And it is delightful. It is lighting up my living room as we speak. So that is a good purchase that I am thankful for the recommendation on that. Well, Kevin, it was a delight to talk with you. I feel like I learned a lot and we'd love to have you back sometime. Yeah, anytime. Thanks so much for having me. That was Christine talking to Kevin Roos, tech columnist from the New York Times, co-host of the podcast, Hard Fork. It's a great show. You can find it wherever you like to listen. And if you like our show, we'd love for you to listen and subscribe to that as well, wherever you like to listen to podcasts. We'll see you next week. The Wirecutter Show is executive produced by Rosie Guerin and produced by Abigail Kiel. This episode was produced by Katie McMurray. Engineering support from Maddie Mazziello and Nick Pittman. This episode was mixed by Sonia Herrero. Original music by Dan Powell, Marion Lozano, Rowan Nemasto, Catherine Anderson and Diane Wong. Cliff Levy is Wirecutter's deputy publisher and general manager. Ben Fruman is Wirecutter's editor-in-chief. I'm Christine Cyr-Clisset. Thanks for listening. If there's like a robot, it's a robot. It's a robot. It's a robot. If there's like a robot around me, I'm definitely saying thank you. I say thank you to all the chatbots. I say, I speak so nicely to all of them. You have to. I'm just trying not to get killed. Exactly. We have to stay on their good side for when the uprising comes.