I gave Claude Code our entire codebase. Our customers noticed. | Al Chen (Galileo)

The minute I realized I couldn't really do my job was when I was trying to reference our public documentation and trying to provide an answer. It just still wasn't coming up with the answer that my customers were looking for. They don't want the docs answer. They want the step-by-step answer of how all these services cascade together. What I realized is that I can actually pull all of these repos into my VS code, and I can now use Claude code to ask our entire code base questions that are not answerable by our public documentation. And so sometimes I'll get really in-depth questions about, well, how does this feature actually work? And so I'll ask Claude code, look into the API repo, look into the AuthZ repo, and help me come up with an answer. If you can't find the answer, reference other repos within my directory, my root directory, and help me figure out the answer. And so that's the key unlock was when I figured out I could get way more in-depth, way more technical. And at the same time, myself, I can learn how our code base works. And how this has really helped me is I don't have to constantly ping our team engineering channel with, hey, what's the answer to this question? The customer just pinged me about this. And you can imagine engineers being really frustrated when I'm trying to post these questions. And then the customer asks me a follow-up, and then I'm posting that follow-up in the Slack thread. So I'm sure many of you who are working on the front lines of customers understand how that feels. But I've basically reduced all of that almost down to zero by pulling all of these repos into my local VS code. I'm really empathetic to this problem because I used to work at LaunchDarkly leading product and engineering, very technical product. We too had an architecture diagram that looked very similar to that. And again, as a more of a, you know, people think that CPOs, chief product officers, or CTOs are internal facing. No, no, no. We're salespeople. We're always salespeople. You trot us out and you put us in front of the customer or you put us in front of the prospect to answer the technical questions. And we had a diagram like that and I would constantly get these very detailed questions that required very detailed answers, like how does your caching work? And, you know, when you have seven layers of caching in your app, you can give the high level docs answer. But when you're sitting with, you know, an architect in the room or somebody highly technical, they don't want the docs answer. They want the step-by-step answer of how all these services cascade together to build a resilient caching mechanism, for example. And I just think how powerful is it to be in a meeting or in an email back and forth and not just sort of give this high level, but be able to query the current code base and really understand at a detailed level how it works. And I think current is very important because you know and I know this is always evolving over time. So even if you got the answer right, you know, a month ago, maybe your team shipped an update or maybe, you know, that method is actually out of date or the docs are a little bit out of date. And so I do think because the code base is, you know, at least your main branch is always the source of truth, it becomes a really reliable, you know, context set for you to answer questions about how the product operates. Yeah. And to quickly address your comment about how your code is obviously always evolving. I mean, we're pushing out, you know, multiple features per day, multiple releases. And one thing I've done, wrote this with cloud code is I have this script at my root directory that says like, I just do something called pull all. And I'm not sure if this is how other people do it, but it just pulls the main branch into my repo for all the repos in my root directory. So if I do this every day, I kind of get the latest code across like all of these directories on the left-hand side of my VS code. So the alternative, which I was doing before for like a few weeks, and I realized this is just asinine that I'm doing this, is doing Git pull origin main on every single directory. And it was just like not scalable because there's now like 15 different repos I have to pull the latest from. So that's kind of how I solved the code base is always evolving problem to make sure that I'm always getting the most up-to-date information for my customers. And I have to ask, did you just say Claude code, write me a script that git pulls all these? Yeah, yeah. I have no idea. This is, I'm opening up the script right now in my VS code. And it's like, what, 16 lines. Didn't have to write this. I just said, help me figure out a way to pull the latest main branches into my local repos. And it just did it in like one shot. Yeah, the other thing I want to call out for folks as I'm looking at your screen is I don't think people use this trick enough, which is in VS code, in cursor, and whatever your IDE is, loading a project at the multi-repo level as opposed to at the individual repo level, if you're trying to answer questions across the product is really important. So, you know, there's some like context bloat stuff that can come into sort of querying across all those repos and all those files. But it would be very painful if you had to go into each of these repos one by one and like query and then go into the other one and query. And so I like this idea of opening them all jointly in your IDE so that when you're querying it with Claude code or you're querying it, you know, with something like Cursor, it can have, it can go across, get traverse across repos and really give you highly contextualized answers. Yeah, our code base just happens to be in multiple repos, but I just pulled them all into this like giant Galileo directory here. And so everything is like at the same parent. But yeah, if you're in a monorepo, it could be, yeah, actually, I don't know how this would work with a monorepo because I've never done it with a monorepo with Claude code. But at least for us at Galileo, this is how it works. Well, I have many monorepos and yeah, you just open it at the right. I would, I would say my advice to folks is open Claude code or open, you know, your IDE at the right level. And sometimes it's narrow, and sometimes you need to go up a directory. And I think really thinking about that, and you can even do that contextualized to the problem you're trying to solve, right? And, and doing that, I think is really helpful. Could you show us just using Claude code, what kind of question you could answer with this code context? I will give you an example of, I guess I'm a big believer in using shortcuts too. So I use a bunch of custom Claude code custom commands to help me do stuff. So one thing I do a lot is helping my customers deploy Galileo into their VPC. So I have a custom command called DPL, which is, it actually references our, the first thing it does is it looks at our Confluence because we have a whole bunch of Confluence pages about how to deploy into Kubernetes using our different images and stuff like that. So I'll say DPL, my customer cannot use CRDs and they are using Google Secrets Manager and want to deploy the wizard image. Give me a step-by-step process on how to do it. This is actually not a super representative query because they're way more detailed than this. And I provide a lot more context, but I want Claude code to focus on looking at Confluence first because I know that we have a whole bunch of deployment stuff there. And then from there, if it can't find the answer, it will go off into all the different repos along the left-hand side of my Claude code VS code to find the answer. So right now it's just using the last C and MCP to pull information from Confluence and then marries that with like our code base to answer a very kind of in-depth deployment question. The one thing I'm not sure if we should talk about this now, but I started doing this in Confluence where we have a, we call it a customer quirks page. These are all kinds of all of our enterprise customers. You typically have air-gapped environments. So they have all these security measures and we have to abide by them when we deploy the product into their environment. And so I literally have a page that looks exactly like this where I have the customer's name at the top level and then a bunch of bullet points with like, you know, here are some things about how they store their secrets. Here's how they do namespaces. You know, here's how they handle sidecars and service to service encryption. Things I know nothing about, but as I'm meeting with my customers, I'm putting this all into this one Confluence page, ever-growing Confluence page. And then this is actually one of the core pages that goes into this DPL custom command, which is look at the customer quirks page. If I'm mentioning a customer that's on that page, look at all their quirks. And then in the response from Claude, it's highly customized, highly tailored to their environment because I've seen from working with our DevOps team that we can provide a generic answer about Kubernetes or about ClickHouse or about whatever for the customer. But it's like something you can just find online by Googling or using AI. But when it's tailored to specific security requirements and deployment requirements, it's way more effective and just gives the customer more trust that we know what we're doing, essentially. What I love about what you showed here, which The right Confluence doc, that was never, never the value add. Yeah, yeah, I mean, I think even if it's as simple as, hey, you came across a really great answer in Slack, like in a really engaging Slack thread, throw that into a Confluence page or save that Slack thread because I also use the Slack MCP to be able to summarize threads. So if you have like just some random, like this ongoing stream of consciousness of documents you want to have Cloud Code scan, I would just say, throw it into Confluence, throw it into Notion, throw it into Slack, whatever, and then that ends up being context you can provide to Claude when you are trying to ask it a question about a customer or about your codebase. Well, and the other thing, and this is maybe going back to how I introduced this episode, which is people use AI so much to compete on the field of the product and engineering velocity. And what I mean by that is like, we're all using Cloud Code to ship more product. We're all using AI and Codex to build, you know, better user experiences or more resilient backends or any of that stuff. But there's also a completely different competitive field, which is how you show up in your relationships with your customers. And I think, you know, what you're showing is you can actually use AI to invest and compete on customer experience. And, you know, my hypothesis is when your very complex enterprise customers have you show up and you don't just say like, here are our general docs to deploy this. And instead you say, I heard you, I understand what your needs are. And here are your custom docs on how you specifically need to deploy this. And I've already pre-thought about all the problems you've already told me about. You know, just looking like in a competitive sense, that's got to come across as a much more enjoyable customer experience on the receiving end and allows you to position yourself not just as a great product, but as a great team that's going to serve as your customers well. Yeah, I hope so. I mean, I think our customers, I think my customers are hopefully enjoying the answers I provide and the in-depthness that I provide. I think I've thought about taking this to the extreme, which is, I've certain customers who are like, you know, very in the weeds. They want to know things like right at this very second and I'm literally taking their question and just like saying, my customer then asked me this because they can't see your code, but me, how I can see the code, help me get the answer. And so if I take that to the logical conclusion, it's like, why can't we just share our repos with the customer? Because then they can just start querying our repos directly to get the answers they need instead of me as kind of like the quote unquote middleman. And, you know, the issue is that like our code is proprietary and all that kind of stuff. But I have seen, there's actually a case study from LangChain. And since a lot of LangChain's repos are, you know, it's open source, like their support agent bot actually does a lot of things I do, but it is able to query, you know, all the public open source repos and any of you out there who are trying to use LangChain or LangGraph, you can just pull all those repos down to your local machine and then ask questions, of course, using Cloud Code or Cursor or whatever. But I've come through that kind of thought experiment of like, I'm still kind of a bottleneck in terms of answering my customers' questions because I kind of like hold the keys to our code. But if they somehow had a sanitized version of it, then maybe they could just self-answer their questions too because they're also all using VS Code and Cursor and Cloud too, but they just don't happen to have our, you know, proprietary codebase. Yeah, I was going to ask you, are you worried that like the owl bot is coming and you're cut out of it? And I'm just curious how you think about then when like, again, like the highest order of view is not to be a pass-through, and I don't think you think of yourself as that. And so where does the human in these relationships powered by AI, you know, add the value? Well, I don't just blindly copy and paste the answers I get from Cloud Code to my customers in Slack or email or wherever. I still try to proofread everything and I actually do like try to make it sound more human. And you can then say the argument, oh, why don't you use Cloud Code to make your answers sound more human? And I think all of us know when we get an answer that's from the AI and it's, you know, things like, you know, you'll see like a bullet point saying like, in summary, here are the things you need to do to make sure your ClickHouse works within. So it's like removing things like that that just make it seem like it's from a bot just makes it seem more human. And we've actually, I mean, this is kind of going into going behind the scenes of how we work, but we've been dinged sometimes where the customer will say, can you just not give me an error response and just give me like a human proofread of it and tell me how it applies to me? Because typically the response is way too verbose. It has way too much information and the customer just wants to know, give me like the bottom line upfront. What do I need to know to like deploy this image onto my cluster? And so that's where the human, I still see myself as a human providing value and culling that down to what they actually need. And I would say, even for some of the more in-depth technical questions, I still try to get an engineer's perspective on it to make sure like Cloud Code is not hallucinating or not saying anything out of the ordinary. In my system prompt, I always, you know, in my Cloud Code, I say things like, don't make anything up, always cite your sources, point me to the line of code where you're getting this information from. But even with that, if I don't fully understand how this function works or whatever, I'm still pinging like the engineering channel to say, hey, this is what Cloud Code told me. Does that jive with what you're thinking? And there are times when I'm wrong or Cloud Code is wrong because engineering years have been thinking about refactoring into this new model, which is not captured in our codebase anywhere. It's just captured in like a meeting note somewhere or just like, you know, hallway conversations. And so those are the things that I'll never be able to query, let's say in Cloud. Yeah, I would say the other thing that, you know, where I see humans adding value and I say this all the time, which is like Rizz is the only moat, which is at some point, you know, people just want to have a face and a trusted personal relationship, you know, with the folks. And this is like my enterprise showing, but like with the folks that are selling them software, you want to know that you have somebody to call. You want to know that you have somebody that can gather the right folks around your team and your deployment. And, you know, you want to enjoy working with that person. And I will just say, I get a lot of, it is very fun for me to build with these tools, with AI tools, but I wouldn't say my AI colleagues are like the most fun to hang out with, which is like, I'm not like always looking forward to like my Cloud Code session. Like I'm going to really chit chat with Good Old Cloud. And I do think you still have that relationship with, you know, your human partners, your human colleagues, all that sort of stuff. And so I think there is a piece of that that's just not going to get cut out. And honestly, I gave this talk, I don't know, two years ago. I said, PM is dead. And people are like, well, what else should we do? And I was like, get into sales. Like that's not going away. Customer facing stuff is not, is not going away. So for anybody that wants to survive, you know, the incoming apocalypse, I do think customer facing roles and spending more time customer facing is a really important part of everyone's job. Absolutely. If you're working enterprise sales, like that is all people, handshakes, lunches, dinners. So that will never be replaced, I think by AI anytime soon. Well, you know, there might be a generational shift though here. I think as we sell, we'll see, you know, I used to say my, my joke in enterprise sales and the biggest, the biggest headwind to enterprise sales was I was starting to sell to millennials who like wanted you to text when you showed up at their door. They didn't want you to knock on their door like they used to. We'll see, we'll see how enterprise sales changes generationally. This episode is brought to you by Tines, the intelligent workflow platform powering the world's most important work. Business moves faster than the systems meant to support it. Teams are stuck with repetitive tasks, scattered tools, and hard to reach data. AI has huge promise, but struggles when everything underneath is fragmented. Tines fixes that. It unifies your tools, data, and processes in one secure, flexible platform, blending agentic AI, automation, and human-led intervention. Teams get their time back, workflows run smarter, and AI actually delivers real value. Customers now automate over 1.5 billion actions every week. Tines is trusted by companies like Canva, Coinbase, Databricks, GitLab, Mars, and Reddit. Try Tines at Tines.com slash howiAI. All right, so we have just to recap, we've tools, processes, and the tools that help you create those things. And so when it comes to, is this the Al way of doing things? Yes, it's my way, but I'm also very, probably one of the more opinionated people on the field engineering team about like how we should be doing things in terms of talking to customers, answering their questions, and pulling in the right context. And so I've told multiple people, like, pull all the repos into your local machine and have CloudCode run an init command to index the whole code base or whatever. And I'm just like constantly sharing these tips and tricks to my teammates to make sure they're also functioning at their capacity. So it's my way, but I would say I'm also very opinionated about how we should do things because I've done things the hard way, the manual way, and this way to me is like just 10 times way more productive. So we don't have like a specific, like, oh, because Al's doing it now, the whole team has to do it. It's more just like, people show, here's the problem I had. Here's the results I had with CloudCode or whatever. And here's why I think you should adopt my solution. And I'm constantly having that conversation internally about like, how do we break out of certain processes that I think are slowing us down and how AI can be infused into all those processes as well. Well, and now you're sharing to all of our How I AI audience on how they can do that. So you're having more impact than just on your team. All right, well, so to just recap again, your code is your source of truth. It can help you answer customer questions. It can help you document customer solutions. You can also do that with other channels like Slack and then like create these virtuous loops of solving a single customer's problem and then a system to solve that problem more scalably across your entire customer base for yourself and for your teammates. Very, very high impact episode. I think people are going to have a lot of takeaways from this one. Super practical for all my friends that are customer facing out there on things they can do starting tomorrow to use AI to give their customers a better experience. Let's jump into lightning round questions. And I have one that's really top of mind, which is, it seems like you have a very healthy culture at Galileo, but I can imagine teams, especially engineering teams that are like, oh, no, no, no, no. I don't really want the customer facing folks like going into our repo, querying it, and then just YOLOing answers over to our customer base, especially in a more technical product that really requires deep technical understanding. I think you've proven that there's a lot of value in doing that, but what would you say to those teams that are a little bit more hesitant about ungating access to the repo to non-technical roles? I think from the engineering engineer's perspective, I would look at it as how many, I would try to think about how many times in the last week, in the last day, have you been asked a last minute question on Slack, a last minute DM ping mentioned in a thread where, how does this thing work? Where is this? How do we make sure that this is functioning the way it should be? And you're constantly the source of, you're the bottleneck for answering that question. And if you provide a system kind of similar to what I have to your customer-facing team, then you kind of just take away that toil and the constant like uncoziness of like answering these random product and engineering questions that is already in your code base or maybe it's already living in your confluence or something like that. So I think that's really the biggest takeaway for me is how much of your time is being sucked away from your customer's team because they don't have access to the code. And I mean, I think there's some no-code. I mean, I think you can maybe pull in your code into cloud co-work, which is a little more no-cody and other kind of like more no-cody ways of doing things. But I think what I've shown is I think the most performant way of being able to pull your code and get answers out. So I think that's kind of, from an engineer's perspective, how much time can you save and then also how more effective your customer-facing org can be. And I think the corollary to that is that our field engineering team is very technical. And so maybe you increase the hiring bar for your customer success or customer engineering team to feel comfortable using GitHub and pulling repos into your local machine. And so that could be today if they're not technical, just doing like a simple tutorial or enablement session on how do you use GitHub? How do you use Git commands? Things like that. And there might be some self-learning you have to do on the side too, but I think once you're, once you have your environment set up, that's always like the hardest part about this whole exercise, getting your environment set up. Once that thing is set up, then using cloud code is just like using any other AI chatbot. So I think there's like a few different ways I'd approach it from to democratize access to your repos. One of the things I was going to say is I often tell people, this is the era of the hard skill, which is no matter what role you're in, sorry, babe, you got to like learn a little bit how to code. You have to learn a little bit what Git works like. You have to be okay opening up some code you don't understand in an IDE because that's just going to be the substrate by which we communicate for the next three years. It's going to go like closer and closer to the code because these LLMs are extremely good at understanding code. And so I think across the board, people just need to become more technical and develop hard skills around code, even if your job is not code. I think the second thing that I tell people is there's no better time to learn how to code. Truly no better time to learn how to actually code. And I think people that are shipping with cloud code, but not using that as an excuse or a support to learn some fundamental software engineering concepts are missing 50% of the value. Like I had to teach myself how to code out of a book, like literally out of a book. It was, I had a book open and then I would look at my single screen because none of us had two screens. That would be crazy. And I would like read the book and I would type the book in the word, like the words in the book in code and press enter and it would say hello world. And that was my life. And now you have this like magic, super patient, infinitely wise, you know, like teacher in your computer that you can use to learn to code. And I think, you know, you talk a little bit about Kubernetes and how you, you scaled up on that. So I'm curious your thoughts on just up-leveling technical skills using some of these, these tools. I think the meta takeaway is like, you just have to be curious about like how things work. I can't really say anything else besides that. It's kind of like, I come from that same world too of like looking at a book and then I would say the graduation above that was knowing how to write a good Google query and then going to Stack Overflow. And then how many of you resonate with this where you go to some Stack Overflow Q&A, you copy and paste the code. It doesn't work perfectly, of course. So you're Googling the error you get from that thing you copied and pasted. And of course everyone on Stack Overflow is super snarky and it's like not a kind of healthy conversation. And then to your point, you have this like infinitely patient, infinitely kind assistant who never gives you the wrong code snippet from Stack Overflow. It's always like tailored to back to everything I'm saying. It's tailored to your needs, to what you want. And then if you go the extra step and like what you said, and then what, if you go the extra step and say, okay, thanks, Claude. You told me this is the answer. Tell me why this works. And then you start getting into Kubernetes and into the deeper in the weeds things. But of course you're not going to know everything right off the bat, so you can say things like, oh, explain to me in simple terms. Explain to me like I'm five. And so you just kind of pulling on that thread. And I sometimes do get lost going down the rabbit hole, but I've never found a situation where not going down that rabbit hole does not help me in my day-to-day job, especially in AI where everything is moving so fast. Yeah, and this is just to make everybody feel comfortable. This is not just a beginner thing. And I find myself doing this with GPT-5.4, which is like a powerhouse model and also like talking to the most esoteric senior software engineer you've ever met where it like explains its plans in these very technical terms. And I'm like, dude, just like explain to me what you're doing in number one. Tell me in plain language. I do not need the technical details. Like just tell me in plain language. And again, it comes from this curiosity of, I want to make sure I understand the fundamental concepts of what you're talking about. And I want to make sure I'm learning both my code base and general principles as we go. And so I do think that curiosity mindset, no matter what your seniority level is, your experience with technology, you can always learn. Learn something better. Okay, my last question before we get you out of here. When AI is not giving you the right answer, it's giving you AI slop that it wants to email to your customer. What is your prompting technique? I'll say one thing first off the bat is like I'm very