Charity Majors: AI and the End of the Three Pillars

Matt Klein: Welcome to another episode of Beyond the Noise: Signals, Stories, and Spicy Takes, the show where we dig into the stories of the people shaping the future of app-based computing, with a special focus on mobile. I'm your host, Matt Klein, co-founder and CTO of bitdrift, as well as the founder of Envoy Proxy. Each episode we'll talk with engineers, founders, and technical leaders who've transformed the way their companies build and understand what's happening inside their systems. We'll dig into the challenges, the breakthroughs, the lessons learned, and we'll wrap it all up with their hottest takes. So let's dive in.

Today I'm with Charity Majors, who needs no introduction, but is the co-founder and CTO of Honeycomb.io. She pioneered the concept of modern observability, drawing on her years of experience building and managing massive distributed systems at Parse, which was acquired by Facebook, and at Linden Lab, where they built Second Life. She's the co-author of Observability Engineering and Database Reliability Engineering, and she loves free speech, free software, and single malt scotch. Welcome, Charity.

Charity Majors: Thanks for having me.

Matt Klein: Before we get into it, what's your favorite scotch?

Charity Majors: Oh boy, that's like asking someone to pick their favorite child or their favorite cat. I'm a big fan of the dirty, peaty ones. Lagavulin, Laphroaig. Ardbeg does a bottling every year that I go out of my way to acquire. They did the Peat Monster, they always give it some jazzy name that's very much in contrast with the rather staid world of scotch.

Matt Klein: I'm more of a Highland person myself, but I do enjoy a peaty scotch occasionally. Where I like to start is with people's backgrounds. You've had a really interesting tech career, and obviously you're super well-known in the industry, but I'd love to hear how you got into tech in the first place. And I know you had a pretty interesting childhood, if you're willing to share.

[00:02:19] NANO FAMOUS AND GROWING UP OFF-GRID

Charity Majors: I try to explain to my parents sometimes. They're like, "So you're well-known?" And I'm like, "Mom, I'm nano famous. There are like 300 or 3,000 people in the world who think I'm amazing, but nobody else knows about me."

Matt Klein: I call it nerd famous.

Charity Majors: Nerd famous, yeah. You and I are about the same age, I think, and the time we entered tech was such a weird, strange, interesting time that we kind of romanticize and also kind of gloss over. It was a time when if you were smart and curious and had access to an internet-connected device, you could make a great living. The tech industry is the only growth industry of my lifetime, and there's a sense in which we've been living on a different planet than the rest of the post-industrialized economies.

Kids these days ask me for advice on how to get into tech, and I'm just like, "I don't know. I have no idea what to tell you." In general, people should try to get advice from people who were in their shoes five years ago, not 20 years ago. To some extent, this is a good thing. When we got into tech, it was a toy. Now it runs nuclear power plants, so it makes sense there's a higher bar to entry.

I grew up homeschooled on a sort of fundamentalist compound in rural Illinois, then Northern Idaho, the scary parts where all the white supremacists went during the Clinton administration. We grew all our own food. I would wake up at 5:00 AM to weed endless rows of garlic fields. To this day, I understand work-life balance with my head, but part of me doesn't believe it's really work if I don't have dirt under my fingernails. It just feels like it could be a dream and could all go away and I'll be back to subsistence farming.

Matt Klein: If you're a startup founder, speaking from experience, you certainly have to get some dirt under your fingernails.

Charity Majors: That's very true. Talk about roles that are glamorized and are really not glamorous. Taking the trash out, for sure. I'm the oldest of six, and I left home after the twins were born because I just couldn't take it anymore. I was like, "I've done all the parenting I'll ever do. I've pulled all the weeds I'll ever pull. I've washed all the dishes I'll ever wash. I'm out of here." The most rebellious thing I could think of to do was go to college. I got a piano scholarship when I was 15, and I just left. That was right when I think the college campus was getting wired for ethernet the year I showed up.

Matt Klein: You went to college in Idaho?

Charity Majors: Moscow, Idaho. It's like the Berkeley of Idaho. They actually have a really good CS department and they have the Lionel Hampton School of Music. There's maybe one Black band in all of Idaho, but somehow he founded a music conservatory that's quite good. I went for piano, but I had serious ADHD and I was working my way through college with three or four jobs. I developed this theory that I should be able to just stay awake Monday through Friday and sleep on the weekends. Just go from job to job. Mind over matter.

Self-medicating through lack of sleep is something I did all the way up until 2020. Not doing that, and having actual medication, has changed my life. So many details of the early days of Honeycomb are pretty fuzzy in my recollection because I was literally sleeping every two to four days. I'm pretty sure I picked a fight with you on the internet, and we were not friends for a while.

Matt Klein: Most likely, but that was a long time ago.

Charity Majors: That's been my self-medication, picking fights. It gives you all this dopamine, this juice and energy. Then I realized I would struggle to focus, and I sort of hacked my way around to, "Oh, I can force myself to focus on things I have no interest in if I've been awake for over 24 hours."

Matt Klein: That does not sound healthy. I'm very hardcore about sleep. If I don't sleep well, it's bad news. Hard for me to imagine actually not sleeping for that long.

Charity Majors: It's physically painful. I've been pulling some all-nighters recently and I'm just like, "I used to walk around like this all the time. It hurts. How did I get anything done?" I genuinely don't know.

[00:07:53] FOLLOWING A CRUSH INTO THE COMPUTER LAB

Matt Klein: Let's go back to college. Obviously you were there for piano. How did you get into tech?

Charity Majors: I had a crush on a boy and I followed him into the computer lab. This is another detail that's both revealing and a little embarrassing. I was like, "Oh, there are no women here. Great. This is where I belong." I just decided I was a straight white dude and was going to make it happen, much like I was just going to decide not to sleep and make it happen. Subconsciously, I also saw that people who graduated with music degrees were still kicking around the music department in their 30s and 40s, working at Subway and picking up odd jobs as teachers and accompanists. I grew up dirt poor. I did not want to go on being poor, and this whole computer thing seemed like it had legs.

I've always loved words. My path into tech probably never would have happened if it wasn't for the Unix command line. I could stay in the computer lab all night hitting Tab-Tab and trying all the commands. It was fascinating to chain them together. They were so descriptive and evocative and rich and powerful. I got a job jockeying backup tapes and doing web server maintenance for the math and stats department, and the next year they gave me root for the whole university, something I'm sure they no longer do with undergrads. Then I got a job in town, and pretty soon after that I moved to San Francisco, partly because it was really hard to try to work my way through school. This is another place where I reflect with horror on how lucky I got, because if I had left even five years later, it would not have been possible for me to work my way through school on minimum wage jobs. That door was open very briefly.

Matt Klein: Even public university now, the costs have gone up so much.

Charity Majors: When I was working at Facebook, my baby sister Claire decided to go to college. She called me up in tears asking for a loan, because one of the interesting bits of color about my family is that my father revoked his and my mom's Social Security numbers. He doesn't believe income tax is constitutional, part of the Patriot Movement, and he's been fighting with the IRS ever since. So Claire didn't have a Social Security number, and she didn't have any record of her parents' income. She literally couldn't work her way through school.

Matt Klein: I heard a story on NPR not that long ago about someone in their 30s who didn't have a Social Security number, and the way the laws work, it's like that person doesn't even exist. To get a number after the fact is almost impossible.

Charity Majors: I can imagine, that's a very difficult situation. But I loved computers, and it was very clear. My first year in the Bay Area, when I was 19, I made more than my father had made in three or four years before. I did go back and try to finish my degree once and, pro tip, you can't actually live in San Francisco for a year and then go back to rural Idaho and finish college as a teenager. It just doesn't work.

[00:12:14] SAN FRANCISCO, LINDEN LAB, AND EARLY STARTUPS

Matt Klein: You wound up in San Francisco and worked at a series of startups. Tell us how that formed you into the engineer you are today.

Charity Majors: I was really lucky. My formative working years were spent at Linden Lab, which made Second Life. There was a moment in time when Second Life was on the cover of Time and Facebook was this weird little thing colleges were using. It's addictive, it's fun, it's a roller coaster, it's exciting. The people at Linden Lab were just good, heart and hands, genuine. In retrospect, there were a lot of women and trans people there, and it was just in the water. A lot of women haven't had that experience in tech, so I try to keep that in mind.

Then I left to go to another startup to do mobile stuff, and that was my "oh wow, not all jobs in tech are great" moment. I never really connected with my coworkers. I stayed for a year and a day. It was buzzy, it was mobile, right down Palo Alto. I stayed for a year and a day because I thought I owed them that. Then I went to another job that was terrible in exactly opposite ways. The technology was old, they were using Perl, doing things by hand. They thought I was great, and I was just like, this is actually the one time in my life I've considered leaving tech because I'm so bored. I stayed again for a year and a day because I thought I owed them that. Kids, if you're listening, you don't owe your employer a year of your life. If you hate it, leave.

[00:14:35] WHY WORK ISN'T FAMILY

Matt Klein: I was going to ask you about this. When you and I were coming up, there was always this sentiment of "work is your family." I think that's total bullshit. How has your feeling on that evolved? You're an employer now, you employ many people. How do you think about it?

Charity Majors: We could talk for an hour just on this. The neural pathways we lay down for family are the first and the deepest, they're grooves. The human brain is a fairly efficient organ, it reuses those pathways. I went through a stage of "work is family." I went through a stage of "anyone who says work is family is terrible, you're a bad person and you should feel bad." Now I feel like at a good job it does kind of feel like family, but you don't say that. You don't place that expectation on someone, and you especially don't do it from the top down, because that guilts people and pressures them. It's not appropriate to ask people to commit and sacrifice for work the way you sometimes sacrifice for family.

But it doesn't mean we don't sacrifice for work. The nature of the professional relationship is: I give you money, you give me output. That's the basis of it, and you can't forget that. But you still, it doesn't mean you aren't friends with these people. It doesn't mean you don't think of them fondly, or get wrapped up in their successes and their failures and their longings. It's dishonest to pretend that doesn't happen. It should happen. But especially when you're the employer, it's very incumbent upon you to be clear about what you're asking for and not exceed what's appropriate.

Matt Klein: I feel very similarly. Working environments should be respectful, people can be friends, all of that. But we're naive if we don't recognize there's an exchange happening. It's a transactional exchange, and if the exchange is not favorable for both parties, being "family" isn't going to save you from that. When you and I were younger, a lot of people were delusional about it, in the sense that you should sacrifice because we're family even when the transactional balance is way off.

Charity Majors: Totally. When I look back on Linden Lab, I remember a specific moment after I'd been there three, three and a half years. We were really proud of the fact that we couldn't remember anyone ever leaving. No one had been let go, no one had quit. Looking back now, I'm like, "That was not good." We had this whole kick about, "we don't need managers." Some things keep coming back like a bad sweater. Everyone chooses their work, which meant it was really frustrating for people trying to get things done and having to convince every single person that it was worth doing. Exhausting, lack of direction, lack of focus.

When I look back on my time at Facebook, I think it's been long enough that I can talk about this now. I was really proud of keeping my big, fractious, brilliant team together. But I spent probably 75 percent of my energy on one particular person, and I felt really proud as a manager of my ability to keep that team on track and intact. I look back now and I'm like, "What else could I have done with that very large slice of my energy?" And I look back at the lessons that person didn't learn and had to learn later in life, when it was much harder.

[00:19:58] SRE, GRAPHS, AND BEING DRIVEN

Matt Klein: You've written a lot about management, but you're also known as someone in the observability and reliability space. Where in your early career did you start caring about that? You were operating systems early on at Linden Lab, then Parse and Facebook. How did the interest come from, both the manager path and the engineer path?

Charity Majors: I tried being a software engineer. That was my title when I was 18 or 19, and I could not focus on it. "You mean I know what I'm supposed to be working on, not just today and this week but next week and the whole month and the month after?" I couldn't. So I went toward operational stuff where there are always fires. I've always been at my best when everything is fucked, everyone's freaking out, and I just get dead calm and I'm like, "This is fine. It's fine. We'll get through this. Boom, boom, boom." I don't think that's uncommon. The neurodiversity angle is what pushed many of us toward SRE.

But it's a little ironic, because I've always hated monitoring and I've always hated graphs. I don't think visually very well at all. My friend Ben Hartshorne, who I worked with at Linden and Parse, and who was our first employee at Honeycomb, I used to joke that as soon as it was time to build graphs, I'd bring Ben in because I don't like dealing with them.

It's always been rage. Similarly with, and this is something I sometimes reflect on and think is not my favorite thing about myself, I don't know what it would be like to be like, "Goodness, this new programming language or AI thing seems interesting. Let's go play with it for a day." That is not how my brain works. I am so driven by "this must be done, this is existential." Whatever it is, I'll go do it and have a great time, but it's driven. There might be a link there to ADHD, and a link to a very hard Scrabble upbringing. I don't know.

Matt Klein: I honestly think what you're describing is a shared trait among many successful people in our industry. I'd call it impact-driven and not process-driven. I feel the same way. I have opinions like everyone else about technologies and programming languages, but honestly, I don't give a shit. I care about the end result. I care about the customer impact.

Charity Majors: Totally. Same for management. Every time I went into management, I did it just like, "Well, somebody's got to do this, and somebody's got to do it right, and I don't trust anybody else." So I went and did it.

Starting a company, though, I've never been one of those kids who was like, "I'm going to start a company someday" or "I want to be a founder." If anything, I kind of despise people who feel like they're too good to work for someone else. It's not a character trait I like. I blame YC for a lot of the distorted expectations and almost work upbringing of many smart, talented young kids who are told to drop out of school and start a company, or 22-year-olds who are like, "What do you know about the world?" The cynic in me thinks the reason YC cultivates this is because they're so young and impressionable, and they'll do whatever their mentors tell them to do. Most companies fail. All companies fail eventually. Then you've got this 22-year-old who's now 28 or 29 and has no skills and thinks they're too good to work for someone else. It's not good.

[00:24:22] STARTING HONEYCOMB TO GET OUT OF BURNOUT

Matt Klein: We started to talk about you starting a company. You just said you hate graphs, but at least when you started Honeycomb, you started a company to build graphs. How did a person who hates graphs start a company about graphing?

Charity Majors: It was just so bad. First of all, I had the opportunity to start a company. When I was leaving Facebook, my work history was startups nobody really knew or cared about. Nobody knew who I was. I was on Twitter a little bit, and then our startup got acquired by Facebook. When I came out of Facebook, for the first time and I assumed the last time in my life, I had the opportunity to start a company. There was a seed stage where it was like, "Would you like a million or two?" And I was like, "I accept on behalf of all women and queers and dropouts everywhere. Yes. Absolutely. I'll try this. We're definitely going to fail, but it's a fun way. I feel burned out. This feels like a great way to spend a year in the corner writing code."

Matt Klein: That is the best quote I've ever heard. Start a company to get out of burnout. I'm going to keep laughing about that.

Charity Majors: You're entitled to your laughter. When we were thinking about what to do, we'd had this experience at Facebook. Parse was a little Ruby on Rails multitenant platform, basically Heroku for mobile. So we were having the problems that characterize pre-AI, microservices and complexity, a couple of years before everyone else got hit with them. You don't know which app is going to hit the top 10 in iTunes, you can't talk to the developers shipping these apps, you just have to understand the code on its merits. That's something we're all dealing with now with AI, right? You don't get to access the person who wrote the code and ask them what their intent was. You just have to figure out how to make it work.

With the tools we had, it was impossible. When we were at Facebook, we started trying some of their tools, and the experience of using Scuba and applying it to these problems was mind-blowing. We could blow a day just visually scanning logs, mapping stories.

Matt Klein: For people who don't know, could you in 60 seconds describe what Scuba is or was?

[00:27:14] SCUBA AND THE THREE PILLARS

Charity Majors: Scuba is the grandmother of Honeycomb. Pretty much every observability tool out there is descended from infrastructure tools. Metrics, logs, and traces, the three pillars of observability. I hate that phrase so much. It makes sense for infrastructure because in infrastructure you're trying to operate a lot of code you didn't write and can't change. You just have to take whatever comes out and stash it someplace cheap. There's some insight in there, but not a lot, and you just have to put it somewhere. So you put your metrics in a time series database, your logs over there, your traces over there, your errors and exceptions over there. You put everything in a different place.

Scuba is much more like a hybrid BI tool. It has high cardinality, high dimensionality, you can have as many attributes as you want, you can slice and dice, and you can zoom into individual requests. You can zoom out to the whole world. When we started feeding the same datasets into Scuba, all of a sudden we could go, "Oh, the requests from that app ID, they're only sending 10 requests per second, but every single one of them is timing out at 30 seconds, and it's saturating the lock percentage on that." You could immediately see what was happening instead of having to know what was happening before you could find it. I could not imagine going back to a world where I didn't have that. So we decided to build it.

Matt Klein: The industry thanks you for that.

Charity Majors: I don't know, Matt. I never would have predicted the three beautiful pillars would still be everyone's entry point at this day and age.

Matt Klein: We could spend an entire hour on that alone, and I'd honestly rather talk about the AI stuff, but quickly: I think you're hitting on a point where in our industry it is hard to move people. You and I operate in the high tech vanguard and we work with tech-forward companies, but you look at the long tail of how people do things and there are still mainframes out there. People are set in their ways. From the bitdrift side, we're firmly a log-forward company with high cardinality and all of that, so I completely believe everything you're saying. But it's so complicated at the intersection of legacy technology and cost and everything else.

Charity Majors: My 60-second summary of why we are where we are: historically, infrastructure teams have given software engineers the tools and said, "Here, this is what you get, you don't get to have opinions." Infrastructure teams deeply need their three pillars because they can't merge them, they don't control the software. And software engineering teams aren't used to treating telemetry like a product decision, but it is a product decision, because the loop of creating value is: you build something, you look at it, you understand it, and then you act again based on what you see.

There's so much inefficiency in the current state where you've got two different feedback loops. You've got the build-test-merge loop, and production plays no part in it. On the other side you've got the operational feedback loop, and these two feedback loops will endure for as long as software does. It's not about who writes software and who operates it, it's about the tension between change and stability. Software engineers don't even know what changes they're introducing. The threshold for operational action is high and has to be. We just don't even understand most of the changes we pump out there, because the operational feedback loop only gets kicked off when someone complains loudly enough or an error emits that threatens the entire stability of the system. So my hope is, to segue, that AI clears the playing field a bit and reminds us of these first principles.

[00:32:19] AI AND THE INTERCOM 2X CHALLENGE

Matt Klein: That is a perfect segue. In some ways it maybe resets things. In other ways, I think it just amplifies all the problems you just described. GitHub is going down basically every day now. We, meaning us and the machines, are outputting more code and running more GitHub actions. The rate of code we're producing is growing exponentially, and we can have a whole conversation about the quality of that code. Everything is fundamentally changing right now, so I wanted to pick your brain. As an engineering leader, how is Honeycomb using AI internally? How are you observing your own systems? And from an industry perspective, what's happening?

Charity Majors: Such a great question. If I had a little Hermione Granger time machine, I would not have chosen to write a book last year. It ended up taking me offline at some really crucial moments, but however it's done, it'll be out.

Matt Klein: Congratulations.

Charity Majors: I kind of emerged from my book-writing hole in February and went, "Oh my god, I went into this hole in November and everything is so different." I started by doing an internal tour, just having conversations with engineers, then writing notes, stewing over it, and posting internally to talk about it. Christine and I, I don't know if you read the Intercom 2X memo from last summer. I've long looked up to the Intercom engineering team. Darragh, their engineering leader, is a beautiful writer. A few years ago they published "Shipping is your company's heartbeat," and I love that. It should happen automatically, almost without thinking. It should be weird if it stops. They've consistently been six to nine months ahead of us as an engineering org, and they issued a challenge to their entire company to 2X their productivity with the tools they already have. They said it in public, drew a line in the sand. Brave.

Christine and I stewed over that. We'd been suggesting people try using AI themselves. As a personality trait, I'm deeply anti-mandate. As soon as somebody mandates that I do anything, it's the last thing I want to do, so I don't like inflicting that on others. But we realized we kind of had to, because that's how you resource things, that's how you tell people, "No, this is important enough that we need managers to make time for this. We acknowledge productivity will go down in order to go up later." So we issued a mandate. We were like, "We think this is reasonable, we think Intercom has a great track record, we'd like to do this too."

It did the job it needed to do. It pushed us all to get out of our comfort zone, to dive into the deep end. There's a lot of ops DNA in Honeycomb: the cranky, cynical, poking-holes, "that sucks, we don't like that" energy. This is one of my personal traits too. What we said was, "You don't have to uncritically love AI. You're allowed to hate it, but you need to hate from a place of expertise. You need to be an expert in order to say, this is why I choose not to do X, Y, Z."

When I came out of my book tour and I was doing this internal tour, it was really interesting. Intercom recently published a follow-up with all these beautiful stats. They said, "We said we'd do this in a year, it's been nine months and we've already 3X'd." And I'm like, "You guys are amazing." We are not quite as, we don't have quite as many resources as you, and I don't think, I wish I could write a blog post, I don't think we quite have the...

[00:37:26] MEASURING PRODUCTIVITY

Matt Klein: Let me ask the obvious question, because it's probably going to come up anyway: 3X what? What does that actually mean? Tokens spent? Code?

Charity Majors: Matt, you sound just like all of my engineers.

Matt Klein: I'm coming at this from the perspective of someone who hasn't written a line of code myself in about six months. I'm all in on the tools. But as with everything else in our industry, we like to make up a lot of useless metrics. So as a manager of a large team, how are you thinking about this?

Charity Majors: I understand why Garry Tan is like, "Well, I committed 30,000 lines of co-." And I'm like, I thought Bryan Cantrill's essay, "Your Intellectual Fly Is So Open" or something like that, was perfect. The thing is, the Intercom post showed that the metrics they're looking at, and we are looking at a specific basket of metrics, the answer isn't don't measure things. The answer is measure more things. Measure a basket. It's almost like the rigor you introduce into the system by asking yourself consistently, what is the outcome we're trying to move here? What does productivity mean to us? Sometimes it's speed. Sometimes it's trying more experiments. Sometimes it's quality. Sometimes it's creativity. There are all these other things, and the answer is yes to all of them. Which ones matter now? That muscle is underdeveloped in all of us.

Matt Klein: It's not even about AI. Our industry is horrible at things that aren't easily measurable. We like to measure everything, but not everything is easily measurable. That's a whole separate topic.

[00:39:15] VIBE CODERS AND WHAT ACTUALLY SHIPS

Charity Majors: That's super true. There's a blog post I'm really hoping I can find time this week to spit out. I've gone to a few little conferences recently, and I am so over the triumphant vibe coder. "We rewrote, we vibe coded the hardest, gnarliest part of our blah blah blah in two weeks and blah blah blah." I know engineers who work at your company. I know that's not exactly what happened. I know that's not the whole story. I know people quit because they got handed a pile of shit.

Matt Klein: Or, did you ship it to production? Did it work? That gets into the second part of what I was asking. You can code all day, but if it doesn't work, what is it doing?

Charity Majors: My favorite in this genre was the journalists who vibe coded a clone of the Slack app in an afternoon and produced it like, "Oh my god, all programming jobs are at risk." Literally that is not what using it means. The mental model of how to interact with that app, nobody wants to wake up and find the buttons on Slack move around slightly from day to day. The hard part is building an app that a billion people can use and that is stable.

Matt Klein: Not to mention all the hidden stuff around compliance, security, auditing, APIs, the millions of things. I don't want to harp on that, because I agree the discourse is as stupid as you're saying it is. I'm more curious about your internal usage and productivity, and I'm assuming you're using it to help with operations of the Honeycomb service itself. But where are you seeing the industry going, both with the massive code that's being produced and how you're seeing your customers use these tools to actually create business outcomes? Because at the end of the day, that's what matters. And you're at the epicenter of this. Talk about how observability is changing in this new world.

[00:41:35] WHERE OBSERVABILITY GOES NEXT

Charity Majors: That's a very large and very good question. One of the most interesting things I read recently was from Kyle Forster. He posted this very brief thing about how their little AI SRE agents in production can only talk to the three pillars about 30 percent of the time. 70 percent of the time, they on their own go out seeking richer sources of data. They go upstream trying to find data that hasn't been split up. They look for all the context to be intact. In the preface of the second edition of the book, I felt like this is a running joke of 2026: oh, you mean we need fast feedback loops? Oh, you mean we need context? We're rediscovering all these first principles, only we can't patch over the lack of them with our intuition and stupid human tricks anymore, because we're not necessarily in every loop.

I think that's why the three pillars have endured as long as they have, because humans have always been there to go, "Well, I have a feeling that log line is probably that metric." Close enough, good enough for government work. If our wisdom isn't encoded into the system, it doesn't exist. I choose to interpret this as something that's very exciting, because I think you and I both feel strongly about a lot of engineering best practice. It's always struck me that our so-called thought leadership has been so much more widely read and cared about than our product, when in my opinion they're the same thing. The product is an emanation of our engineering values. Our engineering values come from building the product and watching people use it. The reality is that without AI, they are not within reach for the median engineering team. They're barely within reach of the top 5 percent.

Matt Klein: Sorry, what's not in reach?

Charity Majors: The feedback loop of instrumenting your code, going and looking at it after it ships, understanding what happened, all the tools and the craft of code crossing over from development to production. Development doesn't end and then production starts. Production is more like the fourth trimester. That's when your code is actually tested for the first time. But a vanishingly small percentage of developers working on anything of size or scale have the tools, because it has taken so much engineering discipline to arrive at that state.

Matt Klein: Setting aside that this is a good business opportunity for observability companies because we're spitting out so much code that people have to figure out what's going on, I am legitimately excited. I'm one of those people who doesn't think AI is going to lessen the number of engineering jobs. I think we're going to have just as many engineers doing just as many things, shipping and debugging things. But I'm excited about this new world, and exactly what you said: I think the AI tools, as they evolve, will allow best practices to be practiced by more teams versus just the elite teams that can do it today through their mental knowledge.

Charity Majors: Yes. In the past, it's been like you have to be a high-performing team in order to be a high-performing team. If your company has ever gotten into a certain state of tech debt, you never recover. It's just not practical.

Matt Klein: Without sharing internal details you might not want to share, do you feel your product is going to have to evolve for this new world?

Charity Majors: Very much. By the time this comes out it'll probably be old news, but we're doing a big release extravaganza in a couple of weeks. At a high level, the past has consisted of a lot of silos. Operations teams look at these silos, developers look at those. Everybody's capturing another copy of production but in a slightly different format using slightly different details, and we're paying to store that 20 or 30 or 50 or 100 times. It's less great than the sum of its parts, because every time you fragment it, you decrease the value and usefulness of any individual segment.

The beautiful thing about MCP, and the impressionistic output of AI, is that some of our biggest MCP users internally have come from sales, marketing, and customer success. They're just in conversation with the data all day. It's approachable. I think we're increasingly going to see it move to a unified data lake. Those relationships between data are actually the most valuable part, and when you carve it up before you write it out, you can never get them back again. So preserving those relationships and moving the work of interpreting to clients or interactions or wherever, making it so that you don't have to be an expert in your tools to be an expert in your company.

Matt Klein: It's so interesting from a product perspective, but even with our reliability hats on, just the rate of queries is about to go through the roof. We're already seeing this. How do you build your system to enable that?

Charity Majors: And even just in-job, they're just boom, boom, boom, boom. I want to iterate so fast. It blows my mind that there are people out there who wait minutes for a query to return, but that's still the state of the art in most of the industry.

[00:48:59] DON'T STAY ANXIOUS. JUMP IN.

Matt Klein: We're basically at time, even though we could talk for hours. Any last words you'd like to leave us with, either on AI or otherwise? How are you feeling about the industry over the next few years?

Charity Majors: There's a lot of anxiety in the air right now. Part of this is the information diet we were just talking about. There's a lot of money to be made by making people feel anxious. I felt a lot of anxiety too when I came out of my book hole and I was like, "Oh my god, is everything I know obsolete?" The answer is that it's not. First principles are more needed than ever. But the only way to soothe that anxiety is to jump in the deep end, to find your feet. You can't inherit someone else's understanding. There's an intuition, a way of using data, and right now is the easiest it will ever be. Right now, we're all effectively beginners. A year or two from now, that will not be true. So if you're not learning what you need to learn at your current job, learn it on the side and look for a new job, because you do not want to be starting this stuff two years from now.

Matt Klein: Those are great words and great advice, which I'd also give people. We're probably going to have to do a follow-up, because there are many topics we could talk about at much greater depth.

Charity Majors: Anytime, Matt.

Matt Klein: That's a wrap for this episode of Beyond the Noise: Signals, Stories, and Spicy Takes. Huge thanks to Charity for joining and sharing her story. You can find this episode and all past ones on the bitdrift YouTube channel. If you had fun, drop us a review, tell your friends, or yell your favorite hot take into the void, and make sure to tag us. I'm Matt Klein, and I'll see you next time.

Charity Majors: Thank you.