Nicki Stone: On-Device ML, Monorepos, and Amazon Scale

Matt Klein: Welcome to another episode of Beyond the Noise: Signals, Stories, and Spicy Takes, the show where we dig into the stories of the people shaping the future of app-based computing, with a special focus on mobile. I'm your host, Matt Klein, co-founder and CTO of bitdrift, as well as the creator of Envoy Proxy. Each episode we'll talk with engineers, founders, and technical leaders who've transformed the way their companies build and understand what's happening inside their systems. We'll dig into the challenges, the breakthroughs, the lessons learned, and we'll wrap it all up with their hottest takes. So let's dive in.

Today I'm thrilled to have Nicki Stone, a principal engineer with 12 years of expertise in mobile technologies, primarily iOS. She has deep experience building mobile frameworks and libraries, with a particular focus on the non-functional requirements that often get overlooked: performance and reliability. She was the original creator of the AWS Swift SDK and Amplify Predictions, AWS's on-device machine learning framework. She's worked at AWS, Meta as part of Instagram, and Amazon, and ran her own VC-funded startup. Currently she oversees development of the Amazon Shopping app. Welcome, Nicki. Thank you for joining us.

Nicki Stone: Thank you for having me. Excited to be here.

[00:01:12] FROM THE GENIUS BAR TO ENGINEERING

Matt Klein: I'd love to start at the beginning. How did you get interested in mobile, and how did your career get started?

Nicki Stone: Actually, that's like the most interesting story I have, so it's a good place to start. I was working at the Apple Store as a genius when I graduated college.

Matt Klein: Really. Okay.

Nicki Stone: My entire life I was always the tech guru. It didn't matter where I went. In my sorority, I was the one fixing everybody's laptops. But I always just saw it as a hobby. I never saw it as a career. So when I graduated college, I went into event planning because I lived in Los Angeles. That was a stupid thing. I did that for a little while, then realized I had morals and I wanted to hold onto them. So I left event planning and went into finance. I worked for a large financial advisor in Century City, got my Series 7 and Series 66, thought I was going to be a financial advisor. And then I realized that was boring.

I would go to work at 6:00 AM, that's when the stock market opens on the West Coast, and I would finish everything I had to do by 8:30. So I'd sit there bored from 8:30 to 3:00. He managed accounts for a ton of celebrities in LA, so it wasn't that there wasn't enough work. I was just efficient. And I was like, this cannot be the rest of my working career. I will not make it to age 60 if this is it.

So I sat there and thought: when I was a kid, what kind of toys kept me entertained for hours? Legos and puzzles. I could sit there and the whole day would go by and I wouldn't even know it. So I started googling careers that involved building and problem-solving, and coding came up. I started teaching myself JavaScript, HTML, and CSS at my desk every day from 8:30 to 3:00. After a couple of months I was like, I need to make this my actual job.

Matt Klein: What were you building while you were sitting there?

Nicki Stone: Mostly front-end stuff. I had some MySpace background, so I was messing around building websites. After about two months I looked into post-bacc programs and realized it would be three years before I could do anything. Then I found boot camps, which were barely a thing at the time. There was really only one: Hack Reactor in the Bay Area.

I applied to Hack Reactor, and while I was waiting to hear back, I searched again and another boot camp had just appeared. I was too naive to know it was brand new. I applied, they told me to meet them at a Starbucks (they didn't even have an office) and they said, "Why do you want to learn to code?" I told them, and they said, "You're in." I was the second cohort ever.

The way their program worked was that a startup would come in, pitch their idea, and you'd spend 20 weekends building it: ten hours each Saturday and Sunday. I also coded every weeknight after my full-time job. I did that for five months, graduated, and had multiple job offers within a week.

Matt Klein: That's a remarkable story. And it actually raises a question I've been curious about. Boot camps have gotten a bad rap in recent years. Do you think it's still a viable path?

I think it's really hard, and you have to be a self-starter. It's not a CS degree with a full curriculum. A lot of what I did was what I'd already been doing on my own: throwing myself at a problem and trying to get from point A to point B using Google and whatever resources I had. How committed and dedicated you are is what will make you successful, and that's also what's going to get you a job afterward.

The boot camp itself doesn't get you a job. They might help with mock interviews or put together a resume, but ultimately it's you: how much you're pounding at the door, how much you're continuing to improve your skills, and not just your coding skills. Your interview skills are a whole separate set. I've seen a lot of people go through boot camps and not get a job afterward, and they don't understand why. I also think boot camps are getting a worse rap right now just because it's a hard market for junior engineers across the board, whether you went to a boot camp or got a CS degree.

[00:08:48] ONEIOTA, HACKATHONS, AND BETA GIG

Matt Klein: So you graduated, got multiple offers, what happened next?

Nicki Stone: I started at a company called One Iota in LA. They did audience casting for shows like Jimmy Kimmel and Jimmy Fallon. I was working on a full-stack .NET application, but I'd always been interested in mobile. When I was at the Genius Bar, I focused a lot on fixing iPhones and iPads, and I was first in line to get my iPhone in 2007. So I started teaching myself how to write mobile apps at night. I'd come home, crack open a beer, and code until I went to bed, but not on what I was doing at work. On mobile.

I built several apps during that time. Back then you could walk into a store and buy a book on iOS development. I had five. I coded until 1:00 AM most nights and kept putting apps out in the store. At that point I had a .NET background from the boot camp, so my first stab at mobile was actually through Xamarin. But I kept feeling handcuffed: I couldn't do certain things to the UI that I wanted to do, I was having trouble with native APIs. So I decided to rewrite an entire app in Swift. I did it in two days, which looking back was crazy. But I learned Swift by doing it, and I got it working and launched it in the App Store.

Matt Klein: And then you and a business partner started trying to self-fund through hackathons?

Nicki Stone: Exactly. We went around winning hackathons. Some of them had prizes of five, ten grand. You show up, come up with an idea, build it in 24 to 48 hours, present it. We just had good ideas and kept winning. We accumulated something like $12,000 to $15,000.

Then I found a hackathon that promised a quarter of a million dollars and entry into an incubator. We flew up to San Francisco, walked around trying to get inspired, came up with an idea, built it in 48 hours, and won.

Matt Klein: What was the idea?

Nicki Stone: It was a startup called Beta Gig: a way to beta test your next job. Sixty-six percent of people look for a new job within 90 days of starting one. When you go through an interview process, especially as an engineer, you're mostly getting grilled on technical tests. You don't get to see what the culture is actually like on the inside. So I built a marketplace where you could try before you buy: job trials that could be inserted either before or after a company's interview process. If a hire was made, we took a percentage, like a recruiter.

The original idea was more about job shadowing for college students. I actually signed on to a community college and had another waiting. But most of the VCs wanted us to go down the recruiting path.

Matt Klein: And you raised money?

Nicki Stone: I raised $1.8 million. We ran for two years, had customers and revenue, and then it came to an abrupt halt. A co-founder who didn't have interest in continuing was ultimately the largest part of it. One thing I'd tell anyone starting a company: know your co-founders very well, and know their motivations. It's a long process, and if the motivations aren't shared, it's going to be very difficult. And if it's your idea, don't give away half your equity. Nobody's ever going to be as passionate about it as you are.

Matt Klein: I agree with both of those.

[00:18:39] AWS: DEVELOPER ADVOCACY AND BACK TO ENGINEERING

Nicki Stone: I had a VP at AWS who was one of my advisors, and the second things came to a close, she said, "Come work for me." So I jumped quickly back out of CEO mode and into engineering. I did developer advocacy my first year back, mostly .NET advocacy. I traveled all over the world and spoke to thousands of developers.

But by early 2020, I wanted to stop traveling and go back to full-time engineering. Being a developer advocate is a bit like being a musician on tour. At some point you want to get back in the recording studio and make music again. I'd been talking to thousands of developers every day, but I hadn't put code into production in so long. I couldn't really empathize with what they were going through, and it started to feel illegitimate.

Matt Klein: Do you think the same thing applies to senior engineering leadership more broadly? Like, does someone who hasn't pushed code in a while eventually lose credibility with the engineers on the front line?

Nicki Stone: If you're a senior engineer working with engineers, staying close to the problem space and in communication with the people writing the code, it's hard to lose it. The developer advocacy situation is different because you're in a silo. I didn't have a team. I would gather feedback from developers and take it back to teams at AWS, and it would fall on deaf ears. I was making demo code: would you put my GitHub in production? I genuinely didn't know anymore, because I wasn't around a team actively doing it. I wasn't reviewing their code.

Matt Klein: That makes sense. So January 2020 you went back to full-time engineering, and then the pandemic hit.

Nicki Stone: Very lucky timing. I joined the mobile team at AWS and worked on mobile frameworks. I built Amplify Predictions, and then pitched the AWS Swift SDK and started building that around March of 2020. I spent two years on it before jumping to Instagram.

[00:22:30] ON-DEVICE MODELS AND THE FUTURE OF EDGE INFERENCE

Matt Klein: Before we get to Instagram: I want to ask about Amplify Predictions and on-device models, because I think the topic is really relevant right now. It feels like over time, models are going to keep getting pushed further and further toward the edge, toward the device. From a privacy and cost perspective alone, there are compelling reasons to run processing on the phone. Based on your experience, how do you think about the on-device versus cloud model tradeoff?

Nicki Stone: At the time, there was a lot of pushback on on-device models: concerns about processing speed, whether the GPUs on device would be fast enough. So what I built was a framework that unioned the on-device APIs with AWS APIs. AWS had these offerings, Polly, Textract, and others, that were essentially easy-to-use pre-trained model APIs. You give it an image, you get back labels. You give it audio, you get back a transcript. Very specific, purpose-built APIs.

The on-device models Apple had launched mirrored those same APIs. So I built a framework that, when you were online, called both the on-device model and the AWS model, unioned the results, and gave you the largest dataset with the highest accuracy. If Apple's model found a toy and a bear in an image at 97% and 98% confidence, and AWS found a car and a bear at 99% confidence, you'd get back all three: car, toy, bear, with the bear's accuracy replaced by the higher 99% figure. And if you were offline, on a train, lost connection, it just fell back to on-device only.

Matt Klein: That's pretty elegant. And it sounds like the thing that you built then is essentially what the industry is going to need to build at scale going forward.

Nicki Stone: I think so. There will need to be some form of switching or syncing between on-device and cloud models: based on network connection, accuracy, cost, privacy, whatever the right signal is. Mine was only taking network connectivity into account, which was novel at the time. But I do think the union approach is underrated, because accuracy wasn't where it is today, and combining the two sources produced noticeably better results.

Nicki Stone: What was actually interesting in my testing was that the on-device models frequently outperformed the cloud models for the use cases that made sense on a phone, primarily image-related tasks. The cloud models were better at things like PDF extraction. But for extracting objects from an image, on-device won more often than not.

Matt Klein: That's counterintuitive, but it makes sense. Apple is training their models on images from the exact devices those models run on.

Nicki Stone: Exactly.

[00:31:36] INSTAGRAM AND THE CULTURE OF NON-FUNCTIONAL REQUIREMENTS

Matt Klein: Let's talk about Instagram. What led you there?

Nicki Stone: I was working on the Swift SDK at AWS, and I was pregnant. The way promotion and maternity leave worked at Amazon at the time meant it was going to be very difficult for me to get promoted given the timing. I was up for promotion, and most of my mentors told me that coming back from maternity leave would set me back about 18 months. So in order to maintain my career trajectory, I jumped to Instagram. They also offered six months of maternity leave versus Amazon's four, and it was my first kid, so that mattered.

Matt Klein: What did you work on there?

Nicki Stone: The production frameworks underneath Instagram Reels: the ones that power the editing experience. Still mobile frameworks, which wasn't that different from what I'd been doing. But the culture was a significant shock, and I didn't enjoy working there.

Matt Klein: Can you say more about that?

Nicki Stone: What I'm really good at, and what genuinely excites me, is the non-functional requirements. Performance, reliability, durability. The things that aren't necessarily visible features but that determine whether the whole thing holds together. Instagram and Meta as a whole just don't value that. They're interested in the cool new features that customers see. They don't necessarily care about the quality of the code servicing that feature as long as it's functioning.

And I know that sounds surprising for apps operating at that scale, but that's what makes it so frustrating. You have billions of customers and you don't care. If you look under the hood, it's pretty scary. You think you've seen bad code until you open a file that's 50,000 lines long and the same function is repeated 25 times with a slightly different name.

Matt Klein: I'd also say that's basically a description of all the code being generated in 2026 by AI.

Nicki Stone: Not where I'm sitting.

Matt Klein: I wasn't talking about you. I was talking about the LLMs just spitting it out everywhere.

Nicki Stone: It's also just a very every-man-for-themselves culture. Not collaborative, not working as a team toward a common goal. And honestly, I think you can trace a lot of it back to the founder. Mark Zuckerberg started the company as a junior engineer. When you become a senior engineer, that's when you start to care about non-functional requirements: performance, code quality, the stuff that doesn't show up in a product changelog. He formed the culture before ever getting there, and in some ways that culture is still in place today.

[00:38:15] BACK TO AMAZON: THE SHOPPING APP AND THE FEDERATED MODEL

Matt Klein: So you left Meta and eventually landed back at Amazon, but not at AWS. How did that happen?

Nicki Stone: When I was working on the Swift SDK, I'd become close with a colleague named McLaren. Nobody in the company had expertise in Swift and C code review except for him, so he reviewed a lot of my code. Before I left for Instagram, he tried to get me to join his team. I wasn't interested: I didn't want to start over and set back my promotion. But we stayed in contact. His director, Sue Hale, called me once a month while I was out on leave, and eventually they got me back.

Matt Klein: That is some long-game recruiting.

Nicki Stone: They really played the long game.

Matt Klein: So now you're working on the Amazon Shopping app. What were the challenges that drew you back in?

Nicki Stone: The enticing thing was they wanted me to help rewrite the store in a new framework: moving away from web pages toward something more native-first. Not fully native, but a step in that direction. Essentially a framework that renders views natively on device, but loads them dynamically over the air at runtime. Web developers who want over-the-air updates to the mobile app can still have that. Native developers who want native rendering for performance reasons, or need access to native APIs, also get that. Best of both worlds.

Matt Klein: So from a layperson's perspective, instead of shipping HTML and JavaScript, you're shipping some kind of DSL that gets interpreted on the client and used to render native experiences, but the author of the component just writes TypeScript.

Nicki Stone: Exactly.

Matt Klein: How do you test that? For an app that size with all those dynamic components, that sounds like a QA nightmare.

Nicki Stone: When you write a component, you also write tests that run in a test service on Device Farm. We build the component, put it onto a device, and test to see that it works. It's a massive testing service, now powered by gen AI: you can write tests like "go to the search bar, type in teddy bear, click on the first result." It actually performs UI interactions and validates real behavior. We haven't mastered it, but we're always moving in the right direction. And not all of the store has migrated to the new framework yet, so it's still in progress.

Matt Klein: And the broader structural challenge you mentioned is the federated model. Can you explain that?

Nicki Stone: The Amazon Shopping app isn't one team working on one app. Each team owns their own feature or page inside the application, and it's not all native code. Web developers are making pages and content for the app. There's no unified networking stack. If you're a team that wants to build a native experience requiring networking, you'd probably build your own networking stack rather than using the foundational module my team maintains. Which means if you need to add a header to every network request, you might be looking at 50 different networking stacks. We also don't have a monorepo: every team has their own pipeline, and all the code eventually has to get merged into one binary.

Matt Klein: And part of what you're working on now is changing that?

Nicki Stone: Yes. I'm doing a rewrite of those foundational libraries: more performant, better, more unified. One networking module. One logging module. A real dependency injection system. Actual architecture where components build on top of each other, rather than everything being a singleton with no reuse. And for the first time at Amazon, we're doing it in a monorepo.

Matt Klein: Was that a tough sell?

Nicki Stone: Extremely. I'm not even the most pro-monorepo person in the room. I had a love-hate relationship with the monorepo at Instagram: there was no real ownership because everyone had access and could modify anything without permission. Someone would stamp a change without really reviewing it, and I'd come in the next day and find something broken in code I was working on, with no one taking responsibility. So we're being very conscious of the pros and cons this time around.

Matt Klein: Are you building your own build system, or how do you feel about Bazel?

Nicki Stone: I like Bazel. We're doing a monorepo with Bazel, not Brazil, which is Amazon's internal build system. It took a lot of convincing and a lot of manpower to get there, but we're already proving out the benefits and people are coming around. That said, I still don't think a monorepo is the answer to everything. At Meta, everything is in one repo: every application, every service. There are pros and cons to that. For mobile, and specifically for foundational libraries like what we're building, I think it's the right call.

[00:49:45] WHAT'S EXCITING NOW

Matt Klein: Last question: what excites you about technology over the next couple of years?

Nicki Stone: AI-assisted development has genuinely reinvigorated me. I feel like a kid in a candy shop. I've always loved coding and always stayed close to the code, even as I moved up, which I know not everyone does. But this is a whole new level of fun. The tools have gotten a lot more effective even in the last three to six months. All the tedious stuff that comes with having been in the industry for a while, the boilerplate, the scaffolding, a lot of that just disappears now. I'm not writing code for this rewrite at all. I'm writing specs and prompting gen AI. Spec-driven development. And honestly, it's working.

Matt Klein: That's a great place to end it. Thank you so much, Nicki. This was a fantastic conversation. That's a wrap for this episode of Beyond the Noise. Huge thanks to Nicki for joining and sharing her story. You can find this and all past episodes on the bitdrift YouTube channel.

Nicki Stone: Thank you. It was a real pleasure.