Why System Design Is So Critical With AI and How to Learn It

In this lesson: judgment is the scarce skill, you guard the one-way doors, and you learn it with Claude as a tutor.

Byron Mackay · 60 min · May 2026

Top 3 takeaways

01

Judgment is the scarce skill

AI-authored pull requests ship roughly 1.7 times more issues and about three times more vulnerabilities, and exploit windows have collapsed to under two months. Companies now hire engineers who can catch those issues before they ship.

02

Guard the one-way doors

System design is the set of decisions that determine how a system behaves, scales, and fails. Be certain on the hard-to-reverse ones like schema, auth, and public APIs, and let the model help while you make the final call.

03

Learn it with Claude as a tutor

Feed a topic list to Claude as an expert tutor, work top-down with quizzes and case studies, and use spaced repetition over a review list in 30-minute pockets.

Byron Mackay

Byron Mackay

Director of Learning, Gauntlet AI

Director of Learning at Gauntlet AI, currently training hundreds of engineers to work AI-first. 16+ years as a mobile/iOS engineer before becoming an AI platform engineer (Savant, School AI), where he built eval platforms from scratch. Led curriculum development at BloomTech (a cohort of his saw nearly every graduate land an engineering role at Amazon) and ran the Amazon partnership/SDR program that moved non-traditional candidates into engineering roles at Amazon. Deep across platform engineering, AI, mobile, and learning.

Connect with Speaker

Lesson notes

A written walkthrough of the lecture, covering the patterns, the code, and the things that trip people up.

Vibe-Coded Apps Skip the Real Decisions

Byron Mackay opens with a simple warning: vibe-coded apps can look impressive while hiding fragile decisions underneath.

A model can quickly generate something that appears functional, but it will not automatically make the architectural choices a real business depends on. Database design, authentication, roles, permissions, scale, and long-term maintainability all require human judgment.

Vibe coding can work for prototypes. It becomes risky when the software needs to support real users, real data, or real operations.

AI Code Creates New Risks

AI-generated code increases both speed and surface area.

Pull requests tend to get larger, which makes them harder to review and easier to break. The risks show up in logic errors, security vulnerabilities, performance issues, and code that may technically work but is difficult for humans to understand.

As exploit timelines shrink, companies increasingly need engineers who can catch problems before release. The value of the engineer shifts from producing code to reviewing, validating, and making the right architectural calls.

System Design Is the New Base Skill

System design is the set of decisions that determines how a product behaves, scales, fails, and changes over time.

In an AI-assisted workflow, this becomes the engineer's core responsibility. The model can help generate options, but it should not own the architecture. Engineers need enough breadth to understand tradeoffs across databases, APIs, authentication, security, infrastructure, and deployment.

The wrong answer is letting the model decide.

One-Way Doors and Comprehension Debt

Some technical decisions are easy to reverse. Others are one-way doors.

Database schemas, authentication systems, public APIs, and data model choices often become expensive to change later. These are the decisions engineers must slow down for, review carefully, and explicitly sign off on.

AI also creates a new problem: comprehension debt. When models generate large amounts of code, teams can quickly lose track of why things work the way they do.

The goal is not to memorize every line. The goal is to understand the interfaces, the business logic, and the tradeoffs between major components. Files like CLAUDE.md and agent.md can help preserve intent, but they do not replace human ownership of the system.

Use Claude as a System Design Tutor

Byron recommends using AI to build system-design judgment, not bypass it.

Start with a structured list of topics and ask Claude to act as an expert tutor. Work through each topic, ask for quizzes, and sort what you miss into a review document. Use case studies to practice real tradeoff decisions, then return to weak areas through spaced repetition.

The topics to study include databases, distributed systems, security, APIs, infrastructure, deployment, and the deeper features of your programming language.

The core habit is simple: use AI to sharpen your judgment before you use it to ship software.

Transcript

Read transcript

Why System Design Is So Critical With AI and How to Learn It

00:00:02.310Byron Mackay: There we go.

00:00:05.340Byron Mackay: Alrighty.

00:00:09.450Byron Mackay: Oh, hello, everyone!

00:00:14.340Byron Mackay: Just kidding. Let's see here.

00:00:19.870Byron Mackay: That's not what I wanted there. Okay.

00:00:22.400Byron Mackay: Hey, everybody!

00:00:24.080Byron Mackay: Welcome, welcome, welcome. I'm gonna actually stop sharing my screen just for a moment while I get a few things configured here, and then we will go from there.

00:00:32.229Byron Mackay: Let's see here… hope everyone's having a good, good evening.

00:00:43.990Byron Mackay: Alright…

00:00:50.970Byron Mackay: Okay, here we go. We're gonna go… Here we go. Alright.

00:00:58.310Byron Mackay: Okay, everybody, I have got chat opened up here. If you have any questions, feel free to pop them in there. And, we're gonna go get started. So, my name is Byron. I'm a Director of Learning here at Gauntlet, and I'm gonna talk to you tonight about a topic that I've seen a lot

00:01:17.310Byron Mackay: as of late here, here at Gauntlet, we do a couple of things. We teach people how to use AI, and we teach them how to build AI products. That's a big one, right? I think that's what we're generally known for.

00:01:32.380Byron Mackay: And then the other thing is we help people through interview processes so that they can get great positions, high-paying salaries, with this new skill set that they have.

00:01:43.570Byron Mackay: Having gone through this experience a number of times with a lot of people.

00:01:47.570Byron Mackay: I have seen the impact of knowing good system design, and how much more it has become a thing as companies start to embrace AI. Now, I want to hit this

00:02:02.010Byron Mackay: topic tonight with all of you, and share some of my experiences that I've had. These are experiences I've had with other challengers that we've had come through Gauntlet who have gone through this process.

00:02:15.740Byron Mackay: Who have gone through the interview process, and just the stories, some of the stories that they've shared with me, and what we can learn and take away from that.

00:02:23.170Byron Mackay: So, before I do anything, I want to just share a really quick story that happened to me when I was talking to a friend.

00:02:31.370Byron Mackay: And this friend needed to buy a product for his company. He needed… well, he needed something for his company to make it work. It was a SaaS platform, and I said, oh, there's a lot of great SaaS platforms out there, you should really go look into one. And I was giving some suggestions, and while I was talking, he goes.

00:02:47.520Byron Mackay: well, wait, I just built it, though. And he, like, showed me, he's like, yeah, while we were talking, I just vibe-coded this one up. Can I just use this?

00:02:54.880Byron Mackay: And I kind of look, and I smile, and I go, you could. But there's a lot of decisions, that go into making good software, that really make

00:03:09.740Byron Mackay: What he just did, unstable and, quite frankly, not trustworthy.

00:03:15.070Byron Mackay: Not that he's not a trustworthy person, he is, but his application is not.

00:03:20.750Byron Mackay: And that's where we have some issues that I think a lot of people don't really see.

00:03:27.010Byron Mackay: They vibecode these things, they get to build these amazing apps.

00:03:31.130Byron Mackay: But there's a lot of thought that they just left out. They're not even aware that they needed to think about some things before they pulled that trigger. Now, it's fine on small little projects, but if this is something you're going to rely on for your business.

00:03:44.490Byron Mackay: That's where good software engineers come in.

00:03:47.680Byron Mackay: So… I want to talk a little bit about what was actually built.

00:03:52.680Byron Mackay: So…

00:03:54.230Byron Mackay: he built this UI that looked really good, that seemed to have some functionality. That's what he built, right?

00:03:59.910Byron Mackay: What did he not know? He didn't know the schema. What kind of database schema was put into this system? Is it expandable? Did he think about how one document might affect another one, or how one table might need to pull some stuff out so that he could make it more flexible in the future?

00:04:18.149Byron Mackay: He had made none of those decisions. And for anyone who has, anybody who has built a database schema knows there's a lot of thought and intention put behind every single data field and type

00:04:32.900Byron Mackay: That goes into that. And… but that was, you know, obviously he just decided that he didn't need to do that, or he just even realized that that was something he needed to do.

00:04:40.660Byron Mackay: Second, security. Was there auth? What kind of auth was it? Do we have roles?

00:04:47.490Byron Mackay: Could an LLM go through and build those things out? I'm sure it probably could, but to what extent… to what extent, and to… and, how much debt would there be that they might have to go back and fix later? And what kind of holes would exist?

00:05:01.120Byron Mackay: And then finally, scale. That's one that really stands out, because LLMs don't have this foresight of saying, I'm gonna build this for the future. They're just building for whatever you prompted it with. That's all the context it's going off of.

00:05:14.200Byron Mackay: So, I'm sure now, I'm sure we've all seen this, we've all probably done it ourselves, right?

00:05:18.740Byron Mackay: Here we've got an app, we want to build it, so let's put something together real quick, and and make it work. Look, I do it, I still do it, from time to time when I just need something really quick. But this is something his business was relying on, and I told him, I don't trust this one bit.

00:05:35.330Byron Mackay: So, let's talk about some metrics. Let's talk about some of the things that we're seeing in the world today of how people are responding to what AI is doing.

00:05:48.480Byron Mackay: First off, what is AI doing? Well, here's some statistics for you. Some of them might shock you. They are real, they are prevalent.

00:05:59.260Byron Mackay: Let's start with the top one. AI-authored PRs produce 1.7x more issues than human-authored. Now, that might sound like it's not very much more, but compounded it does, but let's look at what these issues are.

00:06:13.400Byron Mackay: So, let's see, so we've got more logic errors, okay? We have 2.74 times more security vulnerabilities. We're gonna talk about security vulnerabilities a little bit more later.

00:06:25.170Byron Mackay: But that's a big one. That's a big, big issue that we're seeing, and the market is actually responding quite a bit to that.

00:06:32.780Byron Mackay: We see 8 times more I.O. performance issues.

00:06:36.910Byron Mackay: That's a big one, right? We are seeing

00:06:39.800Byron Mackay: A lot of issues there that, were just performance is just degrading.

00:06:45.720Byron Mackay: Okay?

00:06:47.700Byron Mackay: And of course, readability, that's gonna be a big problem as well. How can we… how can we maintain something we can't even go in and dig through and read?

00:06:57.380Byron Mackay: Alright, here's a scary one.

00:06:59.470Byron Mackay: This has to do with security issues. So, it used to be that when there was a security issue, it took nearly 2 years for that to be exploited.

00:07:11.920Byron Mackay: Right? Two years. More than two years. Almost two and two years, two and a third? Anyway…

00:07:18.890Byron Mackay: No, that would be just a little over 2 years. Don't do math live. Alright.

00:07:24.090Byron Mackay: What is it now?

00:07:25.810Byron Mackay: Less than 2 months Less than 2 months, and these exploits are being found.

00:07:32.590Byron Mackay: So, combine that… With the fact that there are nearly 3 times more security vulnerabilities, We've got a problem.

00:07:42.030Byron Mackay: We have a problem.

00:07:44.190Byron Mackay: And it's up to us to solve it.

00:07:46.580Byron Mackay: So how do we solve it?

00:07:48.920Byron Mackay: Well, let's see how the… let's see what the companies are doing as it relates to jobs.

00:07:54.230Byron Mackay: They're hiring more security people. That's their solution.

00:07:59.910Byron Mackay: So, we are, so that's what they're doing. They're hiring more security. All right, great, great. We have more security issues, more security roles, let's do it. You know, doubling, right, year over year. AIML roles, those are increasing, right? Of course we know that's increasing, because there's so many more opportunities for that nowadays.

00:08:19.500Byron Mackay: Cybersecurity engineers, you know, that's growing. Open cybersecurity roles, nearly half a million.

00:08:26.410Byron Mackay: Now.

00:08:27.830Byron Mackay: I'm not saying we all have to become security experts. That's not what I'm saying at all. What I am saying is that the security issues are being felt.

00:08:38.799Byron Mackay: They're being felt in a very big way, and companies are making adjustments for it. I don't think everybody, every engineer needs to drop what they're doing and go become a security expert. In fact, I'm going to advocate you don't, unless you really like security.

00:08:52.350Byron Mackay: But I will also point out the bottom here, entry-level general software engineering roles, going down. Now, is that a surprise? No, I don't think it is to anybody. We can do more with less these days, that is true. That is an absolute, truth there, fact.

00:09:07.630Byron Mackay: But, there are problems, and we haven't solved for them yet.

00:09:11.490Byron Mackay: So this is what the world is doing, this is what the market's trying to do.

00:09:15.070Byron Mackay: But I will show you what I think we should do instead.

00:09:20.120Byron Mackay: But first, what does this tell us?

00:09:22.200Byron Mackay: Alright, this will lead us into what we should do, right? First, judgment is scarce, alright?

00:09:29.730Byron Mackay: the… it used to be that the problem was, is that we couldn't write code fast enough, right? Obviously, that's not the problem anymore.

00:09:37.820Byron Mackay: Now we're just writing so much code that we don't even know what it's doing, right? So second, catch it, don't complete… don't compete with it. Companies need engineers who can catch the problems before they get released, okay?

00:09:53.820Byron Mackay: And then finally.

00:09:56.510Byron Mackay: Asking the right questions before anything is built is the best way to start to solve this problem.

00:10:04.170Byron Mackay: I will say this on that last note.

00:10:07.310Byron Mackay: Communication.

00:10:08.910Byron Mackay: has always been an essential quality of a good leader, of a good coworker, a good peer. We all know that communication throughout history of human history, right, communication has been a very key, thing for us to progress.

00:10:26.540Byron Mackay: It is far more important for software engineers to be good communicators now.

00:10:31.560Byron Mackay: We need to ask better questions.

00:10:33.720Byron Mackay: We need to listen better.

00:10:35.500Byron Mackay: We need to articulate better what we want to do, and how we want to do it.

00:10:40.320Byron Mackay: I've seen a lot of people go through this program.

00:10:43.160Byron Mackay: And by far, the better communicators go a lot further.

00:10:47.460Byron Mackay: So, if that's something, where you feel like you need to get more up to, you know, better skills with.

00:10:54.210Byron Mackay: Go do meetups, go get presentations, try asking more questions next time you're at work.

00:11:00.810Byron Mackay: To go talk to people about the problems that they're trying to solve, and see if you can understand what's going on, articulate solutions to them. This is a very important skill that we need as software engineers, more now today than ever.

00:11:13.010Byron Mackay: I think in the past, we could just talk amongst engineers, and we could talk our engineering speak, and it was great, right? I long for those days.

00:11:21.180Byron Mackay: But that's not where we're at right now.

00:11:23.150Byron Mackay: Now we have to communicate at a higher level and be more meaningful with the decisions that we make. And I'll talk about what I mean by being more meaningful here in just a moment.

00:11:34.050Byron Mackay: Because now we're going to talk about what system design is. System design, in my opinion, is the new base skill that engineers need to be successful.

00:11:46.160Byron Mackay: It is not enough to just know a little bit about something, and make that your one, you know, specialty. All those specialties can still be important. What you really need, though, is the breadth.

00:11:58.220Byron Mackay: Breath is just as important as depth today.

00:12:03.160Byron Mackay: We need to understand the decisions we're making, We understand what we're telling.

00:12:08.630Byron Mackay: the computer to do, or the AI to do.

00:12:11.230Byron Mackay: So that it can build the right thing for us.

00:12:16.550Byron Mackay: Alright.

00:12:19.030Byron Mackay: So, this… what system design is, is a set of decisions that determine how a system behaves, scales, fails, and changes over time. Now.

00:12:28.210Byron Mackay: Take that all in for a second.

00:12:30.440Byron Mackay: behaves. Okay? I think we've all done that, right? We've all written applications, we talk about how it behaves. Scales. Sometimes, some of us maybe have had the opportunity to really scale an application. Doesn't mean we shouldn't know how to do it, though.

00:12:44.240Byron Mackay: How it fails.

00:12:45.920Byron Mackay: I don't like to think about failure very often, but we need to think about failure, because it will happen.

00:12:52.440Byron Mackay: And then how can it change over time, okay? These are 4 areas, and they aren't the only areas, but these are some major ones that we need to be very aware of and know

00:13:03.520Byron Mackay: How to build an application with these things in mind.

00:13:07.710Byron Mackay: Right?

00:13:12.980Byron Mackay: So, let me go ahead and move on to the next one here, because I want to get to a certain point here, but what is system design not? I want to make sure that we are clear on what it's not. It is not just knowing one language very well. That's what it used to be. We used to be able to get away with that, right?

00:13:27.690Byron Mackay: And I think that's still very valuable. In fact, I would encourage you to still go learn a language really, really well. For example.

00:13:36.930Byron Mackay: I used to do an interview with, a TypeScript interview.

00:13:41.690Byron Mackay: Where we were trying to gauge how somebody would control state within an object.

00:13:48.110Byron Mackay: And what I… what I came up… my solution to it, was to use what's called a discriminated union.

00:13:56.850Byron Mackay: What a descript… if you're not familiar with that concept.

00:13:59.940Byron Mackay: It's all… it's very similar to inheritance in that a single object could be represented as multiple objects.

00:14:08.290Byron Mackay: It's a TypeScript specialty, I would say, but it's one that's very powerful.

00:14:13.680Byron Mackay: And I tell ya, nobody I ever interviewed came up with it.

00:14:17.350Byron Mackay: And that was kind of surprising to me. I thought… I felt like… now, of course, I wrote the question, so it's probably me, like, manifesting myself in the question, so I'll take that if that was the case. I felt like it was pretty evident, though, and I ended up teaching a lot of people, about discriminated unions when I was interviewing them.

00:14:37.130Byron Mackay: Now, I'm not saying that's, like, an end-all, be-all. We hired… I hired people, even if they didn't understand it, because they were good learners, and they understood the concepts really well.

00:14:47.050Byron Mackay: But it's worth understanding what tools are available.

00:14:51.750Byron Mackay: if I were to prompt an AI to build something, and I wanted to use a discriminated union to help better facilitate messages or objects or types.

00:15:02.850Byron Mackay: I would want to be very explicit about that. I wouldn't necessarily trust the LLM to build that on its own.

00:15:10.020Byron Mackay: Likewise with every other language or framework that you use, it's important to understand its capabilities so that you can dictate those specific capabilities to the LLM and know what's going on.

00:15:23.480Byron Mackay: Again, this is not going to probably… it used to be, if you knew one language really well, you could get a job anywhere. I don't feel like that's the case anymore, I'm just not seeing it.

00:15:32.110Byron Mackay: And so, but it is still extremely valuable to understand.

00:15:38.470Byron Mackay: What else is not system design?

00:15:41.810Byron Mackay: Being able to implement a feature quickly. Alright, we all know this, right? Anybody can do it, right? Anybody can go and type whatever, and it's done. It's amazing. It's still magical to me.

00:15:53.450Byron Mackay: Okay.

00:15:54.660Byron Mackay: But it's not gonna get you a job.

00:15:56.640Byron Mackay: Alright.

00:15:58.460Byron Mackay: Breath of understanding matters as much as depth.

00:16:02.860Byron Mackay: And I really believe that.

00:16:04.930Byron Mackay: I really, really believe that. We have to understand all the… all the different facets of what we're dealing with so that we can make really smart decisions.

00:16:14.970Byron Mackay: With what we're building.

00:16:17.350Byron Mackay: The… letting the LLM decide how to architect is not a good answer. In fact.

00:16:24.980Byron Mackay: It's probably the worst answer.

00:16:26.850Byron Mackay: Quite frankly.

00:16:28.340Byron Mackay: Companies don't want that. They could do that on their own if they really wanted to.

00:16:32.090Byron Mackay: But to be very intentional, that goes a long way.

00:16:37.510Byron Mackay: Alright, I'm gonna pause here for a second, because I know the chat's been going down. I just want to make sure I'm not missing any questions. I don't think I have so far. Well, there might have been one here.

00:16:50.330Byron Mackay: I wanna, I wanna highlight,

00:16:53.720Byron Mackay: I want to highlight something that Josh said.

00:16:56.110Byron Mackay: You don't know what you don't know.

00:16:58.650Byron Mackay: That is a huge issue. I'm gonna tangent here for just a second, because there's something that I love. I can't remember the exact name of it, but there's… there's four quadrants of knowledge, that's how I like to put it. There's the…

00:17:14.500Byron Mackay: You don't know what you don't know.

00:17:17.050Byron Mackay: There's the, you know what you don't know.

00:17:20.280Byron Mackay: There's the, I know what I know.

00:17:24.140Byron Mackay: And then the really fun one is…

00:17:26.470Byron Mackay: I don't know what I know.

00:17:29.140Byron Mackay: Now…

00:17:30.930Byron Mackay: There's another way of putting it, and I won't get into… there's, unconsciously incompetent, and consciously competent, and all these things. I won't worry about that, that's just… that's… but big words, won't worry about that.

00:17:43.670Byron Mackay: But Josh is absolutely right, we don't know what we don't know. And that is where learning starts, and it's the scariest step.

00:17:50.800Byron Mackay: Because we don't want to get… we don't want to admit that we don't know something. It's true for… I think, generally speaking, most humans will feel this way. I definitely feel this way.

00:18:00.900Byron Mackay: It takes a step. It takes a step of faith to go out there and discover what you don't know.

00:18:06.740Byron Mackay: And then… start to try and learn it. The great thing is today, so we have LLMs.

00:18:11.740Byron Mackay: And LLMs are really great tools for figuring out things that you don't know. And they're really helpful to learn those things really fast. So that bar to learn is actually significantly lower than it ever has been.

00:18:22.720Byron Mackay: So, it's an opportunity for us. I'm actually going to come back to that thought here in just a second. Dan, Rumsfeld Matrix, the Rumsfeld matrix. There we go.

00:18:33.900Byron Mackay: Love that.

00:18:35.100Byron Mackay: That's it.

00:18:37.470Byron Mackay: Yeah, alright.

00:18:41.280Byron Mackay: Let's talk about…

00:18:42.940Byron Mackay: some of the most important decisions we can make when it comes to system design. These are one-way doors. Now, if you're not familiar with that concept.

00:18:51.360Byron Mackay: In business, and in engineering, and in most of life, there are one-way doors, and there are two-way doors. One-way doors are the ones you walk through once, and it's really hard to come back out.

00:19:04.230Byron Mackay: These are decisions that you make, and when you make them, you better be certain.

00:19:08.360Byron Mackay: Because if you make them wrong, it's gonna be a lot of pain down the road.

00:19:11.720Byron Mackay: Two-way doors, on the other hand, are very easy to switch, right? So, if you have a two-way door, you make it as… if you're spending a lot of time deciding on a two-way door, don't. It's a two-way door. If you need to backtrack, you can.

00:19:24.870Byron Mackay: Knowing which one's which… knowing which one you're dealing with is a really important piece there. But we're going to talk about some one-way doors here.

00:19:32.660Byron Mackay: Where backtracking is really expensive.

00:19:36.730Byron Mackay: Alright.

00:19:38.250Byron Mackay: Database schemas. My favorite. I actually really like database schemas. I love designing tables, and where data goes, and the relationships,

00:19:49.300Byron Mackay: I really enjoy… I think it's a really fun exercise, and it really causes me to be very intentional about how things fit and how I perceive things in our application.

00:20:00.800Byron Mackay: I think it's really, really fun.

00:20:03.060Byron Mackay: But it is a one-way door.

00:20:05.440Byron Mackay: Which is… A good thing to stop and take a lot of time to figure this out.

00:20:10.610Byron Mackay: Because you have to think… you want to think about every table and every object that you hold, and all the data you hold, in a certain way. Do I want this to be, you know, for example.

00:20:21.590Byron Mackay: I never… I wanna make sure I…

00:20:26.990Byron Mackay: when I use the word never, I want to make sure it's true. I don't think I ever have. I've never combined the company data with the user data.

00:20:36.180Byron Mackay: Right, my applications. If I have a company that uses my app, I have a company table, I have a user table. I do not mix those two into one table. If I need to, I'll link them, right? The user should have an ID to the company, but I don't want to link them. Now, I think most LLMs will make that distinction and just make the decision for us.

00:20:53.590Byron Mackay: But that's not true about many other things. And the reason is because we can't possibly give the LLM all of our information.

00:21:01.700Byron Mackay: Right, all of our context.

00:21:03.950Byron Mackay: And so there are trade-offs to be made. There's ways that we're thinking about, how this data works, and LLMs are going to fail eventually if we let them make all the decisions. So, I would personally say anytime I deal with data-based tables and what… and constructing them.

00:21:20.720Byron Mackay: I'm making that call, I'm making it very, very explicitly. I'm using LM to help me design it.

00:21:26.200Byron Mackay: But I'm the one signing off of it every time.

00:21:28.880Byron Mackay: Because you do not want to migrate that table differently if you have to go back later. That is a painful, painful experience. I'm sure many of you are familiar.

00:21:38.670Byron Mackay: Of course, architecture compounds. Point number 2 here. Wrong choices today.

00:21:43.720Byron Mackay: Become bigger problems 18 months from now.

00:21:46.280Byron Mackay: You want to make sure that you are very intentional about that as well.

00:21:50.980Byron Mackay: One interesting thing here that I want to point out is that as we make these intentional decisions.

00:21:57.190Byron Mackay: The LLM of the future does not know our intent. It sees the code, and it has to interpret what that intent is from the code, but it does not always make the right assumption, right?

00:22:10.030Byron Mackay: We know the assumptions. We need to understand what those are.

00:22:13.620Byron Mackay: In fact, it's always kind of baffled me a bit.

00:22:16.740Byron Mackay: When, like, how companies with lots and lots of engineers working on one application

00:22:22.540Byron Mackay: And by lots and lots.

00:22:25.110Byron Mackay: Like, what do I mean? Like, 10, 20?

00:22:27.580Byron Mackay: 20 feels like a lot to work on one application.

00:22:30.110Byron Mackay: And yeah, I know companies that have 20 people working on one application, and not just, like, one back… a couple backends, a couple… no, like, maybe the mobile app has 20 engineers working on it. How in the world are they supposed to pass that tacit knowledge back and forth and intention? They can't.

00:22:45.280Byron Mackay: And so there's ways of organizing that and making that work, but if you think you can't do it with a human, try doing it with an LLM. It's a whole different ballgame, because they make it look really good. And we're in trouble at that point, because then they'll make decisions that we really don't want.

00:23:01.490Byron Mackay: Alright, next one. Auth. Auth is expensive, derivative fit. You also can… it also can be expensive if it goes wrong, right? So you want to make sure that these really important points, these single points of failures, or these ways that people can break into your system, we want to make sure that we understand what's going on here.

00:23:20.260Byron Mackay: Right? It's not enough just to understand the code, but it's also important to understand these other components that go into it, like auth.

00:23:27.800Byron Mackay: We must understand auth better, if you don't already.

00:23:31.530Byron Mackay: It's very expensive to go back and try to fix, or maybe you want to switch it out for something else. You've got tokens you gotta manage, you've got re-encrypting that you have to do. It can be a very painful process. So you want to make sure you get that right, and you want to do it with intention.

00:23:47.750Byron Mackay: Alright, this last one is… depends. APIs.

00:23:51.570Byron Mackay: If you have a public API, that can be expensive. That's kind of a one-way door, right? You don't want to make a… you don't want to make some massive change and then break everybody else's apps, and then they're all mad at you, right? And of course, there's strategies around how to manage that.

00:24:05.110Byron Mackay: If the API is internal, alright, it's a little more of a two-way door there, right? Because I control what the website receives and how it manages it, so I just have to make this, you know, maybe a rollout plan, right, where

00:24:21.520Byron Mackay: you know, where I can add some fields here, make it backwards compatible there, and then we're okay, we're good.

00:24:26.420Byron Mackay: Hard to do that when it's public. So, again, kind of depends a little bit there.

00:24:32.290Byron Mackay: Alright.

00:24:33.520Byron Mackay: Now, these are just a couple examples. I'm not going to go through all the different things that we could go over, but I just wanted to highlight some of the big ones.

00:24:41.140Byron Mackay: And just explain that, once again, this all comes back to system design. And just understanding not just what one piece is doing, but what the whole is doing. And being able to work

00:24:52.150Byron Mackay: Through those… make those decisions in that… in that space, so that you can build a application that can then scale and fail, one that you can change, one that you can, continue to build with.

00:25:09.060Byron Mackay: Alright.

00:25:10.080Byron Mackay: I'm gonna pause here for a second.

00:25:11.970Byron Mackay: Just go through, let's see…

00:25:16.350Byron Mackay: Let's see… there is no repo today, but I do have something that I'm going to share with you after.

00:25:24.630Byron Mackay: But no repo.

00:25:31.920Byron Mackay: Alright, Josh has an interesting, story, it looks like here. I recently ran into an issue with a one-way door. It built a rag pipeline that uses Voyage AI for embeddings. My database table stores vectors of 1024. I was going to switch to a more performant AI model, but it only outputted in 1536. That's too bad.

00:25:51.580Byron Mackay: Yeah, there you go. So,

00:25:54.310Byron Mackay: Yeah, what do you… what do you do, right?

00:25:58.290Byron Mackay: I know exactly… I'm laughing because I know the problem personally as well. But now you know, right? It's a tough lesson, but yeah, that's those kinds of things where it's like, you want to understand that beforehand so you know what you're getting into.

00:26:14.350Byron Mackay: And we just… we haven't… we haven't always hit, that extremely well. I will say this,

00:26:22.890Byron Mackay: When it comes to system design.

00:26:26.610Byron Mackay: It used to be that this was the kind of thing that got you promoted. It got you pushed up that ladder, you became a leader when you understood how systems worked, and how your company's systems worked.

00:26:38.970Byron Mackay: That was a big deal, and now it feels more and more, from what I can tell, that this is becoming more of an expectation of all, not just a few.

00:26:48.650Byron Mackay: Or at least it's expected that everybody will learn to some breadth of what's going on, more than just a few.

00:26:56.060Byron Mackay: So… There you go.

00:27:00.060Byron Mackay: Let's go on to the next one.

00:27:03.440Byron Mackay: Alright, the trade-off layer. I alluded to this earlier, but I want to emphasize this again.

00:27:10.060Byron Mackay: AI sees only the code.

00:27:12.710Byron Mackay: Not why your team made the decision to do it.

00:27:16.350Byron Mackay: I think this is really important, too.

00:27:19.150Byron Mackay: To underst… to… to accept, right?

00:27:23.490Byron Mackay: AI only sees your code, so when it's looking at the code, and when I say your code, it's probably not code you wrote, right? It's what other LLMs wrote.

00:27:31.530Byron Mackay: But when it sees the code, it has to… Infers so much.

00:27:35.420Byron Mackay: From that information, and only that information.

00:27:38.700Byron Mackay: Now, you can write CloudMD files or agent.md files to kind of help it know more about intention. I think I would encourage anybody to do that. But at the end of the day, there's still trade-offs that you made, that you consciously made.

00:27:52.660Byron Mackay: Hopefully, that you are consciously… that you consciously made.

00:27:56.840Byron Mackay: that the AI will have a hard time inferring.

00:28:00.390Byron Mackay: So… Really important that you take the time

00:28:06.380Byron Mackay: To understand those trade-offs and somehow document those. Or…

00:28:11.850Byron Mackay: With a smaller… if you have a team where you can all discuss these trade-offs and understand the implications.

00:28:18.470Byron Mackay: Then you can, relay those into the prompt.

00:28:22.790Byron Mackay: Right? Or you can at least detect it when the PR comes out.

00:28:27.580Byron Mackay: But this is invisible context, right? This is what's… what context is not necessarily known that sometimes we hope the LLM will figure out on its own.

00:28:42.090Byron Mackay: So, there is this idea of comprehension debt, which is not… it's technical debt, comprehension debt,

00:28:48.800Byron Mackay: But this is a problem that we see a lot of with, of course, LLMs producing so much code, and nobody really understanding all the nuance inside of it. I'm not saying that you need to understand all the code line by line. I really don't think that's a requirement these days.

00:29:04.910Byron Mackay: We can get by doing a lot better if we do understand the interfaces.

00:29:11.340Byron Mackay: So, if we have a component in our application, we understand the interfaces of how it interacts, that will go a long way. The business logic inside, we should review and make sure, at least at the time that it's working.

00:29:21.680Byron Mackay: But we don't need to understand all the ins and outs, right? We don't need to understand so much of the deep detail that

00:29:27.670Byron Mackay: That would take way too long. And quite frankly, we never have.

00:29:32.260Byron Mackay: So…

00:29:33.620Byron Mackay: And we've worked just fine without it. But understanding the interfaces and understanding the components it's working with, all very important things.

00:29:42.130Byron Mackay: Okay.

00:29:43.160Byron Mackay: So we've gone through a lot of, you know, what makes system design important, what are the pitfalls that we can pull into when we don't have good system design.

00:29:53.320Byron Mackay: What do we even want to know about system design? Where should we even begin?

00:29:57.510Byron Mackay: And what are companies doing about it? Like, we've seen them hire a lot of security roles. I talk about all the system design stuff. Let me give you an example, really quick, two stories.

00:30:07.670Byron Mackay: Of questions that were asked in interviews.

00:30:10.960Byron Mackay: That related to even just basic understanding of system design.

00:30:16.980Byron Mackay: And the outcomes that happened there. The first instance was when an interviewer asked the other… asked the interviewee.

00:30:28.140Byron Mackay: What is the difference between Postgres and Mongo?

00:30:34.150Byron Mackay: And that's what… that was the whole question. There's nothing else to it.

00:30:37.840Byron Mackay: Now, some of you might go, oh, I know exactly what that is.

00:30:41.390Byron Mackay: And some of you might be going, I actually have no idea. And you're fine either way. I'm not here to call anybody out.

00:30:48.690Byron Mackay: But the fact is, that was the question… those were the kinds of questions that I'm seeing me ask more and more and more.

00:30:53.920Byron Mackay: There are people who are asking these kind of questions to see, do they understand the breadth?

00:30:59.620Byron Mackay: Not necessarily the depth. That question is a fairly high… is a very, very high-level question, I would say.

00:31:05.110Byron Mackay: But, do they have that… they want to see if you have that comprehension of understanding the decisions

00:31:11.860Byron Mackay: Of when to choose one or the other. And you may not even be a database guy, right? I mean, I talk about building schemas all the time, but did I ever live outside of, relational databases? Hardly ever, right? I'm a relational database guy.

00:31:26.190Byron Mackay: I don't know if I'm a traditional database guy, but that's what I've worked in a lot, and I'm really good at building those tables.

00:31:30.870Byron Mackay: Understanding them, and going through them.

00:31:33.220Byron Mackay: But I would, but anyway, in this particular situation, the person had no idea.

00:31:40.510Byron Mackay: And, as you can imagine, That… went down.

00:31:45.760Byron Mackay: Daniel, you nailed it. Postgres structured, relational, Mongo, unstructured, document-based.

00:31:53.600Byron Mackay: I don't think it's too overly simple. I think for that question, that's perfectly fine.

00:31:57.570Byron Mackay: Now, they might go deeper, right, after that first question. Maybe it was one of those cases where it's like, let's start high level and then dig down and see how far they go, but he didn't go very far.

00:32:06.590Byron Mackay: Obviously.

00:32:08.470Byron Mackay: Let me tell you another interview story.

00:32:10.480Byron Mackay: This was from a very accomplished engineer who went into an interview, and they said, they were talking about their architecture, and they said, we're using this Postgres database, and here's all of our systems, and here's how everything works, and so she had a pretty good idea of how the data was being passed through, and the person asked.

00:32:29.830Byron Mackay: What would you do when you got to 50,000 users in the database?

00:32:35.340Byron Mackay: Now, that is… I'm missing a lot of context here of how this answer was produced.

00:32:42.380Byron Mackay: But that was the question.

00:32:44.650Byron Mackay: And her response, and again, I can't explain it entirely, because I wasn't the one that interview, and I was hearing this from her, was I wouldn't use Postgres anymore.

00:32:55.770Byron Mackay: I thought that was such an interesting answer. But then she goes on to explain why.

00:33:01.120Byron Mackay: And the person agreed with her. He said, that's absolutely right, that's what we should do, is get off of Postgres at that point.

00:33:07.320Byron Mackay: And I thought that was so interesting. There was this understanding that she had of what happens when you start to scale an application. I don't think 50,000 is necessarily super big. I don't know, and again, I can't remember all the details, but she had this understanding. This is a separate company, by the way. Different company.

00:33:22.700Byron Mackay: Asking similar questions. They want to see our breadth as engineers. This is what we're seeing, this is what's happening.

00:33:31.350Byron Mackay: And so we need to… we need to figure out, we need to obtain this breadth so that we can become…

00:33:39.640Byron Mackay: engineers that companies want to hire. Features right now today are so cheap to build, anybody can build them, right? So what do we offer? Judgment.

00:33:48.340Byron Mackay: We offer breadth of understanding. When we build systems, when we build an application, it's not just the website anymore, but it's everything that goes around it. And how do we make those decisions? How do we make them very clear?

00:34:00.550Byron Mackay: And how do we articulate that? When we tell somebody in business we're using Mongo over Postgres, we need to come up with a good reason to explain why. Well, maybe not to them, they'll just take our word for it. At least that's what I usually have seen in the past, when I've offered Mongo over, you know, or something else over a relational database.

00:34:17.949Byron Mackay: But how do we make sure that our decisions are going to help the product scale and build, and fail appropriately, and all those kinds of things?

00:34:25.739Byron Mackay: So,

00:34:28.050Byron Mackay: Anyway, I'm sending this out to you all to say this is what we're seeing, it's becoming very evident, and this solves, by the way, coming back a little… coming back to what I talked about in the beginning.

00:34:40.010Byron Mackay: In a lot of ways, knowing how to design a good system will resolve a lot of the security errors that we're starting to see.

00:34:49.810Byron Mackay: We will understand how…

00:34:54.620Byron Mackay: how encryption works, how roles work, how access to data works. There's all these pieces that we need to be very aware of.

00:35:03.440Byron Mackay: And companies are picking up on it.

00:35:07.680Byron Mackay: That's… that's what I want to… that's kind of the big takeaway… one of the big takeaways I want you to have tonight.

00:35:12.540Byron Mackay: Is that's where companies are seeing the need.

00:35:15.680Byron Mackay: They're afraid of the security issues, so some companies are hiring more security people.

00:35:20.180Byron Mackay: And we are seeing the need for companies needing engineers who understand applications at a higher level, so that they can avoid those security issues entirely, but also build a more scalable solution.

00:35:35.230Byron Mackay: We can all take a good guess what would happen if we just let an LLM churn on an app for a year.

00:35:41.230Byron Mackay: Probably run into a lot of issues.

00:35:43.970Byron Mackay: Especially in large legacy codebases.

00:35:47.620Byron Mackay: So we need people who understand the system design principles a lot better.

00:35:55.030Byron Mackay: And companies are keying in on it.

00:35:59.030Byron Mackay: Tanya, I'm gonna answer your question here.

00:36:02.620Byron Mackay: Yeah, because I think it's… it might lead into my next point here, so I'm gonna stop here and answer that.

00:36:08.830Byron Mackay: How should a beginner with limited hands-on experience in architecting systems prepare for solutions architecture or system design interview questions like these?

00:36:17.490Byron Mackay: I am so glad you asked that question.

00:36:20.270Byron Mackay: I'm gonna go through a few of those right now.

00:36:27.400Byron Mackay: Okay, this is… this part here is largely…

00:36:30.940Byron Mackay: Callbacks to something we had earlier.

00:36:35.370Byron Mackay: But, so let's… let's… I'll move on here, to this part. How do you do this?

00:36:43.350Byron Mackay: First off, this is not in order. Don't do this in order. These are just…

00:36:49.090Byron Mackay: Some helpful resources for you.

00:36:52.090Byron Mackay: There are a couple books I would recommend.

00:36:54.970Byron Mackay: Screenshot this page.

00:36:58.090Byron Mackay: Read, read System Design Interviewed Volume 1 and 2.

00:37:02.390Byron Mackay: Very, very helpful. Designing data-intensive applications. I…

00:37:08.610Byron Mackay: I think this book is phenomenal.

00:37:11.750Byron Mackay: I think this book is really hard to read. I wouldn't read this as your first book. Let me put it that way. I would make sure you have a very good foundation first.

00:37:23.560Byron Mackay: Before you go into this.

00:37:27.100Byron Mackay: Understanding distributed systems is another one I'd also recommend.

00:37:31.490Byron Mackay: I've talked about this, number two, I talked about this a lot. Learn all the features of your language. So, I mentioned things like description unions, this is for TypeScript records, type nearing, that's all TypeScript…

00:37:42.570Byron Mackay: jargon, if TypeScript is not the one you're using, go learn it for Java, go learn it for whatever else you're learning.

00:37:49.980Byron Mackay: But they exist, these features exist for a reason, right? Somebody really smart, probably a lot smarter than me, definitely a lot smarter than me, put these together, and there's a really good intention around it. Understanding that intention and when do you leverage those things, becomes very important, and LLMs will generally regress to the mean, will not always leverage those.

00:38:10.050Byron Mackay: But, if we're aware of them and understand how to use them.

00:38:13.320Byron Mackay: then we can just dictate to the LLM how and when to use those appropriately.

00:38:18.650Byron Mackay: Very, very powerful there.

00:38:20.580Byron Mackay: Number 3. This is where I would actually start.

00:38:24.740Byron Mackay: Use Claude as a teacher.

00:38:26.910Byron Mackay: Okay?

00:38:27.950Byron Mackay: And when you, coming back to Tanya's question about where do I begin.

00:38:34.290Byron Mackay: I'm gonna give you a resource tonight to help you do that.

00:38:37.940Byron Mackay: And I will talk about that here, I think, in just the next… actually, I might do it right now, might be the next slide.

00:38:46.680Byron Mackay: Actually, I may have missed this part in the slide, so I'm going to tell you, but we'll come back to it in a second. The fourth thing you do is, once you kind of feel comfortable, try deploying this for real. I think Railway's actually a really great place to start.

00:38:58.590Byron Mackay: I always get nervous… I know AWS is where most people are, GCP, Azure. I get really nervous on those, because I've had, as anybody who's probably used them to an extent, has used those, I have definitely cost a lot more money than I intended to.

00:39:14.310Byron Mackay: And and I know many other people who have done the same thing.

00:39:21.210Byron Mackay: So I get really nervous about using AWS or Azure to try to do anything new, unless I am really comfortable. Like, I can spin up a Lambda just fine, right? But if you're… if I start to venture outside of that too much.

00:39:35.370Byron Mackay: I get nervous. I get nervous.

00:39:38.470Byron Mackay: So…

00:39:40.660Byron Mackay: But Railway's great, because they handle a lot of the behind-the-scenes work that you don't need to worry about. You can still practice with different databases, you can practice with different servers, and all these different configurations, so it's a really nice place to kind of live.

00:39:56.040Byron Mackay: and practice in. It does cost a little bit of money, but you're probably not gonna spend the…

00:40:02.180Byron Mackay: Thousands? Tens of thousands?

00:40:05.270Byron Mackay: Hundreds of thousands that I've seen some people spend by accident.

00:40:10.820Byron Mackay: Okay.

00:40:14.080Byron Mackay: So, I'm gonna send out a resource, but here's what the resource is.

00:40:18.480Byron Mackay: I'm gonna send out a resource after this that is a top-to-bottom, list of… System design, topics.

00:40:28.630Byron Mackay: And here's what I do, that I recommend you do.

00:40:31.570Byron Mackay: And I still do this, even today. So, it's right and true method here.

00:40:37.590Byron Mackay: You put this into Cloud Chat, or ChatGPT Chat, or whatever it is, and you give it this document. And this document, again, is going to have just a list of all these topics of system design.

00:40:47.800Byron Mackay: And you're gonna… you can say something like, okay, I want to get through all of this, and I want to understand it. You are… yeah, give it a role, right? You're going to be a teacher, an expert tutor that's gonna help me understand these, and I would encourage you to say.

00:41:03.540Byron Mackay: I want you to occasionally quiz me, or rather, I will occasionally ask for quizzes to help me check my understanding of each topic, and occasionally I will ask for case studies.

00:41:15.510Byron Mackay: Those two alone will go a long, long way to learning this. So, to Tanya's question, and anybody else wondering…

00:41:23.210Byron Mackay: That's how I would address it. I would set it up that way.

00:41:26.730Byron Mackay: And then I would say, let's start from the top. I think the first one's gonna be databases, so they'll go through databases, and it'll go through each kind. And as you go through each topic, at the end, you can go back and forth and ask all the questions you want to, get all the clarification you need.

00:41:40.010Byron Mackay: And then at the end, you can say, okay, quiz me. Quiz me right now. Let's find where the gaps are, right? Coming back to what I don't know what I don't know.

00:41:51.580Byron Mackay: Even after you learn it, sometimes you don't realize you don't know it, so get the questions, take a quiz, and based on those answers.

00:41:58.590Byron Mackay: You say, okay, anything I got right, put into another document called mastered.

00:42:03.640Byron Mackay: And anything I got wrong, put into another document called Review.

00:42:08.950Byron Mackay: And then you keep going. Okay, next topic, next topic, next topic.

00:42:12.730Byron Mackay: Every once in a while, after a few topics, I encourage, and you can do it after every topic if you want, but sometimes some topics are simpler than others, I stop and go, okay, give me a case study.

00:42:22.010Byron Mackay: Give me a case study to help me actually implement this. And, of course, that generates some pretty decent case studies.

00:42:28.760Byron Mackay: Usually not very… anyway, I won't get to that, because your experience is probably going to be a little different than mine, but in terms of the output the LM gives you. But those case studies are really critical, I find, because it gives you these problems where you have to make a decision, and it's not always clear what to do.

00:42:44.650Byron Mackay: Which is great, because life's like that as well.

00:42:47.610Byron Mackay: So, it produces this case study, you work through it, and it can give you great feedback and highlight areas where, hey, you missed this, you missed that. Again.

00:42:55.990Byron Mackay: If you miss anything, put it into review.

00:42:59.140Byron Mackay: And you can continue that way, just going through it.

00:43:04.080Byron Mackay: After a few days of this, you can go back to it. If you find the time every day, you do this for 30 minutes, do it on a walk.

00:43:12.720Byron Mackay: Do it between sets if you work out.

00:43:15.660Byron Mackay: Just find little pockets of time to do it, and after a few days, you say, okay, let's go over everything in review.

00:43:22.170Byron Mackay: Give me a quiz on everything in review, let's review it. And you'll review it, and it'll start to strengthen those connections in your brain to help you understand the material better. That's spaced repetition for everybody who knows that concept.

00:43:34.360Byron Mackay: Anyway…

00:43:35.970Byron Mackay: That has helped me solidify so many topics that I've always just kind of let exist without really digging into it.

00:43:43.950Byron Mackay: And you can go through this full list, and it can just keep going. After every topic, you can say, let's go deeper, or let's do an adjacent, or let's just go somewhere completely different, so it really gives you a lot of flexibility, to go into these different spaces.

00:43:58.010Byron Mackay: Anyway, Tanya, that's where I would start, and for anybody else, that's where I would start. I'm gonna have that resource be sent out to everybody after this, and it'll have…

00:44:08.060Byron Mackay: It'll help you get started.

00:44:10.360Byron Mackay: From there, I would actually start, recommend starting with that, and then going to the books.

00:44:16.270Byron Mackay: Reason being is because those books will assume you already know that.

00:44:19.460Byron Mackay: So start with that, get Claude, get Claude to help you get up to speed.

00:44:23.420Byron Mackay: Then go to the books, and the books are fascinating, because the books cover topics that Claude does not even realize exist.

00:44:29.180Byron Mackay: Okay?

00:44:30.780Byron Mackay: Alright.

00:44:32.210Byron Mackay: I'm gonna pause there for a second and go through some questions here. So, give me just a second.

00:44:38.760Byron Mackay: And then I have a little visitor. So… one second, sorry.

00:44:42.820Byron Mackay: Hate.

00:44:43.980Byron Mackay: Can you go upstairs? I actually cleaned up the toilet. Good job. Alright, you need to go upstairs now. Your niece now. I will after. Okay, I'll see you later, okay?

00:44:56.390Byron Mackay: Bye.

00:45:00.960Byron Mackay: Alright.

00:45:03.080Byron Mackay: Let me… oh, it looks like a lot of questions have been here, let's see. I'm just trying to find my spot here. Okay, here we go.

00:45:11.470Byron Mackay: Alright.

00:45:12.780Byron Mackay: Let's see…

00:45:15.020Byron Mackay: Daniel, Opus, I've… can't remember which one I've been using. I want to say I've been using Opus, but maybe I've just been using Sana, and it's been fine, I think, for myself. Opus 47, yeah, there's…

00:45:27.190Byron Mackay: I don't know, I don't feel like it… they did a great job with that one. I'm with you there.

00:45:32.310Byron Mackay: Okay, let's see.

00:45:37.920Byron Mackay: Okay, yep, there are… Peyton points out the certifications, those are out there too.

00:45:43.640Byron Mackay: Very, very helpful.

00:45:48.010Byron Mackay: Awesome, awesome.

00:45:53.270Byron Mackay: Okay, I don't see a lot of questions. A lot of great comments, though.

00:45:56.420Byron Mackay: Okay, Lauren has a question here.

00:46:00.270Byron Mackay: I've been using ChatGPT to teach me about some design combined with my own learning of articles and YouTube videos and do mock interviews. Excellent. Well done. Do you think it would help me if I build my own system so I can see how I can fail and scale? Yes.

00:46:15.590Byron Mackay: Absolutely. It's not enough just to know, we've got to be able to do, right? We have to understand there's nothing like actually doing to show you what you don't know, and that's, that's oftentimes where we get stuck. So, it's wanting to walk the walk.

00:46:29.700Byron Mackay: Talk the talk.

00:46:31.670Byron Mackay: but it's something else entirely to sit down and actually implement. So I would recommend trying to set up some simple applications where you can test some functionality there.

00:46:41.830Byron Mackay: Sometimes when I do mock interviews, I draw a blank. Yeah, that's… that's… that would be helpful with just some practice. And I think you're doing that.

00:46:51.400Byron Mackay: But yeah, that's what I would… that's what I would go with. Definitely go with some implementation. Go… start simple.

00:46:57.400Byron Mackay: It doesn't need to be huge, but definitely be intentional, right? If you want to test,

00:47:03.840Byron Mackay: you know, encryption, right? Try various encryption methods out, that exist. Of course, learn security, right? If you don't know what OWASP is, that's a really good place to start.

00:47:17.760Byron Mackay: You know, understanding all those things so that, you can have that context and build on it when… and again, even if you're not, like, a system expert, you're just building the software, still great to understand what those… those different pieces are so that you can… you know how your code is, is interfacing with that.

00:47:35.540Byron Mackay: Alright, let's see here…

00:47:39.600Byron Mackay: Do, Tonya asked, do professional cloud certifications from Amazon or Google add value? And Daniel answered, and I would agree, and big companies, yes.

00:47:49.050Byron Mackay: It's not bad to have them, doesn't hurt you to have them. Nobody's gonna think less of you if you have them. So, always good.

00:47:56.840Byron Mackay: Alright. Let's see here… Alright, let's see. Is there a question here?

00:48:07.950Byron Mackay: Okay, let me keep going…

00:48:14.210Byron Mackay: Rob, thank you for your comment. Yeah, apparently my daughter cleaned the toy room, so… we're a happier family now.

00:48:23.780Byron Mackay: Okay, so Josh, brings up, mentioned at the beginning that AI-authored PRs produce 1.7 more issues in humans. Wondering if this is related to the fact that AI-authored PRs are an average 1.5 to 2x the size of humans' PRs. More code doesn't mean better. Hope CEOs will take notes sooner rather than later.

00:48:42.010Byron Mackay: I would agree with that to some extent, yeah. I don't know if it's this… it's… I don't think it's the sole, sole reason, but I've definitely seen PR bloat, as I'm sure many of us have.

00:48:51.380Byron Mackay: And that can definitely lead to a lot more issues. The more code, the more opportunities for failure, right?

00:48:57.500Byron Mackay: Okay.

00:49:00.570Byron Mackay: Alright, those are all the questions. Alright, so, I want to make a couple of…

00:49:08.280Byron Mackay: plugs here at the end of this. Again, this recording will be available, and I will also make the slides available, and I'll make this resource that I'm talking about available that you can drop into any chat and say, let's get started on learning system design.

00:49:23.050Byron Mackay: Now… a couple things. First off, I want to, highlight,

00:49:30.500Byron Mackay: And Sunny, stop me if I'm stealing your thunder here, but I want to highlight that we have Gauntlet that we would… if you are interested in learning more, and having a lot of time to dedicate to just learning AI, learning how to leverage it, learning how to build AI products.

00:49:50.360Byron Mackay: Learning how to be a more proficient software engineer.

00:49:55.190Byron Mackay: Come and talk to us at Gauntlet.

00:49:57.230Byron Mackay: Come and talk to us, we will help you get to that where you want to be. We'll help you get placed at great companies with great salaries for engineers.

00:50:05.370Byron Mackay: I also want to put another plug out there.

00:50:08.380Byron Mackay: for,

00:50:11.100Byron Mackay: for a special event that we're doing in Austin in, in about two and a half weeks here.

00:50:17.710Byron Mackay: We're calling… this is an event that… where you get to come and do gauntlet for a day, and we're calling it Fired Festival.

00:50:24.770Byron Mackay: This is, it is an opportunity… it is a chance for you to come in and learn, have an opportunity to learn about AI, learn about how to build it, and just experience our learnings the way that we learn here at Gauntlet.

00:50:38.880Byron Mackay: and then have time to build and work alongside, current Gauntlet students and also our staff.

00:50:45.930Byron Mackay: to really push the limits of AI, and really gain a lot more knowledge and understanding, and also learn about the job market, and what's out there, and what companies are looking for. We're gonna have some, we're gonna have somebody come in from, from another company, a CEO, and talk about what they're looking for.

00:51:05.720Byron Mackay: And, and software engineers these days, that'll give you some very valuable insight. We're gonna have some of our instructors talk about Agentech harnesses, and how we can leverage those.

00:51:15.490Byron Mackay: some of the details that go into that. We're gonna… you're gonna have an opportunity to come into our office and learn right alongside with us. It's a great opportunity, to come in and just learn, or if you want to see the Gauntlet experience and what it's like.

00:51:29.250Byron Mackay: We will feed you, and we will have prizes at the end. So, awesome networking opportunity. If you live in Austin, and you want to apply, or you want to come to the Fired Festival, come on in. We'll post a link, and you can register there. It's going to be a lot of fun. Really excited for it. So…

00:51:48.800Byron Mackay: Alright, I just, put a few more feelers out there. I'm gonna check one more time for some questions. Sunny's got some of those links there in the chat.

00:51:57.390Byron Mackay: So I want to… let's see, I did see, Daniel had a question here. Bit of a tangent, but can you comment on what you've seen with AI-proficient, super icy engineers and Comp?

00:52:09.920Byron Mackay: Yes, they do get some pretty high… we've seen some amazing, highly high salaries.

00:52:18.250Byron Mackay: what I've seen those individuals do, I think there's definitely… they have, kind of, all of the skill sets in space, from communication, system design, good coding, acumen. They have the experience that backs it up, right? They've done a lot of things where they've built a lot of great things.

00:52:38.370Byron Mackay: a lot of scalable things. Those are just skills that…

00:52:42.470Byron Mackay: you know, not everybody even has an opportunity to… to work at such high… high levels of scale, and those people are highly sought after. So, I think it just comes with opportunities, I think it comes with knowing, know-how, and, just taking, and again, being a really good communicator with somebody that can work well with leadership.

00:53:01.760Byron Mackay: But yes, it's interestingly enough, I've seen… I've seen… I would say, generally speaking, salaries have gone up. And on the other side, the number of positions may have… it feels like theirs are decreasing.

00:53:17.760Byron Mackay: But if you can do it, then those salaries go up.

00:53:20.400Byron Mackay: Quite a bit.

00:53:22.440Byron Mackay: I have seen it with C. As far as other companies, I can't recall the exact companies.

00:53:32.260Byron Mackay: But I have seen things close to that number, or even half that number or more. So, they're definitely out there.

00:53:41.210Byron Mackay: Okay.

00:53:43.570Byron Mackay: Josh asks, is system design a strong emphasis in the Gauntlet AI program? Are there projects in the program that help strengthen system design acumen? Great question, yes.

00:53:53.300Byron Mackay: It is a stronger emphasis, it's becoming stronger as things go on. We've modified the way our assignments work so that you have to think through problems more and implement more system design thought. We have an app that we use internally right now to help learn.

00:54:08.820Byron Mackay: System design principles, and to help people get caught up.

00:54:12.090Byron Mackay: So, those are all things that we do internally. We also talk about it, and we're developing more curriculum right now to address the system design need there. So, yes, very much something that we focus on.

00:54:25.420Byron Mackay: Along with all of the other AI topics that we… I've always covered.

00:54:31.150Byron Mackay: Mike asks a forward-looking question here, when will we know the dates for Cohort 8? Okay.

00:54:37.720Byron Mackay: Let me think about this, because I have to remember what… we're in 5.

00:54:41.760Byron Mackay: 6 and 7 will be this year, and 8 will be the beginning of next year. Look for late January.

00:54:48.400Byron Mackay: I think it's about when that one will be happening, so…

00:54:53.810Byron Mackay: Alright, Pedro asks, there is any way to learn from Gauntlet if I am not authorized to work in the US, even if I have to pay for it? That is something that I would say you probably could figure out without us, quite frankly.

00:55:07.820Byron Mackay: So, if you require sponsorship to be here, or, sorry, to work here, excuse me, then that's… then that's something that… where you would be authorized.

00:55:18.500Byron Mackay: And generally speaking, if you're… If you're not,

00:55:26.300Byron Mackay: Yeah, generally speaking, if you need a sponsorship, then… then yeah, that's… that's kind of how we would go off of it. There's a… might be one or two exceptions, but very few and far between.

00:55:36.160Byron Mackay: Alright.

00:55:37.380Byron Mackay: Let's see.

00:55:39.520Byron Mackay: Is the program at Gauntlet only for software, engineers? asked Tanya.

00:55:45.300Byron Mackay: it does focus on software engineers as it stands. Now, that being said, I have had a few people

00:55:53.690Byron Mackay: Phew.

00:55:55.150Byron Mackay: Come in with more of a product management background,

00:56:00.460Byron Mackay: you know, less of an engineering, but some engineering. Those people, if they are willing to kind of catch up really fast.

00:56:09.300Byron Mackay: And they can do it. It's harder… a lot harder, though, and not everyone makes it. But it is possible, it just takes a lot of drive, a lot of grit, a lot of willing to…

00:56:20.240Byron Mackay: A lot of willing to just know when you don't know, right?

00:56:25.370Byron Mackay: But we do focus mainly on software engineers.

00:56:29.850Byron Mackay: Mina asks, any prerequisites for participating in Fired Festival?

00:56:34.060Byron Mackay: The biggest one is you have to be in Austin, because it's just in Austin. We're gonna have it at our headquarters, in Austin, so, I would…

00:56:42.870Byron Mackay: reach out there, apply, and and you'll hear from me. So, when I say, yeah, just let… yeah, so just go to that website, fill out the form, and we'll… we'll get you, we'll get you put in.

00:56:56.710Byron Mackay: Alright.

00:57:01.800Byron Mackay: So, yeah, and about Fido Festival,

00:57:07.510Byron Mackay: It's… it's… it's really… we're gonna be doing a lot of events, software engineering as well, so if you're not a software engineer, you're probably gonna feel lost.

00:57:16.690Byron Mackay: Quite frankly, but, it's, it's, it's definitely something that I feel,

00:57:25.730Byron Mackay: But yeah, so if you don't have any, then I probably wouldn't consider too much, but if you do, then why not? Come on in.

00:57:31.980Byron Mackay: Is there a place for SDETs at Gauntlet? For those of you who don't know what SDTET is, it's Software Development Engineer in Test. So this is your QAs, kind of engineering,

00:57:45.470Byron Mackay: I would say… I think there actually… actually really is. And because… the biggest thing that stands out to me is in the world of evals.

00:57:54.380Byron Mackay: Llms require a lot of testing to ensure that they're good, and we need good evals. It's a full-time job.

00:58:02.210Byron Mackay: It was my job at the last company. I was part of the platform team, and I slowly migrated over to the evals, because I was the only one that was willing to really take on that kind of beast there. It's a big need, and people are willing to pay a lot for it.

00:58:16.030Byron Mackay: Silis, yes, there is this fired festival… yes, there is a little bit of play on words there with the other one. But it is definitely… it's something to help us as an engineering community to really understand the market and how we can continue to be successful in this market as it changes.

00:58:36.650Byron Mackay: Alright, lots more questions. I probably don't have enough time, so I'm gonna really quickly try to answer a few, and then we're gonna end it. Can I build something during the first, during the Fyre Fest?

00:58:46.680Byron Mackay: thing side-by-side, an expert willing to help. I have something I want to build. So, we will have a specific project that we're building. We're going to talk about a very specific topic, and we'll dig into it really deep that we feel like is

00:58:58.520Byron Mackay: kind of where everything's starting to go in terms of AI, so that would be something… but it is open to where you can incorporate… we're not… we're saying you have to reveal it this certain way. We're making it very open. You can apply any

00:59:12.130Byron Mackay: vertical, any, other industry to it. So, you probably can, yes.

00:59:17.710Byron Mackay: Is Gauntlet right for me if I'm a Scrum Master and familiar with web design and cybersecurity? That cybersecurity one stands out to me. It might be. Apply, let's take a look at your profile and see what we've got.

00:59:29.440Byron Mackay: Side of this, have you any thoughts about rolling out to more geographic areas? For Fyre Festival, I'm assuming?

00:59:35.830Byron Mackay: Yes, this is our first one, so we're starting in Austin. Be fun to go outside of that as well.

00:59:41.550Byron Mackay: Let's see here…

00:59:44.470Byron Mackay: I got a good system analyst and design textbook and learned to draw data flow diagrams. Anything else? Or get a good book and learn those data flow diagrams? Yes, as you start going through the topics, you'll identify more and more things that you can learn and build, and it'll be, they'll become very apparent.

01:00:03.650Byron Mackay: Hank, you wanna speak a few minutes after this? I've gotta run, actually, but…

01:00:08.840Byron Mackay: Feel free to reach out to us, and we can get you the answers that you need.

01:00:15.000Byron Mackay: Let's see, a sample application for Gauntlet for America.

01:00:18.930Byron Mackay: We are still automatically considered for both.

01:00:21.560Byron Mackay: Blaze, that's the answer to your question there. But if you have a preference, let us know. If you want to go out for America, I think it's an excellent program myself.

01:00:29.430Byron Mackay: I would definitely just let us know.

01:00:31.440Byron Mackay: Alright, when I started scrolling at this point, it was 18, and now there's 21 new messages. So, I am… we are just over time, I don't wanna… I don't wanna… I gotta… I gotta hop myself.

01:00:41.690Byron Mackay: But thank you all for coming, really appreciate y'all being here. This is gonna be something that we'll talk more about later, but you all have a great, great evening, and we will go, have, have a great evening, and we'll see you next time at night school. We will see you later.

Download transcript (.vtt)

FAQ

What is AI system design? +
AI system design is reasoning about how a system behaves under load, scales, fails, and changes over time, which is the architectural judgment that counts more now that AI writes the code. It is the skill that separates juniors from operators.
Why does system design count more now that AI writes code? +
AI-authored code ships more bugs and security holes, and exploit windows have collapsed, so companies now hire engineers who can reason about how systems behave, scale, fail, and change over time.
What do the numbers actually show? +
AI-authored pull requests ship about 1.7 times more issues and nearly three times more security vulnerabilities than human-authored ones, and exploit windows have dropped from roughly two years to under two months.
Why does that push hiring toward system design? +
When feature output is cheap, architectural judgment becomes the scarce and valuable skill, so interviews probe deeper on design.
What system design fundamentals do interviewers probe? +
How a system behaves under load, scales, fails, and changes over time, with the focus on the trade-offs behind those choices rather than memorized diagrams.
How can you use Claude to learn system design? +
Treat it as a tutor and have it generate quizzes, walk through case studies, and run spaced-repetition review so the concepts actually stick.
What is the step-by-step workflow? +
Pick a topic, have the model teach and quiz you, work through real case studies, and then space the reviews so you retain it, all of which the lesson walks through.
Is this only useful for interview prep? +
No, since the same judgment that passes interviews is what keeps AI-assisted systems reliable in production.

What's next?

Keep building with the rest of Night School, or apply to Gauntlet — ten weeks of technical intensity with the best AI engineers we can find.