The Caring Programmer's Manifesto
It’s 2021, and we’ve broken the way we program. We have more tutorials, Stack Overflow answers and programming courses than ever. Our computers are more powerful than ever. There are many software developers who can piece together a mesh of libraries to create cross-platform applications.
However, we seem to have lost our understanding of what the computer is actually doing under all that code. We barely know how the dependencies we are
npm install-ing work, let alone what the CPU is doing. More than that, if we had to develop, from scratch, one of these tools or frameworks that we use on a daily basis, we’d probably be lost.
You might think this is a good thing. After all, this means that programming is more accessible and that more people can build more software more easily, right? It is true that having a low barrier to entry is important. However, we’re also starting to see the repercussions of having completely lost understanding of the computers we are programming.
Some of the most popular desktop applications, such as Slack and Word, are frequently laggy and unstable. Our reliance on a Jenga-like chain of dependencies makes our programs fragile. In 2016, hundreds of thousands of the web’s largest packages, including React, were broken because an 11-line package named
left-pad was deleted by a disgruntled programmer, leading to worldwide outages. Today’s websites are so slow, that we are building browsers that render websites in the cloud and stream them to your computer, because even that is faster than simply rendering them directly.
You can chalk these issues up to “bad web programmers”. But bad software even pops up where you would expect it the least. It would be reasonable to think that game programmers do a much better job, since their profession centers heavily around performance. However, in 2021, a programmer released a fix that sped up GTA Online loading times by 70%, without even having access to the source code. The problem they fixed? A ridiculously slow JSON-parsing function that took four minutes to parse 10MB of JSON every time the game loaded. This issue went unfixed for years at one of the world’s biggest videogame companies, affecting more than one million players every month.
The worst part is, it’s not just the performance of our programs that is suffering, it’s also our way of thinking — this
npm install culture is actually destroying our ability to think rationally about what we’re building. In the past 7 or so years, I’ve helped hundreds of people learn to program, and there’s an issue I see over and over: people get stuck in the mentality of “what do I need to install?” or “tell me what I should write to get this to work”. When you’re in this rushed, short-sighted mindset for an extended period of time, you become unaccustomed to stepping back and truly understanding what’s happening in your code, truly understanding everything your program is doing — you’re just looking for the next hotfix. Combined with deadlines and pressure from your manager, this is a recipe for disaster.
All of this is bad because it leads to worse products, which lead to unhappy users. But there’s something I find even sadder: it’s leading people to enjoy programming less. If you have to struggle with a chain of tangled dependencies and technical debt every day, if you have to get things to work before your deadline but you don’t completely understand your own code, that can be nothing but a deeply frustrating and alientating experience. I know how that feels, because I’ve been there.
I have good news: there is a better way. We can turn our everyday work into something more beautiful and enjoyable. How can we do this? The answer depends on each and every programmer, as well as each and every project. However, I want to offer a few principles which I think are necessary.
- Profile your code and be unforgiving about performance.
I’ve seen too many programmers assemble programs together and not even care to measure their performance — it just runs as fast as it runs. This is not how you should go about it. Performance is one of the most critical aspects of user experience. You hate it when Slack or Outlook are slow to start, so why would you do that to someone else? Recognise that performance is important and figure out how to measure it. Once you’ve done this, set strict performance requirements, and enforce them whenever you make changes to the code. For example, make sure your application never takes more than 500ms to open, that your web API endpoints never take more than 100ms to respond, and so on.
- Write your own code.
So many of the dependencies that we use on a daily basis are things we could have written ourselves in a matter of hours, minutes or seconds. Very often, the perceived time save gained from adding a dependency is really not worth it compared to the maintenance cost and technical debt. You shouldn’t reinvent the wheel when you don’t have to, but if you assess the situation honestly, you’ll often find that you don’t really need to reach for npm for that UI element, for that string splitting function, for that API integration. Sometimes, not using a library means you’ll have to do a little bit of learning first to understand how to write the code. That’s fantastic — your should celebrate taking time to learn things. You’ll end up with a deeper understanding of the problem you’re trying to solve, and I think programming will bring you more joy as a result.
- Understand that programming isn’t magic.
Programming gives us a huge advantage that we don’t have in natural sciences like physics — we understand it all the way down, because we built all of it. You can always go one level deeper and read the code, all the way down to the CPU instructions. This is a very valuable thing to take advantage of. Yet so often, people seem to be afraid to dig deeper, as if they’re reaching deep into the computer and are about to break something. Stack Overflow is filled with answers claiming that X or Y library can’t be beaten because it’s doing something magical under the hood, and we shouldn’t touch it. This is the wrong mindset. Celebrate popping the hood to have a look inside. It’s just code all the way down. Well, at least until you get to the electrons.
- Understand the lower-level technologies you are building your work on.
Where to start
Maybe this all sounds well and good, but you don’t know where to start. And I don’t blame you! Fortunately, there are plenty of good resources.
- Destroy All Software.
Gary Bernhardt has created a series of “from scratch” screencasts where he builds various programming tools from scratch, including a compiler, which I find very enjoyable and interesting. The screencasts cost money, but you can watch a couple of the best ones for free.
- Handmade Hero.
Casey Muratori does a long-running series of streams where he codes an entire game from scratch in C++ with zero dependencies except the operating system. His work has also inspired the creation of the Handmade Network, where people share their joy of lower-level programming.
- clumsy computer.
I’ve also started a series of YouTube videos where I teach how to build everyday tools we take for granted completely from scratch with zero dependencies. So far we’ve implemented a regex engine, OCR with machine learning, and an extensible Twitch bot, all with zero imports.
If you’re feeling extra motivated, nand2tetris takes you through the journey of building your own computer, starting with NAND gates and ending up with a fully functional game of Tetris.
Talks and articles
- “Data-Oriented Design (Or Why You Might Be Shooting Yourself in The Foot With OOP)” by Noel Llopis.
A clear and gentle introduction to data-oriented design and the problems with OOP.
- “Why Python is Slow” by Jake VanderPlas.
A major contributor to scikit-learn and scipy examines how Python works internally, and why that can lead to performance issues.
- “CPU Caches and Why You Care” by Scott Meyers.
An absolutely invaluable look at how the reasons for a program being slow might shock those unfamiliar with how CPUs work. Plus, Scott Meyers is pretty funny.
- “Path Tracing Three Ways: A Study of C++ Style” by Matt Godbolt.
The creator of Compiler Explorer gives a very practical overview of multiple styles of programming in C++, letting you draw a lot of conclusions yourself.
- “The Thirty Million Line Problem” by Casey Muratori.
Casey talks about why our programs are slow, and how getting closer to the hardware can help.
- “Data-Oriented Design and C++” by Mike Acton.
A classic talk on how the way we organise our programs often leads to bad performance, and why OOP can lead to problems.
- Ben Eater’s YouTube channel.
Ben Eater does videos mainly oriented around hardware, such as building a video card from scratch, or writing a “Hello World” program from scratch on a custom computer. I think these can really spark interest in understand how computers work.
So, what do you think? Does that sound good to you? For me personally, having a deeper understanding of how my programs work has brought me so much enjoyment as a programmer. I’ve heard many people say that this kind of approach has led them to rediscover the joy of programming. If that sounds interesting to you as well, let’s try that out together, and build a better future for programming.