Estimated reading time: 13 minutes
“If you’re thinking about buying a book, just buy it. Don’t waste five seconds debating it. Even one idea makes it more than worth the price.”
Ramit Sethi
Among the most frequently submitted questions I get from IT professionals and recent CS grads alike is: ‘What books would you recommend for anyone interested in performance?’ In fact, I’ve answered that question enough to warrant this dedicated blog post. And I’m especially qualified to answer it given my lifelong adherence to Ramit’s aforementioned advice. Oh, I’ve bought all sorts of tech books ranging widely in price and content quality. Many were duds. Some light on depth (e.g., glorified man pages). Others light on facts, believe it or not. But every now & then I’d stumble upon a goldmine. Even among a lot of the duds sprung an occasional leak of insight which led me to a breakthrough.
It was a long trudge to reach this point. But to quote the preeminent wordsmith Shawn Carter a.k.a. Jay-Z, “Hov did that, so hopefully you won’t have to go through that.” So, by all means, benefit from the fruits of my labor as I present you my personal Top 7 list of performance books for engineers.1All book images use paid affiliate links
Book Selection Criteria
I already know what you’re thinking. ‘He’s just gonna plug the book that he helped work on for Denis Bahkvalov.’ No, no, no, that’s not gonna happen at all. While I firmly believe it’s a great addition to any engineer’s introduction to the beautiful art & science of Software Performance, I purposely left it off the list. There’s no room for personal bias on this blog.
Also, I’ve left off performance books published more than 15 years ago despite the impact they may’ve exerted on engineers and authors of subsequent works. If not for this somewhat arbitrary cutoff point, more Adrian Cockroft, Richard McDougall, and Jim Mauro references would appear here.
My Top 7 Performance Books for Engineers
This is NOT an exhaustive list of every good performance book I’ve ever read. This is only a list of the ones I most often recommend to technologists from other disciplines who express an interest in the area of performance analysis & engineering.
NOTE: Each book image is a clickable link.
1. The Every Computer Performance Book
When I first learned to box as a kid, I was excited to learn all the dazzling punch combos. But what did my coach teach me in my first lesson? Proper stance. After that? How to step forward & back. Then? How to step right & left. I don’t even remember when I actually threw my first punch! But a firm grasp of proper footwork and balance paved the way for eventually learning effective punch combos. No matter what new skillsets I picked up later on, my footwork *always* served me well for incorporating them.
It’s similar in software performance. Overeager techies wanna tinker with magic knobs & secret tunables hidden behind names with leading underscores. They want the tricks of the trade w/o first understanding the trade itself. But that’s not Performance Engineering, and those tweaks only pertain to specific software packages and versions. It’s the transferable concepts and skills which retired Performance Consultant Bob Wescott shares in his swan song.
It’s a short & witty, yet dense distillation of key concepts that will serve you well throughout your career. He could’ve rattled off nitty-gritty details of Queuing Theory, or all the technology-specific knobs he picked up along the way. Instead, Bob homes in on the essentials he most commonly encountered across engagements. That’s where this book shines. It brims with hard-earned experience instead of ethereal mumbo-jumbo or urban legends handed down from Usenet groups of yore.
And his book runs the gamut from Monitoring & Modeling to Capacity Planning & Load Testing. He even schools us on handling political issues that arise when presenting performance analyses. Yep, pesky little facts can potentially land us in hot water, even in cases where the boss requests them! At only ~200 pages, it’s a quick yet engaging & informative read for any engineer aspiring to enter this space.
2. Analyzing Computer Performance with Perl::PDQ
My point in recommending this book is not to push his Perl::PDQ tool (it’s also available in C, Python, and R these days). It’s all the foundational performance principles the author establishes leading up to the tool’s introduction that earns its spot on the list. The author I’m referring to is none other than Neil Gunther, father of Universal Scalability Law (USL). Where Wescott’s book gives us the gist of Queuing Theory, Gunther dives fully into the subject.
But this is not some academic book full of formulas but woefully lacking in real-world application. On the contrary, he cogently illustrates just how useful Queuing Theory is with ample, concrete examples (e.g., multicore/multiprocessor architectures, multi-tier web apps, virtual machine configs, benchmark analysis, etc.). You walk away understanding that anything can be modeled as a queue or network of queues and, therefore, can be reasoned about using these laws. His handy PDQ tool just makes them much easier to wield. It’s truly a remarkable performance book, and I can’t recommend it enough.
On a side note, Neil is still active on Twitter. He once used the graphs from my “5-level vs. 4-level Page Tables” article & calculated the alpha coefficient (i.e., the Contention factor in the USL model) imposed by 5-level kernel page tables for my Twitter followers. I thought that was pretty cool. By the way, USL packages are available in Python and R.
3. Understanding Software Dynamics
No other performance book evokes the mentor-mentee relationship in literary form more than this post-retirement entry (the 2nd retiree on this list) from the venerable Dick Sites. It teems with realistic examples and Case Studies which impart his wisdom gained through decades in the trenches.
The book offers a framework for reasoning about tail latency in complex software systems. He shows how building a mental model for how long things *should* take, within an order of magnitude, is essential to this framework. Without such an understanding, how can you recognize anomalous behavior that requires investigation? And it’s in this first section where many software developers will benefit most as he lays out that mental model. There he describes the performance behavior of the 5 fundamental resources: CPU, Memory, Disk/SSD, Network, and Software Critical Section. For a generation of CompSci grads entering the workforce knowledgeable in Frameworks & Design Patterns w/o even the slightest idea of how code interacts with underlying HW, this first section should be required reading.
Yet, this performance book offers much more. It covers tooling and effective methods of observing software<->hardware interplay that provide valuable clues for tail latency investigation. Finally, it wraps up with a section on reasoning from the observational data we’ve obtained. This final section comprises the aforementioned Case Studies that build upon everything laid out in the previous sections.
Because this book addresses topics at a lower level than the previous two on this list, the author had to settle upon a few technologies from which to pull examples. He chose C/C++ and Linux running on Intel, AMD, and ARM CPUs. But the lessons & principles transfer well across any technology stack. I only wish this book was available when I first entered the workforce.
4. Systems Performance: Enterprise and the Cloud
Y’all saw this one comin’ from a mile away, didn’t ya? This is Brendan Gregg’s 2nd edition, and is a performance book that works primarily from the Linux OS/kernel perspective, with comprehensive coverage of cloud technologies added. While it covers the performance characteristics and observability tools for the same 5 computing resources covered in Dick’s book, Brendan’s dives quite a bit deeper given its OS-level theme. Another difference between their books is the breadth of coverage on the idiosyncrasies of VM/Container performance analysis, whether on-prem or in the cloud.
While the aforementioned points would be enough to warrant a spot on my list, the book goes further by outlining and ranking commonly used methodologies of Performance Analysis and Benchmarking. Regarding the latter, I believe the industry as a whole will benefit from reading and revisiting on a monthly basis section 12 on Benchmarking. Anyone who has ever read my blog will understand why I say this. Lastly, all software developers should print out & frame a copy of the “7 Performance Mantras” from section 2.5.20. I know several software engineers who have done so already.
At over 800 pages, this book is a comprehensive treatment of Performance Engineering for the modern, cloud-native age. My guess is that you already have it in your library, but this list would lack credibility w/o it.
5. Pro .NET Benchmarking
Hands down, this is the best benchmarking book I’ve ever read. There, I said it. “But, but, but. . . it’s about .NET.” No, it simply uses .NET for illustrative purposes but all concepts and takeaways are largely transferable.
Look, I’ve read many performance books for engineers that dedicated a chapter or even an entire section to the topic of Benchmarking (See previous heading). And the quality of handling of the topic varied widely in those publications. But, other than a couple of huge academic tomes filled with more formulas than sentences, this is the only *practical* book entirely devoted to a competent coverage of the subject. Best practices, gotchas, measurement bias, proper analysis, tooling, etc. I particularly like that he stresses the importance of visualizing runtime distributions – performance is a shape, not a number.
But what I love most about Andrey’s book is the chapter he devotes to Statistics. As a Performance Engineer, you absolutely CANNOT offer much benefit to customers w/o a solid grasp of Statistics. But who wants to study any of those 900-page, inscrutable volumes on the topic?! I certainly didn’t want to. So, I absorbed only what was required to perform my analysis duties well. But I went about it the hard way because a book like this wasn’t available back then. But for you, chapter 4 contains 79 pages dedicated to only the amount of Stats essential for effective benchmarking. If you’ve ever read Andrey’s blog, you’ll note it’s a topic about which he easily could’ve written an entire book. So, when he declares it the Minimum Effective Dosage, believe him.
Read this book and then go back and re-read one of those lazy benchmark articles everyone loves so much. You’ll finally understand why I get annoyed by them.
6. The Art of Writing Efficient Programs
Dick Site’s “Understanding Software Dynamics” briefly touches on things to consider when crafting software for modern hardware in its first section. But its main focus is performance analysis & investigation of existing software, using Linux and C/C++ as a base. While Fedor Pikus pulls from that same tech stack, he mainly concentrates on writing original code that will run efficiently on modern hardware.
Another difference in Fedor’s book is that he only deals with 3 of the 5 main computing resources outlined in Dick’s book; namely, CPU, Memory, and Software Critical Section. He also lays a solid foundation for understanding these resources before taking us more deeply into the matter at hand. Again, while this book chose a specific tech stack from which to pull examples, its concepts, principles, and takeaways are readily transferable to your preferred tech stack. Examples of such transferable topics include algorithm selection, optimal data structures, memory models, branch prediction, lock-based/lock-free/wait-free concurrency, etc.
But the area where Fedor sets his book apart is his choice of topic for the final chapter: “Design for Performance.” Using all the lessons learned in the preceding chapters, he describes how a proper Shift-left Software Organization should discuss performance considerations w/o falling victim to the specter of “premature optimization.” After all, countless cautionary tales illustrate that performance isn’t something you can bolt on easily after the fact. He’s particularly effective when he expounds on examples of navigating tradeoff considerations that a team may encounter during these design meetings.
One reviewer stated that this book is a perfect companion to the one which we agreed I would not include in this list, and I must say that I agree.
7. Troubleshooting with Wireshark
I know what you’re thinkin’. “Why on earth would he include a book about a network diagnostic tool in this list?” Wait! Hear me out! In this era of highly-distributed, microservice-based software, who of us can afford NOT to consider network performance? Although several Wireshark books exist, I only recommend this one due to its emphasis on performance debugging.
Do you realize how much application performance telemetry you can glean from a Packet Capture (PCAP)? Telemetry that doesn’t require impacting application runtime with a single line of additional code? Do a web search right now. You’ll find videos of people analyzing SQL DB query performance using PCAPs. HFT organizations use PCAPs to calculate tick-to-trade latency. And what’s the most popular tool for analyzing PCAPs? Wireshark. And ever since the guys at NTop integrated an eBPF plugin/library into Wireshark, it’s even useful for examining Linux container network traffic, too!
Most, if not all, major cloud providers offer some type of Traffic Mirroring service for usage with Wireshark. On-prem usage is as simple as deploying an optical network tap or a Layer 1.5 Exablaze or MetaConnect switch to copy traffic non-disruptively to a capture host. Or you could always just run “tcpdump” on the application host itself, though this method will impact performance.
“But don’t I have to be a CCIE or something to work with Wireshark?” Absolutely not, as author Laura Chappell expertly demonstrates in this excellent book. She even provides a link to her immensely useful Performance Troubleshooting Profile which plugs right into your local Wireshark installation. And the community provides a plethora of protocol decoding plugins which turn what might’ve appeared to a developer as gobbledygook into useful, actionable information.
It’s an essential book for performance specialists.
Further Reading
While this list comprises my personal list of top performance books for engineers, there are far more blogs I’d recommend to keep abreast of all the new hotness.2Most authors in this list host some of my favorite blogs In fact, I find myself reading more online articles and white papers than anything else. Books simply can’t compete with these other mediums when it comes to up-to-the-minute information. But these books provide a foundation from which to better grasp insights you’ll gain from these alternative venues, as well as from other technology-specific performance books for engineers (e.g., Oracle, Java, or MySQL Tuning books).
While your recommendation list may differ from mine, we can agree on one thing for sure: the learning process never ends. And you know what? I dig it.
- 1All book images use paid affiliate links
- 2Most authors in this list host some of my favorite blogs
