Programmers are Getting Worse

Originally published on Tumblr.

In my last post I talked about the degeneration of Apple’s software. I notice that degeneration because I’ve been around a long time. I used pre-releases of NeXT’s operating system, which evolved and eventually became OS X. In other words, I’m what people in the industry call a “gray beard”. That may sound sexist, I realize, but I’m no more likely to grow a beard than any of my female colleagues, and they’re no more likely to be offended by the moniker than I am.

When my peers and I sit around the campfire and talk about this degeneration, we don’t spend much time debating it’s root cause. We know what it is. It’s “the kids”.

The kids, we say, don’t know anything about memory management. They don’t understand basic constraints and tradeoffs. Most of them can’t analyze the characteristics of an algorithm. Some of them don’t even seem to understand what they’re doing. They write code and test to see if it works. So where do these kids come from, and how do they end up in our office?

They come from the very best schools, and they end up in our office because we need help. Unfortunately, academia is churning out computer science graduates who understand little of what they’re actually doing. These graduates are taught to program using high-level languages and tools that not only keep them far from the underlying hardware but also shield them from the steps required to turn human-readable code into machine code. They’re rarely taught any theory, and they’re rarely taught how computers and networks actually work.

The idea that every generation is getting worse isn’t new. In fact, that idea seems to be a historical constant. I’ve read that people who programmed in machine language were incensed when the kids started using assembler. And those programmers in turn were appalled when the next generation started using higher level languages that further abstracted the underlying hardware.

But that observation doesn’t resolve the issue. Are programmers more proficient the lower-level their knowledge? And does that proficiency make them better programmers?

The answer to the first question is undoubtably “yes”, and the answer to the second is almost certainly “no”, which seem paradoxical.

The world of physical construction offers a good analogy.

A builder (BA) understands how geology and climate will affect a building, and adjusts his design accordingly. This makes him a better builder that someone who doesn’t (BB). BA, however, may be a poor user of building tools, while BB may have mastered them all. In most environments BB will produce better buildings — and by better I mean "fit for purpose” — but in extreme environments, environments prone to earthquakes and flooding, for example, BA will produce better buildings. If BB had the same understanding of the external fundamentals that might affect his buildings, he would always produce better buildings.

To the outside world, BB will appear to be the better builder, most of the time.

The same is true when it comes to software. A programmer with a deep understanding of fundamentals (PA) won’t necessarily produce great fit for purpose software, while a programmer highly skilled in the use of modern tools (PB) usually will. But when PB’s output is subjected to unexpected stress, it may turn out to be completely unfit for purpose, i.e., it may not work

This explains the paradox. Critical software is getting worse because software management fails to recognize the difference between PAs and PBs.

Although some of my peers will disagree, I don’t think it usually matters that PBs are responsible for software projects. But it matters if they’re building software likely to hit a resource wall. Unfortunately, it’s sometime hard to predict when and if that’s ever going to happen.

So does that mean that only PAs should be responsible for running software projects? No, of course not. In many cases resources are abundant, so abstraction and waste are acceptable.

So when should gray beards be required, and how should they work with the kids? And will all those kids grow up to be gray beards?

The kids are doing pretty phenomenal stuff, but the experience and knowledge of the industry’s gray beards is critical in some circumstances. Would you want your plane’s avionics coded by someone who doesn’t think about constraints? I can barely put up with my mail reader slowing down in the face of 40,000 messages. I’d be even more unhappy if my plane plunged to the ground because the auto-pilot ran out of memory.

Most of the kids won’t grow up to be gray beards. Most will move into management, and with luck some of them will recognize where their former peers need to be leveraged. I hope that some day this evolution and necessary use of resources is widely recognized.

Software development is both and an engineering discipline and a craft. Schools should teach engineering and stay out of the craft business. Industry in turn has to treat the early years of programmers’ careers as an apprenticeship. In other words, we have to go back to a time when craftsmen were taught by better craftsmen, not by scholars.

Some programmers will be good engineers, and some will be good craftsmen. Very few will master both.