Monday, February 08, 2010

How the Mainstream Lost Software Development Productivity

(Continued from: Denying Productivity)

The greatest software development productivity comes from software that can be easily and readily understood. When you need to make changes to software, you can easily and readily understand the repercussions of those changes. When you need to add new abilities to software, you can easily and readily understand where the additions need to be made. When you're new to an existing project, you can easily understand the code that has already been written by your new team. Understanding how to design software so that you're able to easily control and observe it gives you the ability to create software that can be understood, and software that doesn't obstruct productivity.

Despite the number of software developers who understand the mechanics and principles of software design that cultivate software development productivity, the vast majority of software developers don't understand how to wield the techniques, or aren't convinced by the promises, or simply haven't heard about higher order developer productivity at all, let alone how to get it.

Somewhere along the line, the software development mainstream lost its access to productivity, and the vacuum was filed with counter productivity or productivity proxies.

Two things contributed to the mass extinction of software development productivity in the mainstream: Eggheads and Shills. The eggheads allowed a divide to open between the intellectual haves and the have-nots, and the shills exploited that gap, filling it with misinformation and misrepresentations, verily redefining "productivity" and supplanting it with mere efficiency.

Yes, these labels are chosen for shock value. Specifically because this is an issue that is largely rooted in unconsciousness: the unconsciousness of failing to communicate, and the unconsciousness of deeper meaning.

I've learned quite a bit about software development and design from some very insightful and intelligent people. I'm grateful to the authors who've been my long distance mentors and who have provided me with inspiration. I've met many of them in person over the past half decade or so and these encounters have deepened my pursuit of understanding ever further.

Now that I have a much better understanding of what it is that they taught me, I'm shocked at how poorly this material was often communicated and at how our self-indulgent vanity continues to get in the way of broader communication of ideas that are vital to contemporary society's productivity at a time when productivity is so desperately needed.

I use the term "Egghead" with some affection and admiration, but I also use it in the hopes that some of the great teachers of software development and design will come to learn that they've barely even scratched the surface of the full audience that they need to reach. Eggheads write and speak for other eggheads. The language they chose and the names given to design principles, as well as practices and patterns are stimulating to other eggheads and yet utterly obstructive to the other part of the software development population. When you measure the Egghead population against the mainstream population, the Egghead population doesn't appear to be much more than a rounding error. There's a lot of work to do, and we can do it much better.

Eggheads preach to the choir, and this is often so stimulating that they never leave the echo chamber. They don't learn how to reach the mainstream. They hope that the people they do reach will somehow reach the mainstream, but those people usually end up emulating their egghead heros and become eggheads themselves. The circle expands slightly, but not merely enough to have the meaningful effect on software development productivity that their knowledge and understanding should already have had.

Their communication style is rife with self-stimulating, overly-academic legalese that does little more than enforce an over-estimation of the value of the terminology itself rather than subjugate the terminology to the absolute must of communicating simple and powerful ideas to the mainstream. Eggheads don't like to engage directly with the mainstream. They don't learn to talk to the mainstream and how to teach the mainstream. In this, their self-indulgence leads to terrific loss for human society and is quite possibly one of the most negligent acts perpetrated against the potential for software development productivity.

The divide between the Egghead echo chamber and the mainstream has been readily filled by Shills. While the Eggheads fascinate themselves with themselves, the Shills convince the mainstream to buy snake oil solutions to problems that could be easily solved with plain old soap and water.

The potential for abuse and the leadership vacuum left behind by the Eggheads' self-glorification is readily taken advantage of by Shills willing to convince the mainstream that mere efficiency tools are good solutions for productivity problems. They further exasperate the productivity problems that result from applying efficiency solutions to productivity problems and use the ensuing mainstream customer panic to justify yet more efficiency tools. The mainstream becomes ever more isolated from more meaningful understandings of software development productivity and continues to spin out of control on the back of software systems and software projects that are themselves out of control.

Between the Eggheads and the Shills, things look pretty grim. Those who have good answers to the software crisis are unwilling to forgo the stimulating academic formalism in favor of connecting with the mainstream, and those who would take advantage of ignorance do take advantage of ignorance.

Neither of these two archetypes can be counted on to fix the problem. Eggheads are far too ensconced in the comforts of elitism and Shills have built behemoth networks of companies all selling the same pack of lies, damned lies, and demos. The momentum and justifications of either aren't likely to change anytime soon.

What's needed is a new generation of productivity missionaries who are willing to master the field of knowledge and are willing to learn to connect with the mainstream. They're willing forgo the all-consuming pursuit of elite celebrity and serve society by taking on the software crisis head-on as a matter of honor and duty in the face of continuing negligence and abuse on all sides. They are willing to speak plainly and openly and forgo the perceived legitimization that comes from falling into step with esoteric formalism.

The mainstream has lost track of software development productivity, but it's not so far removed that it can't be recovered. It just can't be recovered through the continued self-indulgence underpinning both the Egghead and the Shill entitlement to ease and winnings in the face of disastrous effects on modern productivity. It can be recovered if pathfinders and communicators are willing to get dirty amongst the "unwashed masses" for an even greater return, and to stand fast against the encroachments of suboptimal local efficiencies bundled with six-figure price tags and promises of yet more software development pain.

Next: Productive by Design

Ampersand GT

Working with software developers and organizations to help realize the potential of software product development through higher productivity, higher quality, and improved customer experience

Learn more about my work and how I can help you at