Management is central to the productivity crisis we face in software development. It's a crisis that pervades software development so profoundly that we usually don't recognize it. It's like water to a fish - not something that our attention is drawn to, not something that we notice. It's the environment that encloses everything we do. It's not that we see it as normal, we just fail to see it at all. Frustration, burnouts, blown budgets, missed expectations, and dissatisfied customers; it's just the way that most software development is done.
We start with the best of intentions, and despite our intentions, we usually end up in the weeds. We end up in the weeds most of the time because that is where software development teams are directed. It's because of one salient, painful truth about software development:
Software development managers don't understand the work of software development sufficiently to be able to direct the work of software development, and the value of hands-on work experience they've had in the past decays so rapidly that it's increasingly unsafe to make new decisions based on it.
Said more plainly: at a time when senior staff should be best informed by their lengthy and varied experience, they remove themselves from the work of software development long enough to lose the understanding of how day-to-day software development work is done. They lose the knowledge needed to direct the work of day-to-day software development. A certain egoic mania takes root that leads senior staff to feel entitled to believe that their insight has moved beyond the necessity of mere practical work experience.
How long does it take to become unqualified to set expectations for software development work? Not long. It happens as quickly as six months and often not longer than two years. This is why the proposition of Self-Directed Work Teams that has been popularized by new, opportunistic interpretation of Agile Methods has become so attractive in software development - even though SDWT is a well-known failure mode in most development industries and is known to be sustainable for no longer than a couple of years in the majority of cases (there are exceptions, but exceptional organizations usually aren't the rule for unexceptional organizations, despite our best hopes and ideals).
Plotting software development management on a continuum between the extremes of Managers and Bullies, we don't find many managers who are only either extreme. A software development authority is rarely either only a manager or only a bully, but software development managers are usually weighted more toward the bully end of the spectrum. It's an innevitable consequence of having lost interest in the details of software development work, and having become quickly dated as a consequence.
Someone who sets expectations but who isn't able to show workers how to fulfill expectations isn't managing, he's using the implicit threat of crossing an authority to make demands without knowing if those demands can be met. And he's hoping for the best. But we need more than hope to manage software development. We need to know that the work being done every day is the work that leads to the desired outcomes.
Setting expectations without knowing how to meet those expectations almost always leads to unqualified or unrealistic expectations. Using authority to require unrealistic expectations to be met always creates a compounding accumulation of obstructions to subsequent work. It's a sure-fire path to lower productivity. Most software development organizations live perpetually under this cloud of depressed productivity.
A bully sets expectations for workers but doesn't have the relevant and necessary capacities to help those workers fulfill the expectations. A bully has been disconnected from work for long enough that he (or she) is no longer able to personally act to ensure that his workers meet his expectations - to show them how the work is done, and to qualify the work that they do.
In other words, the person who needs to be the most experienced and most expert software development worker on the team is often the least capable.
A direct manager knows how the work should be done. He may know this better than his workers. When necessary, a manager can sit down with the worker's current tools, and teach the worker how to meet expectations, and to teach the worker how the specifics and tactics are tied to a greater whole. A bully hides behind assertions that his time and intellect are too valuable to be spent on such trivialities as the "how" of doing the work in his purview.
It's fair for a person with authority to no longer have the capacity to ensure that workers are materially able to meet expectations, but then that authority must defer to someone who does have the necessary and relevant capacity. And this is an entirely undesirable circumstance. It might be the circumstance that you're dealing with, but it's not the one you should settle for.
An authority who isn't capable of knowing whether and how work can be accomplished and whether expectations can be met is qualified to set only summary expectations. An authority who over-steps the material bounds of his capacity will continue to inject productivity obstructions into his organization's workflow by causing workers to make unsafe and counterproductive decisions that innevitably lead to rework, overwork, and relearning.
A bully believes that his workers are soft and lazy, and often puts extra pressure on workers believing that this is what it takes to get merely nominal productivity. A manager recognizes that workers who retreat from work are likely facing some distressing obstructions in the workflow. He uses his skills as a teacher and his insights into the work as an expert worker to help his workers to recognize and clear obstructions.
It's often hard for us to accept the cold reality of the depressed productivity in our organizations. We fail to manage software development as a product development discipline and tend to manage it as a manufacturing discipline (usually without even understanding or recognizing the difference), undermining productivity at every turn by tuning the wrong variables and turning the wrong organizational and procedural knobs.
In an environment where management leans more to the bully side of the continuum, we see work teams stretch the intentions of "self-organization" into the realm of "self-determination" or "self-direction". It's not necessarily an unwarranted reaction, but its a reaction to a root cause that should be the focus of organizational and procedural improvement. It's in these environments that we also see a need for coaches to fill in the management void, but this is also a lesser palliative that doesn't necessarily get to the heart of the matter.
A manager is someone who balances qualified and expert worksmanship with the ability to teach and is comfortable with using directed authority when necessary. He goes with his team into new learning and new understanding, and re-directs the team if the creative drive for experimentation starts to occlude the imperatives of delivering material product.
Doubtlessly, it's sensationalistic to divvy up the software development management landscape into extremes, but doing so is meant to frame the issue so that we can begin to recognize a real problem and begin to do something about it.
The way that software development is practiced at large, it's perfectly understandable for software development workers to become burned out on software development, and to look for a change. Unfortunately, moving upward to software development management isn't where people who are burned out on this work should go. It's little more than an escape, and it needs to be more of a deeper commitment to the work rather than a means to justify avoiding the details.
Ultimately, software development work will remain palatable for much longer when it is managed much better. But we are where we are, and the general population of software development managers will remain skewed heavily toward refugees from the work of software development for some time.
This is the software management crisis: software development managers at large aren't experts in software development work now, and they often weren't expert workers in the days when they were hands-on. They were the people who came to dislike the work enough to want to escape to software development management. They usually don't know enough about the details of the work to understand how subtle decisions can result in tremendous outcomes, and which decisions lead to tremendously good outcomes, or tremendously poor outcomes.
Today's software development managers largely haven't made a study of managing software development and the principles of product development work and work management. Software continues to be managed as one would manage material assembly or manufacturing. Our current generation of managers isn't aware of this problem because software development management hasn't really been a field of interest for most of them - it's just the job that they have, and it's one that keeps them from having to work in the painful niggling details of poorly-managed software.
None of this suggests that we need to wipe the halls of software development management clean and start from scratch with newly-installed personnel lifted from the rank-and-file. One of the worst mistakes that our current management ideology continues to make is the failure to recognize the material cost of losing institutional knowledge when we lose any worker (NOTE: the flip side of this problem is the colloquial horror of not knowing who to hire, and bringing people into our organizations that we dismiss in short order).
The software development mangers that we have today may have become burned out on the details of software development, but those details can be changed to make the work less tiresome, and make the work less like an uphill battle being fought on a cliff face rather than a gentle slope. Their institutional knowledge is far too valuable to lose. Their many years of experience - if paired with a sharper understanding of which decisions make software development good and which make it bad - provides a perspective of the whole of the effort that is invaluable.
It's entirely reasonable to expect that we can transform the software development industry. The root of the problem is in the software development management system. When software development managers are willing to look without flinching into the reality of their situation and the circumstances that have led to their positions, then we can begin to rehabilitate the system. When that happens, many of the software development industry's most futile pathologies will fall by the wayside and we'll begin to tap the potential that is this industry's unclaimed birthright.
Working with software developers and organizations to help realize the potential of software product development through higher productivity, higher quality, and improved customer experience
Learn more about my work and how I can help you at ampgt.com
