Relearning: The Productivity Problem that We're Not Supposed To Talk About

Imagine that you had no memory; that everything you learned had to be re-learned again and again as you did your work. If you worked in software development, you wouldn't have to stretch too far to imagine it. Re-learning is so much a part of the moment by moment work of software development that it's considered normal. In fact, it's the unrecognized backdrop that software development plays out in front of.

Because relearning is not recognized as a problem as software development, it's almost never talked about it. And frankly, it's not a welcome subject in polite programmer society. Nonetheless, relearning accounts for a lot of unnecessary lost productivity and waste in software development. It can account for wide swings in productivity loss on most software teams, affecting the manageability of software projects and a truthful and meaningful assessment of programmers' real abilities as designers, analysts, problem solvers, and real contributors.

I'll offer an observation from my own experience and from the anecdotal evidence of others in my professional circle: I would be comfortable saying that half of the lost productivity on software teams comes from re-learning. It can come in the form of poor testability - the inability to easily provide proofs that expectations are met - and it can come in the form of source code that can't be understood at a glance. Even teams who have solved the testability problem still largely suffer from the lack of usability of their code, and the inevitable mass of relearning that comes of it.

First We Scan, and then We Read

Text in a text editor is interactive media; subject to the same fundamental usability principles that apply to a web page, a desktop app, or even a billboard on the side of the highway.

Before users read the content of interactive media they scan the content. Programmers scan
code before they read it. This is the singular human behavior that programmers consistently fail to recognize. It's the fundamental behavior that when recognized, becomes one of the pillars that code usability efforts can be built upon, and the starting point for recouping losses from relearning.

A singular focus on readability in code is a concern that, while encouraging, still misses the point. It's the same point that programmers missed as a human-computer interaction industry grew from the ashes of our continued failed attempts to create productive user experiences.

Usable code dissolves into understanding at a glance. It doesn't need to be coerced into understanding. I like to call this kind of code "soluble" code for the image it brings to mind of program text readily dissolving into awareness and understanding. Soluble code is likely readable code, but readable code isn't soluble unless it's written to be soluble.

Reading code isn't like reading a good article, where you start at the top and read to the bottom, enjoying the experience and being fulfilled by it. Granted there is beautiful code in the world, but the typical reasons for reading code are not the reasons that we read articles, books, papers, and the like.

In fact, the nature of the media that this article is published in, and the context that this media is typically consumed in, is such that you are more than likely to start jumping around the text with your eyes and with your mouse, looking for the nuggets and pearls while avoiding having to consume the entire thing linearly as it is written.

The first thing we do with code is ascertain whether it is indeed the code that we need to be working with in order to accomplish whatever task we've taken on. This even happens when we're reading code for the pleasure of it. The first eyes-on experience with some code usually involves rapid scanning of the code to ascertain if we're in the right place; if
we're at the worksite. Much of this happens pre-cognitively as it does with any media that we've been called to act upon. First we scan to get our bearings, and then we read.

We take high-level structural scans, followed by smaller, more detailed scans, followed by reading. If the code that we're scanning is written so that it only yields its meaning from a detailed read, then not only are we not optimizing for the natural human interaction patterns, we're also not taking advantage of providing the incidental knowledge that can be yielded to someone while they're scanning. This incidental knowledge accumulates and becomes a significant part of the material invested in the metal map of a codebase that a programmer accumulates through exposure to code that yields its meaning at a glance. We retain a codebase's textual geography only when we absorb its meaning.

If we don't code for solubility, we force programmers to take detailed reads in order to tease knowledge and understanding from the text. Forcing these detailed reads doesn't lead to greater advantage down the road. Less meaning is retained when forcing detailed reading into a context where a programmer is instinctively trying to scan.

Scanning is as much about a process of elimination as it is a process of accumulation. Both are happening at once during scanning unless code isn't amenable to these processes. If we fail to take advantage of scanning by failing to write soluble code, we're stealing productivity from our team mates, and from ourselves. On the surface, this seems like a negligible issue, but lack of solubility is responsible for a tremendous amount relearning and degraded productivity.

Solubility as a code style permeates a codebase. It's a pervasive quality; it has a constant, pervasive effect. Small improvements add up to significant advances when they have a pervasive effect.

Making Code Soluble

There are no cookie cutter patterns for making code soluble. Some things are obvious: meaningful symbol names (class names, method names, variable names, etc), and higher-level methods that tell the story of some process that call lower-level methods that contain the details. Both of these go a long way to create soluble code. Soluble code yields its meaning immediately.

The particular refinements that a team will make depends on the team, the product being built, the technology, and a host of other conditions. Pushing further for an canonical definition of soluble code patterns might lead to just as much productivity loss as productivity gain. Further practices, beyond meaningful names and anecdotal methods, are contextual and shouldn't be dropped into the code as if they are interchangeable parts.

Soluble code is an experience like the table of contents in a novel. It offers and allows multiple levels of reading, with each deeper level yielding ever greater detail. A table of contents is an affordance to the reader that acts as a navigational aid, or map, of the text. But a novel isn't program code. A table of contents at the beginning of a novel can be sufficient for that kind of media with its implicit user experiences, whereas program code itself must be it's own table of contents, right down to the very small, five-line, composable methods. The user of a novel is typically well-served by a table of contents whose resolution goes no further than mapping out chapters, expecting the reader to start reading linearly from the beginning of the chapter. But this isn't the case with program code.

When we're scanning, we're mapping the code by the what of the code; what the code does, what it's responsibilities and behaviors are. Soluble code allows a reader to immediately understand what is does before forcing the reader to understand how it does it. Soluble code serves both modes: scanning for what, and reading the how. Code that isn't styled this way largely deprives a user of the process of elimination, the incidental knowledge, and the mapping that can be had from scanning.

One of the worst mistakes that programmers make in writing code is in failing to recognize that more productivity will be spent over the life time of code navigating through the code than will be spent writing the code. It doesn't take much more effort to write soluble code that serves the elimination of relearning. Choosing to not write soluble code means choosing to keep the relearning waste well-entrenched, but there's more to this problem than mere choice.

Resistance to Code Usability

There are reasonable objections to styling code for solubility and usability. The how
of the system ends up spread over many small methods rather than concentrating it in fewer, larger locations. The root cause of the aggravation is often not that the code is factored in
small semantic units, but that the semantic units are not the right ones, or that the factoring is just not good. Yes, this is the you're not doing it right response, and as unfashionable is this response is, it can nonetheless end up being the root cause. Nonetheless, some programmers are just not going to want to get used to soluble code style, and there will be inevitable grumblings from people who prefer to use more traditional, procedural structure. It's not hard to bring usability to code, but it can be discomforting at first - like transitioning to a new programming language.

Programmers aren't traditionally the folks on a software team who have their heads in the usability game. And we've gotten to an unfortunate point in programmer culture where the answer to many subsequent problems with programmer ineffectiveness has been to create further specializations and allow programmers to be responsible for a narrower and narrower set of expectations rather than deal with knowledge problems as organizational and cultural problems.

Expecting programmers to be considerate of code usability creates friction. For many programmers it's going to be as comfortable as thawing out frozen, front-bitten fingers. But it's not just a programmer responsibility. A good chunk of the responsibility for change rests squarely with the surrounding and supporting organization and its protocols and mechanics. There's more to rehabilitating software development productivity than introducing programmers to new coding pattens. Organizations that have a commitment to learning cultures will do much better at this.

The staunchest resistance to efforts to reclaim lost productivity due to relearning will come from hero programmers. Hero programmers are those guys in any organization that can get the job done with any code in any state. They are typically blessed with what seems like a supernaturally high-definition mental map of a codebase. This is their best and worst quality. It's their best quality because they often know where to fix a problem in a codebase and have a reasonable grasp of the myriad side effects that might result. It can be their worst quality because it's often an effect of mild Asperger Syndrome common to programmers, engineers, and jobs that require extended, intense focus. It's often accompanied by the lack of awareness and empathy toward peers typical to the condition.

Hero programmers can suboptimize the efforts of a team by not having to rely on soluble code, often navigating a codebase entirely from memory. The ability can be incredibly useful, but the need to write soluble code rarely manifests because it's not a personal need, and the typical lack of empathy obstructs the ability to recognize how this advantage undermines their team mates' efforts to be as effective.

The code created by hero programmers isn't soluble because the heroes rely on uncommon facilities that preclude the need for solubility. They don't notice the design smells because they rely almost exclusively on echo location for navigation. The resulting work product can often only be as effectively worked on by the heroes themselves, which inevitably leads to predictable resource bottlenecks, bus factor risks, excessive specialization, and the general malaise on a team as the effects of code toxicity spill over into the human realm.

The analogy to the baseball shortstop who was famous for making great plays applies: his coach often pointed out that he was out of position to begin with.

Usability is rooted in an ability to have empathy for users. That empathy gives designers the pause to consider whether they got the user experience right. It's the empathy that leads to the questioning that leads to the recognition of interaction design problems in the form of cognitive obstacles that undermine the user's productivity. In an environment with traditional and institutionalized lack of awareness and lack of empathy, dysfunction can drive unconscionable waste.

Hero programmers rarely stop to question whether they've left behind a good experience for others on the team who need to navigate, then understand, then make changes to the code. And most programmers will fail to recognize the two distinct mindsets that are in play when working with code.

Writer's Mind and Reader's Mind

Without ever doubting whether code is usable, programmers will presume that the right thing has been done. Programmers who are good at creating soluble code have learned to doubt every line of code written. It's not that they have all of the answers. More importantly, they've learned to be constantly questioning.

In the words of the anthropologist Claude Levi-Strauss (not the guy who invented blue jeans), "The scientific mind does not so much provide the the right answers as ask the right questions."

Asking the right questions means constantly switching from the writer's mind to the reader's mind. That means that after each bit of code is written, a programmer switches mental contexts and assesses the code from the perspective of someone who has never seen the code before, asking, "What have I done to undermine the immediacy of someone else's understanding of my work? What unrecognized presumptions have I made about their context as a reader that only applies to my context as a writer?"

That's quite a trick, and it's not uncommon to hear the complaint that it can't be done, but it's what interaction designers do all the time. It's not uncommon for the deleterious effects of programmer autism to cause programmers to fail to have awareness sufficient enough to break out of the laser-like focus on writing code and switch back into questioning.

Code that doesn't incur the cost of relearning is code that can be immediately understood by someone who hasn't seen it before with minimal orientation to the code and the problems it solves. It's code that can be understood at a glance. That code is rarely if ever produced by a mind that has lost its awareness of its context and its mode. It isn't produced by a mind that fails to concede that code is written to be read, that the readers are other humans, and that the reader's context and needs are not the context of the writer - at lest not until the reader has found the worksite in the code, and gained sufficient understanding of the worksite to begin to make the necessary changes that they're tasked with.

The writer's mind is a context that often fails to recognize that the focused and relatively linear mechanics of writing code is quite different than the mechanics of consuming code as a reader. And this is where the misconception over readability comes from. The writer's mind, working relatively linearly is also consuming code in that mode. Readability is a quality that pertains to the linear consumption of code as text, and that kind of optimization of experience is only relevant some of the time in some user scenarios.

Break the Habit

The autistic mind is lulled by the hypnotic cadence of constantly pumping out code. It will complain that the constant switching between the writer's perspective and the reader's perspective is ruining their ability to get in the zone. And in truth, it is, but it's that particular zone that is causing a lot of relearning debt to mount up. There are other zones to get into with equally pleasing effects, but it is a matter of breaking some habits and replacing them with new ones. There are a few tricks and techniques that programmers can use to break out of the fog and get into the zone.

The most important thing to practice is the constant self-questioning of whether the code just created is soluble; that it can be understood at a glance and yields enough meaning while scanning to contribute to the mental map. Instead of presuming that everything I do is made of gold, I presume instead that it's made of fools gold. From that perspective, I can usually gain the right amount of objectivity to assess solubility.

Pair programming and test-driven development are two techniques from Extreme Programming that are extremely effective at clearing the fog. When these techniques are practiced together, it becomes very difficult to be lulled into the unexamined mind that produces unexamined code.

If you don't like pair programming, try using some kind of timer on an interval that is just short enough to be uncomfortable that will remind you to come back up for some critical thinking.

And lastly (for this article anyway; there are more tricks out there), Context Specification is a form of Test-Driven Development that recognizes solubility, usability, reader's mind, and authorship, and forces the issue of contextual analysis to bring more practice of awareness into programming.

Commit to Learning

Ultimately, there's no where left to hide. The anti-productivity that comes from inviting and accepting relearning continues to accumulate. We've pushed it into the lowest level of software development where only programmers can (or may) see it, but it still affects everyone touched by the software project or the product.

Relearning waste is a fundamental organizational behavior. Until we shape organizations around dealing with waste and learning, we continue to fail to see in the range of the spectrum where the sheer magnitude of the relearning waste is visible. Relearning is inculcated into software organizations. Shifting from re-learning organizations to learning organizations and learning culture is how this problem is ultimately solved.

It's self-evident: counter-act relearning with learning. The term learning organization isn't the trite and trivial perspectives that see learning as something external to team and organization; a destination where people are sent once in a while to be "trained". Learning is no more about receiving training than quality assurance is about giving click recorders to test monkeys. The emphasis we put on "training" in the software industry undermines our ability to see the "learning" side of the same issue, and to build truly meaningful teaching/learning experiences in our organizations, and to subordinate organizational mechanics and protocols to these imperatives to counteract the constant, tireless forces that drag us back into relearning.

The value we throw away on relearning is recouped when we counter it with meaningful learning. Learning gets real when it's expressed in every organizational protocol and business process, and importantly for software development, when it is expressed in every single line of code written by everyone involved in bringing a solution to life.