Wednesday, February 03, 2010

To Control and Observe - Productive Software Development

There isn't a productive software development effort where controlling and observing aren't the substance of productivity. If you break down productivity to it's most fundamental element, you'll find that the understanding of how your software works is the essence of it all. Control and observe your software and you understand how it works. When you understand it, you can change it, work with it, and continue working with it. When it gets harder to make observations of how your software works, it becomes harder to remain motivated to do the right thing. In the extreme, we live out software projects in survival mode, and we saddle someone else with the responsibility of observing whether our software works. It's not the only way. It's the worst way.

The more that you can make observations about your software, and the more that those observations can be made in thought-sized chunks, the more easily it is to understand your software. To make an observation about your software, you have to be able to control that bit of software to do the thing who's side effects you want to observe. The easier it is to do that, the easier it is to understand it, and the more productivity you have. The time that you spend deciphering software that should be readily understandable is mostly wasted time. It's time spent that shouldn't be necessary and that can be eliminated.

Software that is hard to understand is software that is hard to control. Software that is hard to control is hard to understand. A small bit of code that you want to observe should be controlled with as little code as possible. If you can run that small bit of code directly without having to also control adjacent code, then you've got code that is inherently easy to control, and a level of productivity that is inherently greater. We tend to overlook the amount of time that software teams spend controlling and observing, and when things start to become difficult, we tell ourselves that we can't afford to pay attention to it. Ironically, paying attention to it solves the root problem.

Here are a couple of examples of code that is easy to control:
- In-memory objects that can be observed without loading their data from the database
- User interface workflow that can be observed without using the user interface
- Business logic events that can be observed without the auditing software making recordings
- Steps of complex business processes that can be observed without starting the process from scratch
- Secure operations that can be observed without having to authenticate
- Payment logic that can be observed without connecting to a payment processor

If I'm concerned with the functioning of in-memory objects, and if I need to load those objects from a database, then I have to add the database to the things I need to control, and to be concerned with as well. This isn't a natural, defacto quality of software, regardless of how common it is.

We generally accept that the lack of control as a defacto quality of software projects. We overestimate the control that we have, and we underestimate the value of increased control. We don't pay it much attention. We believe that that the limits to the control that we have is just part of the very fabric of software development, that it's unchangeable, and that there's no reason to dwell on it. We may not even have an awareness of it, or of the issue. This is the fatal flaw in software development thinking, because it's the issue of control that explains why software projects rapidly loose productivity and why the extent of that productivity loss is so great.

Maintaining the ability to control and observe is a conscious, intentional effort. If you're not on top of it all the time, it slips away. The traditional productivity loss curve and the traditional cost curve subsequently begin to immediately express themselves. You can tell yourself it's just the way it is, or you can realize the root cause of the problem and the solution and not only regain that lost productivity, but then go on to achieve a level of productivity that you wouldn't have believed possible from your previous perspective.

This issue isn't unknown. In fact it's well-known. The most common mistake in production management is not paying attention to how productivity loss happens and focus instead on increasing the efficiency of the work that is done amidst the accelerating losses in effectiveness. This mistake is the basis for most software production management at this point of industrial software development.

Next: Controlled Productivity

Ampersand GT

Working with software developers and organizations to help realize the potential of software product development through higher productivity, higher quality, and improved customer experience

Learn more about my work and how I can help you at